Artificial intelligence-based fracture line detection and fracture type identification method and system

Abstract

The application relates to the technical field of medical image processing, and discloses a fracture line detection and fracture type identification method and system based on artificial intelligence. The method comprises the following steps: acquiring a bone image and normalizing; inputting a cross-stage local dense connection network to extract multi-scale trabecular bone textures by means of recursive gate convolution and splicing the multi-scale trabecular bone textures into a multi-level feature map; feeding into a double-path collaborative attention module to cooperatively strengthen fracture region features in the channel and space; inputting an adaptive deformation perception classification network to dynamically sample the irregular trend of a fracture line by means of deformable convolution, and outputting an end-to-end fracture type label and a pixel-level fracture line segmentation mask, so that the detection and typing precision is improved.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, specifically to a method and system for fracture line detection and fracture type identification based on artificial intelligence. Background Technology

[0002] Fractures are a common type of bone trauma in clinical practice, and accurate detection of fracture lines and identification of fracture types are crucial for treatment planning. Currently, clinical practice mainly relies on radiologists to interpret X-ray or CT images visually. However, fracture lines often appear as narrow, low-density bands, which are easily affected by trabecular bone texture, bone sclerosis, and image noise. Irregular fractures such as impacted and spiral fractures are more likely to be missed.

[0003] Existing deep learning-based auxiliary detection methods mostly employ conventional convolutional neural networks. Regular sampling grids are difficult to adapt to the irregular orientation and misaligned intercalation of fracture lines. Furthermore, multi-scale texture extraction of bone trabeculae is insufficient, and a single attention mechanism is difficult to simultaneously suppress background response and enhance subtle fracture lines. Most solutions only output segmentation masks or perform classification, failing to provide pixel-level fracture lines and fracture types end-to-end, thus limiting their clinical application. Summary of the Invention

[0004] To address the aforementioned technical shortcomings, the purpose of this invention is to provide a method and system for fracture line detection and fracture type identification based on artificial intelligence, thereby solving the problems of fracture line detection relying on manual labor, low accuracy, and the inability to simultaneously perform classification and segmentation.

[0005] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: In a first aspect, the present invention provides a method for fracture line detection and fracture type identification based on artificial intelligence, including: Step S1: Obtain medical imaging data of the bone region to be detected and perform normalization processing to obtain standardized image data; Step S2: Input the standardized image data into a cross-stage local dense connection network, extract the trabecular texture features at different scales through recursive gated convolution, and stitch them together along the channel dimension to form a multi-level feature map; Step S3: Input the multi-level feature map into the dual-path collaborative attention module. The first path filters the channel response values ​​related to the fracture line through channel attention, and the second path generates the boundary weight map of the fracture area through spatial attention. The outputs of the two paths are multiplied element-wise and then connected with the original feature residual to obtain the enhanced feature representation. Step S4: Input the enhanced feature representation into the adaptive deformation-aware classification network. The adaptive deformation-aware classification network uses deformable convolution to dynamically sample and model the irregular direction of the fracture line and outputs the fracture type label and the pixel-level segmentation mask of the fracture line.

[0006] Preferably, in one possible implementation of the first aspect, the normalization process in step S1 includes: The medical imaging data of the bone area to be detected is used to perform coarse localization of the bone region and extract the region of interest containing the target bone. The pixel grayscale values ​​within the region of interest are mapped to a preset grayscale range. An affine transformation is performed on the region of interest according to the anatomical dimensions of the target skeletal part, and standardized image data with uniform size is output.

[0007] Preferably, in one possible implementation of the first aspect, the cross-stage local dense connection network in step S2 is composed of several sequentially connected cross-stage local dense blocks and transition layers stacked alternately. Each cross-stage local dense block contains several densely connected sub-layers, and the sub-layers achieve local dense reuse through feature splicing in the channel dimension. The local dense block has cross-stage splitting branches at the beginning and end, splitting the input features into two paths along the channel. One path enters the dense connection sub-layer, and the other path skips the dense connection sub-layer and directly concatenates it with the sub-layer output in the channel dimension. The transition layer compresses and downsamples the channels of the spliced ​​features to form a multi-level scale-decreasing feature map sequence; The multi-scale feature maps output by each cross-stage local dense block are spliced ​​along the channel dimension after channel alignment to form a multi-level feature map.

[0008] Preferably, in one possible implementation of the first aspect, the recursive gated convolution is performed recursively within each densely connected sublayer; A single gated convolution uses the current input features and the features of the previous recursive output as the basis for generating the gate signal; The output of the previous recursion in the first recursion is initialized with the input features of this sublayer; The gating signal is modulated channel by channel with the current convolutional kernel weights and applied to the input features, outputting the intermediate features of this recursion; The output at the end of the recursion is concatenated along the channel with the jump input of the densely connected sublayer.

[0009] Preferably, in one possible implementation of the first aspect, the structure of the dual-path collaborative attention module in step S3 includes: Multi-level feature maps are input into the first path and the second path respectively; The first path connects the global pooling layer, the fully connected layer, and the activation function layer in sequence, outputting a channel-dimensional weight vector; The second path connects several convolutional layers and activation function layers in sequence, and outputs a spatial weight map with the same spatial size as the input feature map; The outputs of the two paths are multiplied by the multi-level feature map through channel-wise multiplication and spatial position-wise multiplication, respectively. The multiplication results are then multiplied element-wise and added element-wise to the original multi-level feature map to obtain the enhanced feature representation.

[0010] Preferably, in one possible implementation of the first aspect, in the first path, the global pooling layer performs global average pooling on each channel of the multi-level feature map; The fully connected layer maps the pooled channel vectors to a low dimension and then back to the original number of channels, inserting non-linear activations in between. The final layer outputs a channel weight vector with values ​​between 0 and 1 via a sigmoid function. The channel weight vector is multiplied by the multi-level feature maps channel by channel along the channel direction to output a weighted channel feature map.

[0011] Preferably, in one possible implementation of the first aspect, in the second path, the multi-level feature map first extracts multiple receptive field boundary cues through convolutional layers with progressively increasing dilatation rates, then compresses the number of channels into a single channel through 1 to 3 convolutional layers, and outputs a spatial weight map through the Sigmoid function. The spatial weight map is multiplied spatially with the multi-level feature map, and the product is multiplied element-wise with the output of the first path. The result of the multiplication is added element-wise with the multi-level feature map to form the residual, and the output is the enhanced feature representation.

[0012] Preferably, in one possible implementation of the first aspect, the adaptive deformation-sensing classification network in step S4 consists of an encoder and a decoder; The encoder receives an enhanced feature representation, wherein at least two convolutional layers are deformable convolutional layers, and the deformable convolutional layers contain an offset prediction sub-layer, which predicts the two-dimensional spatial offset based on the enhanced feature representation. The deformable convolution kernel performs irregular offsets on the sampling grid based on the offset, and dynamically samples the direction, misalignment, and intercalation of the fracture line. The decoder performs upsampling and skip-level splicing on the output of each level of the encoder, and the end branches into a classification head and a segmentation head.

[0013] Preferably, in one possible implementation of the first aspect, the offset prediction sub-layer consists of one convolutional layer, with the input being the enhanced feature representation of the current layer, and the number of output channels being twice the number of sampling points of the deformable convolutional kernel, corresponding to the offset of each sampling point in the height and width directions, respectively. The offset is normalized and superimposed on the regular grid coordinates to obtain the deformation sampling coordinates. The deformable convolution is performed at the deformation sampling coordinates, and the corresponding feature values ​​are extracted from the input feature map of the current layer through bilinear interpolation. The classification head performs global pooling on the features of the last layer of the decoder and then outputs the fracture type label through a fully connected layer; The segmentation head convolves the features of the last layer of the decoder pixel by pixel to output a binary segmentation mask of the same size as the input image. The foreground pixels in the mask constitute pixel-level annotations of the fracture lines.

[0014] Secondly, the present invention provides an artificial intelligence-based fracture line detection and fracture type identification system, the system being used to implement the artificial intelligence-based fracture line detection and fracture type identification method as described in the first aspect, comprising: The image acquisition module acquires medical image data of the bone region to be detected and performs normalization processing to obtain standardized image data. The feature extraction module inputs standardized image data into a cross-stage local dense connection network, extracts trabecular texture features at different scales through recursive gated convolution, and stitches them together along the channel dimension into a multi-level feature map. The collaborative attention module inputs multi-level feature maps into the dual-path collaborative attention module. The first path filters the channel response values ​​related to the fracture line through channel attention, and the second path generates the boundary weight map of the fracture area through spatial attention. The outputs of the two paths are multiplied element-wise and then connected with the original feature residual to obtain the enhanced feature representation. The classification and segmentation module inputs the enhanced feature representation into the adaptive deformation-aware classification network. The adaptive deformation-aware classification network uses deformable convolution to dynamically sample and model the irregular direction of the fracture line and outputs the fracture type label and the pixel-level segmentation mask of the fracture line.

[0015] The beneficial effects of this invention are: by normalizing the ROI of the skeleton and affine it to a uniform anatomical size, the scale and grayscale differences between different parts and devices are eliminated.

[0016] By employing a cross-stage locally densely connected network in conjunction with recursive gated convolution, the convolution weights are modulated channel by channel with gate signals while densely reusing local features, thus fully extracting multi-scale trabecular texture and reducing redundant computation.

[0017] The dual-path collaborative attention method filters fracture-related channels through the channel path and captures multi-receptive field boundary cues through spatial convolution with increasing void rate. The two outputs are then connected by element-wise multiplication and residual connection, which significantly enhances the response in the fracture area and suppresses interference from normal bone.

[0018] The adaptive deformation-aware classification network uses deformable convolution to dynamically sample the fracture line direction, misalignment, and embedding based on the predicted offset. The encoder-decoder ends split into a classification head and a segmentation head, and outputs fracture type labels and pixel-level fracture line masks end-to-end, significantly improving detection and classification accuracy. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A flowchart of an artificial intelligence-based fracture line detection and fracture type identification method is provided for this application. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Example 1: As Figure 1 As shown, this invention provides an artificial intelligence-based method for fracture line detection and fracture type identification, including: Step S1: Obtain medical image data of the bone region to be detected and perform normalization processing to obtain standardized image data.

[0023] In this embodiment, medical image data of the bone region to be detected is acquired, which is a CT scan image or an X-ray image. The normalization process includes: firstly, coarsely locating the bone region in the medical image data of the bone region to be detected; secondly, using a connected component analysis method based on grayscale threshold, setting the grayscale threshold range to 150 Henle units to 400 Henle units, extracting the connected components of the bone region; and thirdly, using the bounding rectangle of the connected component as the center, expanding outward by 20 pixels to extract the region of interest containing the target bone.

[0024] Then, the pixel grayscale values ​​within the region of interest are mapped to a preset grayscale range, which is set to 0 to 255. The mapping formula is as follows: ,in The original grayscale value. The minimum grayscale value within the region of interest. The maximum grayscale value within the region of interest is such that the pixel grayscale values ​​within the region of interest are linearly stretched to the integer range of 0 to 255.

[0025] Finally, an affine transformation was performed on the region of interest according to the anatomical dimensions of the target bone. For the femur, the target anatomical dimensions were set to 512 pixels × 512 pixels; for the tibia, the target anatomical dimensions were set to 256 pixels × 256 pixels; and for the wrist bones, the target anatomical dimensions were set to 384 pixels × 384 pixels.

[0026] The affine transformation uses a bicubic interpolation algorithm to calculate the floating-point coordinates of the output pixel in the original image, and uses the gray values ​​of the surrounding 16 pixels for weighted averaging to output standardized image data with uniform size.

[0027] Step S2: Input the standardized image data into a cross-stage local dense connection network, extract the trabecular texture features at different scales through recursive gated convolution, and stitch them together along the channel dimension to form a multi-level feature map.

[0028] In this embodiment, the cross-stage local dense connection network consists of four sequentially connected cross-stage local dense blocks and three transition layers stacked alternately. Each cross-stage local dense block contains five densely connected sub-layers, and local dense reuse is achieved between sub-layers through feature concatenation at the channel dimension. An initial convolutional layer is set at the network entry point, with a kernel size of 3×3 pixels, a stride of 1, padding of 1, and an output channel count of 64. This layer maps the standardized image data output in step S1 from 3 channels to 64 channels, thereby providing 64 channels of input features for subsequent cross-stage segmentation branches.

[0029] The first cross-stage local dense block has an input space size of 512×512 pixels and 64 channels. Each densely connected sublayer contains a set of convolutional operations with a kernel size of 3×3 pixels, a stride of 1, padding of 1, and 32 output channels. Batch normalization and ReLU activation functions are used within the densely connected sublayers to enhance training stability and non-linear expressive power.

[0030] The local dense block has cross-stage segmentation branches at both ends, splitting the input features into two paths along the channels. Taking the first cross-stage local dense block as an example, its input feature has 64 channels. The cross-stage segmentation branch splits the channels into two paths, each with 32 channels. The first path enters five densely connected sub-layers, each outputting a 32-channel feature map. Features are accumulated between sub-layers through channel concatenation, resulting in a final output of 160 channels. The second path skips the densely connected sub-layers and directly concatenates the output of the sub-layers along the channel dimension, resulting in a total of 192 channels. The transition layer compresses and downsamples the concatenated features. The transition layer uses a 1×1 pixel convolutional kernel to compress the number of channels to 128, and then uses a 2×2 pixel max pooling layer to halve the spatial size, forming the input for the next level. The second cross-stage local dense block has an input space size of 256×256 pixels and 128 channels. Its cross-stage splitting branch divides the channels into two paths of 64 and 64. After five densely connected sub-layers, the sub-layer output has 160 channels. After concatenation with the 64 channels skipped from the path, the total number of channels is 224. The transition layer compresses to 128 channels and downsamples to 128×128 pixels. The third cross-stage local dense block has an input space size of 128×128 pixels and 128 channels. Its cross-stage splitting branch divides it into two paths of 64 and 64. The sub-layer output has 160 channels, and after concatenation, the total number of channels is 224. The transition layer compresses to 128 channels and downsamples to 64×64 pixels. The fourth cross-stage local dense block has an input space size of 64×64 pixels and 128 channels. The cross-stage segmentation branch is split into two paths of 64 and 64, and the sub-layer output has 160 channels. After concatenation, the total number of channels is 224. There is no transition layer after this block, and the multi-scale feature map is directly output.

[0031] The multi-scale feature maps output from each cross-stage local dense block are concatenated along the channel dimension after channel alignment and spatial upsampling to form a multi-level feature map. The first cross-stage local dense block output feature map has a spatial size of 512×512 pixels and 192 channels; the second output has a spatial size of 256×256 pixels and 224 channels; the third output has a spatial size of 128×128 pixels and 224 channels; and the fourth output has a spatial size of 64×64 pixels and 224 channels. For each output feature map, a 1×1 pixel convolution kernel is first used to uniformly adjust the number of channels to 192. Then, the second, third, and fourth feature maps are upsampled by 2x, 4x, and 8x bilinear interpolation, respectively, to make their spatial size all 512×512 pixels. Finally, all four feature maps are concatenated along the channel dimension to obtain a multi-level feature map with a total of 768 channels and a spatial size of 512×512 pixels.

[0032] Recursive gated convolution is executed recursively within each densely connected sub-layer; in this embodiment, the recursion is performed twice. Each gated convolution uses the current input features and the features from the previous recursive output as the basis for generating the gate signal. The previous recursive output from the first recursion is initialized as the input features of that sub-layer. Taking the first densely connected sub-layer as an example, its input feature channels are 32, and its spatial size is 512×512 pixels. During the first recursion, the gate signal is generated by fusing the current input features and the initialized recursive output features element-wise, then processed by a convolution kernel generated independently of the main convolution kernel. This convolution kernel has a size of 3×3 pixels, 32 output channels, a stride of 1, and padding of 1. The gate signal value is then normalized to the 0-1 range using the Sigmoid function. The gate signal and the weights of the feature extraction convolution kernel are modulated channel-wise. The feature extraction convolution kernel has a size of 3×3 pixels, 32 output channels, a stride of 1, and padding of 1. The modulation process is as follows: the gate signal tensor and the feature extraction convolution kernel weights are multiplied channel by channel in the channel dimension to obtain the modulated convolution kernel weights. Then, the modulated weights are used to perform convolution operations on the current input features to output the intermediate features of this recursion. The number of channels is 32.

[0033] In the second recursion, the gating signal is fused element-wise with the intermediate features output from the first recursion, and then a new gating signal is generated by a convolutional kernel produced by the same gating signal. This gating signal is then modulated channel-wise with the weights of the feature extraction convolutional kernel. The modulated convolutional kernel convolves the current input features, outputting the intermediate features of the second recursion. The output at the end of the recursion is this intermediate feature of the second recursion, which is concatenated along the channels with the skip input of the densely connected sublayer. The skip input is the original input feature of the sublayer before recursion. The number of output channels after concatenation is 64. Each subsequent densely connected sublayer is executed using this recursive gating convolution method to ensure that the trabecular texture features related to the fracture line are fully extracted at different scales.

[0034] Step S3: Input the multi-level feature map into the dual-path collaborative attention module. The first path filters the channel response values ​​related to the fracture line through channel attention, and the second path generates the boundary weight map of the fracture area through spatial attention. The outputs of the two paths are multiplied element-wise and then connected with the original feature residual to obtain the enhanced feature representation.

[0035] In this embodiment, the dual-path collaborative attention module receives multi-level feature maps with a spatial size of 512 pixels × 512 pixels and 768 channels. The multi-level feature maps are input to the first path and the second path, respectively. The first path sequentially connects a global pooling layer, a fully connected layer, and an activation function layer, outputting a channel-dimensional weight vector. The global pooling layer performs global average pooling on each channel of the multi-level feature maps. The global average pooling operation sums the pixel values ​​at 512 × 512 spatial locations in each channel and divides by 262144 to obtain a one-dimensional channel vector of length 768. Each scalar value in this vector represents the global response intensity of the corresponding channel.

[0036] The fully connected layer maps the pooled channel vectors to a lower dimension and then back to the original number of channels, inserting non-linear activations in between. Specifically, the first fully connected layer compresses the 768-dimensional channel vector to 48 dimensions, a compression ratio of 1 / 16. The weight matrix of this fully connected layer has a size of 48 rows × 768 columns, the bias vector length is 48, and the output is processed by the ReLU activation function to obtain a 48-dimensional low-dimensional feature vector.

[0037] The second fully connected layer restores the 48-dimensional low-dimensional feature vector to 768 dimensions. The weight matrix of this fully connected layer has a size of 768 rows × 48 columns, and the bias vector length is 768. The output is processed by the Sigmoid function, the formula for which is: , The input values ​​are used as input values ​​to the function, and each output value is limited to a closed interval between 0 and 1. The final output is a channel weight vector, which has a length of 768. Each element takes a value between 0 and 1, representing the importance of the corresponding channel for fracture line detection. The channel weight vector is multiplied channel by channel along the channel direction of the multi-level feature map, that is, the nth element in the channel weight vector is multiplied. The element and the first multi-level feature map The pixel values ​​of all 512×512 spatial locations within each channel are multiplied one by one to output a weighted channel feature map. The spatial size of the weighted channel feature map is 512 pixels × 512 pixels, and the number of channels is 768.

[0038] The second path sequentially connects several convolutional layers and activation function layers, outputting a spatial weight map with the same spatial size as the input feature map. The multi-level feature map is first processed by convolutional layers with progressively increasing dilation rates to extract multi-receptive field boundary cues. Specifically, the first convolutional layer has a kernel size of 3 pixels × 3 pixels, a dilation rate of 1, padding of 1, 256 output channels, and a stride of 1. This convolutional layer performs standard convolution operations on the input multi-level feature map, extracting texture and edge information within the local neighborhood, and outputting a first intermediate feature map with a spatial size of 512 pixels × 512 pixels and 256 channels.

[0039] The second convolutional layer has a kernel size of 3 pixels × 3 pixels, a dilation rate of 2, padding of 2, an output channel count of 128, and a stride of 1. This convolutional layer expands the receptive field based on the first intermediate feature map, enabling the convolutional kernel to cover a larger spatial range, capture the contextual structure around the fracture line, and outputs a second intermediate feature map with a spatial size of 512 pixels × 512 pixels and 128 channels.

[0040] The third convolutional layer has a kernel size of 3 pixels × 3 pixels, a dilation rate of 3, padding of 3, an output channel count of 64, and a stride of 1. This convolutional layer further increases the dilation rate to obtain spatial distribution clues of trabecular bone fractures and misalignments over a wider range, and outputs a third intermediate feature map with a spatial size of 512 pixels × 512 pixels and 64 channels.

[0041] Each convolutional layer is followed by a batch normalization layer and a ReLU activation function layer. The batch normalization layer standardizes the feature map of each channel by subtracting the mean of that channel and dividing by the standard deviation. The ReLU activation function sets all negative values ​​to zero and keeps positive values ​​unchanged.

[0042] After three convolutional layers with increasing dilatation rates, a further convolutional layer compresses the number of channels to a single channel. This convolutional layer has a kernel size of 1 pixel × 1 pixel, outputs 1 channel, has a stride of 1, and zero padding. The 64-channel third intermediate feature map is fused into a single-channel response map with a spatial size of 512 pixels × 512 pixels. The value at each spatial location reflects the confidence that the location belongs to the boundary of the fracture region. This single-channel response map is processed by a sigmoid function, mapping each pixel value to a range of 0 to 1, outputting a spatial weight map. The spatial weight map has a spatial size of 512 pixels × 512 pixels and 1 channel. Each pixel value in the spatial weight map represents the importance weight of that spatial location for fracture line detection.

[0043] The spatial weight map is multiplied spatially with the multi-level feature map, that is, the coordinates in the spatial weight map are... The pixel values ​​and coordinates in all 768 channels of the multi-level feature map are The pixel values ​​are multiplied one by one to obtain a weighted spatial feature map, which has a spatial size of 512 pixels × 512 pixels and 768 channels. The weighted spatial feature map is then multiplied element-wise with the weighted channel feature map output from the first path. That is, at each spatial location and each channel, the pixel values ​​at the same position and in the same channel of the two feature maps are multiplied together to obtain a collaborative attention feature map, which also has a spatial size of 512 pixels × 512 pixels and 768 channels. Finally, the collaborative attention feature map is added element-wise with the original multi-level feature map to form a residual connection. That is, the pixel value at each spatial location and in each channel of the collaborative attention feature map is added to the pixel value at the same location and in the same channel of the original multi-level feature map, outputting an enhanced feature representation. This enhanced feature representation has a spatial size of 512 pixels × 512 pixels and 768 channels.

[0044] Step S4: Input the enhanced feature representation into the adaptive deformation-aware classification network. The adaptive deformation-aware classification network uses deformable convolution to dynamically sample and model the irregular direction of the fracture line and outputs the fracture type label and the pixel-level segmentation mask of the fracture line.

[0045] In this embodiment, the adaptive deformation-aware classification network consists of an encoder and a decoder. The encoder receives the enhanced feature representation output from step S3, which has a spatial size of 512 pixels × 512 pixels and 768 channels. The encoder consists of four stacked convolutional layers, where the third and fourth convolutional layers are deformable convolutional layers. The first convolutional layer is a standard convolutional layer with a kernel size of 3 pixels × 3 pixels, a stride of 2, padding of 1, and 384 output channels. This layer performs preliminary feature extraction on the enhanced feature representation and downsamples the spatial size to 256 pixels × 256 pixels.

[0046] The second convolutional layer is a standard convolutional layer with a kernel size of 3 pixels × 3 pixels, a stride of 2, padding of 1, and 192 output channels. This layer downsamples the feature map space to 128 pixels × 128 pixels. The third convolutional layer is a deformable convolutional layer with a kernel size of 3 pixels × 3 pixels, a stride of 2, padding of 1, and 96 output channels. This layer includes an offset prediction sublayer.

[0047] The offset prediction sub-layer consists of a single convolutional layer. The input is the feature map from the third convolutional layer, which has a spatial size of 128 pixels × 128 pixels and 192 channels. The kernel size of the offset prediction sub-layer is 3 pixels × 3 pixels, with a stride of 2 and padding of 1. The number of output channels is 18, which is equal to the number of sampling points of the deformable convolutional kernel (9) multiplied by 2. These correspond to the two-dimensional spatial offset of each sampling point in the height and width directions, respectively.

[0048] The offset prediction sublayer outputs an offset tensor with a space size of 64 pixels × 64 pixels and 18 channels. The offsets are normalized and then superimposed onto a regular grid coordinate system to obtain the deformation sampling coordinates. The regular grid coordinate system consists of a 3-pixel × 3-pixel grid with the kernel center as the origin and a spacing of 1. The set of grid point coordinates is as follows: .

[0049] The normalization process divides each offset value in the offset tensor by the spatial size of the current layer's feature map. For the third convolutional layer, the dimensions in both the height and width directions are 128 pixels, so each offset value is divided by 128 to obtain the normalized offset. The normalized offset is then added point-by-point to the regular grid coordinates to obtain the deformation sampling coordinates.

[0050] Deformable convolution extracts the corresponding feature values ​​from the input feature map of the current layer at the deformable sampling coordinates using bilinear interpolation. Bilinear interpolation calculates the weighted average of the four nearest neighbor pixels around the deformable sampling coordinates, with the weights determined by the distances from the deformable sampling coordinates to the four nearest neighbor pixels.

[0051] The deformable convolutional kernel performs irregular offsets on the sampling grid based on the offset, dynamically sampling the direction, misalignment, and intercalation patterns of the fracture lines. The output feature map has a spatial size of 64 pixels × 64 pixels and 96 channels. The fourth convolutional layer is a deformable convolutional layer with a kernel size of 3 pixels × 3 pixels, a stride of 2, padding of 1, and 48 output channels. This layer includes an offset prediction sublayer.

[0052] The offset prediction sublayer consists of a single convolutional layer, taking the input feature map from the fourth convolutional layer. This input feature map has a spatial size of 64 pixels × 64 pixels and 96 channels. The kernel size of the offset prediction sublayer is 3 pixels × 3 pixels, with a stride of 2 and padding of 1, and the output has 18 channels. After normalization, the offsets are superimposed onto a regular grid coordinate system to obtain deformation sampling coordinates. The normalization process divides each offset value by the spatial size of the current layer's feature map, which is 64.

[0053] Deformable convolution extracts feature values ​​at the deformation sampling coordinates using bilinear interpolation, outputting a feature map with a spatial size of 32 pixels × 32 pixels and 48 channels. The feature maps output by each level of the encoder are as follows: the first convolutional layer has a spatial size of 256 pixels × 256 pixels and 384 channels; the second convolutional layer has a spatial size of 128 pixels × 128 pixels and 192 channels; the third convolutional layer has a spatial size of 64 pixels × 64 pixels and 96 channels; and the fourth convolutional layer has a spatial size of 32 pixels × 32 pixels and 48 channels. The decoder performs progressive upsampling and layer skipping concatenation on the outputs of each encoder level.

[0054] The decoder consists of four upsampling layers. The first upsampling layer performs a 2x bilinear interpolation upsampling on the feature map output from the fourth convolutional layer, resulting in a feature map with a spatial size of 64 pixels × 64 pixels and 48 channels. This upsampled feature map is then concatenated with the feature map output from the third convolutional layer along the channel dimension, resulting in a feature map with a spatial size of 64 pixels × 64 pixels and 144 channels. Subsequently, a standard convolutional layer is passed through the layer with a kernel size of 3 pixels × 3 pixels, a stride of 1, padding of 1, and an output channel count of 96. This layer fuses the concatenated features.

[0055] The second upsampling layer performs a 2x bilinear interpolation upsampling on the feature map output by the first upsampling layer. After upsampling, the feature map has a spatial size of 128 pixels × 128 pixels and 96 channels. This upsampled feature map is then concatenated with the feature map output by the second convolutional layer along the channel dimension. The concatenated feature map has a spatial size of 128 pixels × 128 pixels and 288 channels. Subsequently, a standard convolutional layer is passed through it with a kernel size of 3 pixels × 3 pixels, a stride of 1, padding of 1, and an output channel count of 192. This layer fuses the concatenated features.

[0056] The third upsampling layer performs a 2x bilinear interpolation upsampling on the feature map output by the second upsampling layer. After upsampling, the feature map has a spatial size of 256 pixels × 256 pixels and 192 channels. This upsampled feature map is then concatenated with the feature map output by the first convolutional layer along the channel dimension. The concatenated feature map has a spatial size of 256 pixels × 256 pixels and 576 channels. Subsequently, it passes through a standard convolutional layer with a kernel size of 3 pixels × 3 pixels, a stride of 1, and padding of 1, outputting 384 channels. This layer then fuses the concatenated features.

[0057] The fourth upsampling layer performs a 2x bilinear interpolation upsampling on the feature map output from the third upsampling layer. After upsampling, the feature map has a spatial size of 512 pixels × 512 pixels and 384 channels. This feature map is the final feature map of the decoder. The decoder ends by branching into a classification head and a segmentation head. The classification head performs global average pooling on the final feature map of the decoder. Global average pooling sums the pixel values ​​at each spatial location of 512 pixels × 512 pixels in each channel and divides by 262144 to obtain a one-dimensional feature vector of length 384. This one-dimensional feature vector is input to the fully connected layer. The weight matrix of the fully connected layer has a size of 10 rows and 384 columns, the bias vector has a length of 10, and the output is a vector of length 10. Each element in this vector corresponds to a score for a fracture type. The fracture types include no fracture, simple fracture, wedge fracture, complex fracture, comminuted fracture, transverse fracture, oblique fracture, spiral fracture, impacted fracture, and compression fracture, totaling ten types.

[0058] Applying the Softmax function to this vector, the formula for calculating the Softmax function is as follows: ,in For the first Scores for each category For the first The scores for each category are converted into probability values, and the fracture type corresponding to the category with the highest probability value is the output fracture type label. The segmentation head performs pixel-wise convolution on the feature map of the last layer of the decoder. The convolution kernel size of this convolutional layer is 1 pixel × 1 pixel, the stride is 1, the padding is 0, and the number of output channels is 1. This compresses the 384-channel feature map into a single-channel response map with a spatial size of 512 pixels × 512 pixels.

[0059] Apply the Sigmoid function to this single-channel response plot. The formula for calculating the Sigmoid function is as follows: Each pixel value is mapped to a range of 0 to 1 to obtain a binary segmentation mask of the same size as the input image. Each pixel value in the mask represents the probability that the pixel belongs to the fracture line region. A threshold of 0.5 is set, and pixels with a value greater than or equal to 0.5 are marked as foreground, while pixels with a value less than 0.5 are marked as background. Foreground pixels constitute pixel-level annotations of the fracture lines, and a pixel-level segmentation mask of the fracture lines is output.

[0060] Example 2: This invention provides a model training process for an artificial intelligence-based fracture line detection and fracture type identification system.

[0061] In this embodiment, the training dataset consists of 5000 clinically collected CT scan images of skeletal sites, covering three anatomical locations: the femur, tibia, and wrist. Each sample includes the original CT image and gold standard labels independently annotated by two senior radiologists. The gold standard labels include fracture type labels and a pixel-level segmentation mask for the fracture line. The fracture type labels are divided into ten categories: no fracture, simple fracture, wedge fracture, complex fracture, comminuted fracture, transverse fracture, oblique fracture, spiral fracture, impacted fracture, and compression fracture. The pixel-level segmentation mask for the fracture line is a binary image of the same size as the CT image, with a foreground pixel value of 1 and a background pixel value of 0. For samples with inconsistent annotations by the two physicians, a third attending physician reviews and determines the final label. All samples are randomly divided into a training set, a validation set, and a test set in an 8:1:1 ratio. The training set contains 4000 samples, the validation set contains 500 samples, and the test set contains 500 samples.

[0062] Before model training, data preprocessing and data augmentation operations were performed on all samples in the training set. The data preprocessing process first involved coarsely locating the bone region in the CT images using a connected component analysis method based on a gray-level threshold. The gray-level threshold range was set to 150 to 400 HJ units to extract the connected components of the bone region. The region of interest (ROI) containing the target bone was then extracted by extending 20 pixels outward from the bounding rectangle of the ROI. Then, the pixel gray-level values ​​within the ROI were linearly mapped to the integer range of 0 to 255 using the following formula: ,in The original grayscale value. The minimum grayscale value within the region of interest. The maximum grayscale value within the region of interest is defined as [value]. Finally, an affine transformation is performed on the region of interest according to the anatomical dimensions of the target bone parts. The anatomical dimensions of the femur are set to 512 pixels × 512 pixels, the tibia to 256 pixels × 256 pixels, and the wrist bones to 384 pixels × 384 pixels. The affine transformation uses a bicubic interpolation algorithm to output standardized image data.

[0063] Data augmentation is performed online after preprocessing. In each iteration, six augmentation methods are randomly applied to the standardized image data: horizontal flip, vertical flip, clockwise rotation, counter-clockwise rotation, brightness adjustment, and contrast adjustment. The probability of horizontal and vertical flip is set to 0.5. The angles of clockwise and counter-clockwise rotation are... Uniformly and randomly selected within the range. The brightness adjustment factor is... Uniformly and randomly selected within the range, the contrast adjustment factor is in Uniformly and randomly selected within the range. The fracture type label in the gold standard label remains unchanged, and the pixel-level segmentation mask of the fracture line performs the same geometric transformation synchronously with the image to ensure that the label is consistent with the spatial position of the enhanced image.

[0064] The loss function for model training consists of two parts: classification loss and segmentation loss. The classification loss uses a weighted cross-entropy loss function to measure the difference between the fracture type probability distribution output by the classification head and the true fracture type label. The formula for calculating the weighted cross-entropy loss function is as follows: ,in The total number of fracture types is set to 10. For the first The true label indicator variable for fracture type, if the sample belongs to the first Class otherwise , The first output of the classification head Predicted probability of fracture type For the first Weighting coefficients for fracture types. These weighting coefficients are set based on the reciprocal normalized value of the number of samples in each category in the training set. Specifically, the calculation method involves first counting the number of samples for each fracture type in the training set. Then calculate the weight coefficient for each class. ,in The total number of samples in the training set is set to 4000.

[0065] The segmentation loss is a weighted sum of the Dice loss function and the binary cross-entropy loss function, used to measure the difference between the binary segmentation mask output by the segmentation head and the pixel-level segmentation mask of the actual fracture lines. The formula for calculating the Dice loss function is as follows: ,in and The height and width of the image are set to 512, respectively. Output the coordinates in the mask for the segmentation head The predicted probability value at that location. Coordinates in the real mask The label value at each point is either 0 or 1, with 1 added to both the numerator and denominator for smoothing and to prevent division by zero. The formula for calculating the binary cross-entropy loss function is as follows: .

[0066] Total loss function .

[0067] Model optimization uses the AdamW optimizer, with an initial learning rate set to... The weight decay coefficient is set to 0.01, and the momentum parameter... Set to 0.9, Set to 0.999. Set as The learning rate is scheduled using a cosine annealing strategy, with a total of 200 epochs. Each epoch iterates through the entire training set once. The warm-up phase is set for the first 5 epochs, where the learning rate linearly increases from 0 to the initial learning rate. Then, following the cosine annealing curve from Decrease to 0. Batch size is set to 8, meaning 8 samples are processed simultaneously in each iteration. Gradient clipping threshold is set to 1.0 to prevent gradient explosion.

[0068] The model weights were initialized using the Kaiming normal distribution initialization method, with the bias term initialized to 0. During training, model performance was evaluated on the validation set every epoch. Evaluation metrics included fracture type classification accuracy and the Dice coefficient for fracture line segmentation. Classification accuracy was calculated as the percentage of correctly classified samples out of the total number of samples in the validation set.

[0069] The formula for calculating the Dice coefficient is as follows: ,in To predict the set of foreground pixels for the mask, This is the set of foreground pixels for the real mask. The model weights with the highest segmentation Dice coefficient on the validation set are saved as the final model. After training, the final model's performance is evaluated on the test set, recording six metrics: classification accuracy, classification precision, classification recall, classification F1 score, segmentation Dice coefficient, and segmentation intersection-union ratio (IoU). The formula for calculating classification precision is... ,in For the actual number of cases, The number of false positives; the formula for calculating classification recall is: ,in The number of false negatives; the formula for calculating the categorical F1 score is: ,in To improve classification accuracy, For classification recall; the formula for calculating the segmentation intersection-union ratio (IoU) is: ,in To predict the set of foreground pixels for the mask, This is the set of foreground pixels of the real mask.

[0070] Example 3: This invention provides an artificial intelligence-based fracture line detection and fracture type identification system, comprising: The image acquisition module acquires medical image data of the bone region to be detected and performs normalization processing to obtain standardized image data. The feature extraction module inputs standardized image data into a cross-stage local dense connection network, extracts trabecular texture features at different scales through recursive gated convolution, and stitches them together along the channel dimension into a multi-level feature map. The collaborative attention module inputs multi-level feature maps into the dual-path collaborative attention module. The first path filters the channel response values ​​related to the fracture line through channel attention, and the second path generates the boundary weight map of the fracture area through spatial attention. The outputs of the two paths are multiplied element-wise and then connected with the original feature residual to obtain the enhanced feature representation. The classification and segmentation module inputs the enhanced feature representation into the adaptive deformation-aware classification network. The adaptive deformation-aware classification network uses deformable convolution to dynamically sample and model the irregular direction of the fracture line and outputs the fracture type label and the pixel-level segmentation mask of the fracture line.

[0071] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for fracture line detection and fracture type identification based on artificial intelligence, characterized in that, include: Step S1: Obtain medical imaging data of the bone region to be detected and perform normalization processing to obtain standardized image data; Step S2: Input the standardized image data into a cross-stage local dense connection network, extract the trabecular texture features at different scales through recursive gated convolution, and stitch them together along the channel dimension to form a multi-level feature map; Step S3: Input the multi-level feature map into the dual-path collaborative attention module. The first path filters the channel response values ​​related to the fracture line through channel attention, and the second path generates the boundary weight map of the fracture area through spatial attention. The outputs of the two paths are multiplied element-wise and then connected with the original feature residual to obtain the enhanced feature representation. Step S4: Input the enhanced feature representation into the adaptive deformation-aware classification network. The adaptive deformation-aware classification network uses deformable convolution to dynamically sample and model the irregular direction of the fracture line and outputs the fracture type label and the pixel-level segmentation mask of the fracture line.

2. The method for fracture line detection and fracture type identification based on artificial intelligence as described in claim 1, characterized in that, The normalization process in step S1 includes: The medical imaging data of the bone site to be detected is used to perform coarse localization of the bone region and extract the region of interest containing the target bone. The pixel grayscale values ​​within the region of interest are mapped to a preset grayscale range. An affine transformation is performed on the region of interest according to the anatomical dimensions of the target skeletal part, and standardized image data with uniform size is output.

3. The method for fracture line detection and fracture type identification based on artificial intelligence as described in claim 1, characterized in that, In step S2, the cross-stage local dense connection network is composed of several sequentially connected cross-stage local dense blocks and transition layers stacked alternately. Each cross-stage local dense block contains several densely connected sub-layers, and the sub-layers achieve local dense reuse through feature splicing in the channel dimension. The local dense block has cross-stage splitting branches at the beginning and end, splitting the input features into two paths along the channel. One path enters the dense connection sub-layer, and the other path skips the dense connection sub-layer and directly concatenates it with the sub-layer output in the channel dimension. The transition layer compresses and downsamples the channels of the spliced ​​features to form a multi-level scale-decreasing feature map sequence; The multi-scale feature maps output by each cross-stage local dense block are spliced ​​along the channel dimension after channel alignment to form a multi-level feature map.

4. The method for fracture line detection and fracture type identification based on artificial intelligence as described in claim 3, characterized in that, The recursive gated convolution is executed recursively within each densely connected sublayer; A single gated convolution uses the current input features and the features of the previous recursive output as the basis for generating the gate signal; The output of the previous recursion in the first recursion is initialized with the input features of this sublayer; The gating signal is modulated channel by channel with the current convolutional kernel weights and applied to the input features, outputting the intermediate features of this recursion; The output at the end of the recursion is concatenated along the channel with the jump input of the densely connected sublayer.

5. The method for fracture line detection and fracture type identification based on artificial intelligence as described in claim 1, characterized in that, The structure of the dual-path collaborative attention module in step S3 includes: Multi-level feature maps are input into the first path and the second path respectively; The first path connects the global pooling layer, the fully connected layer, and the activation function layer in sequence, outputting a channel-dimensional weight vector; The second path connects several convolutional layers and activation function layers in sequence, and outputs a spatial weight map with the same spatial size as the input feature map; The outputs of the two paths are multiplied by the multi-level feature map through channel-wise multiplication and spatial position-wise multiplication, respectively. The multiplication results are then multiplied element-wise and added element-wise to the original multi-level feature map to obtain the enhanced feature representation.

6. The method for fracture line detection and fracture type identification based on artificial intelligence as described in claim 5, characterized in that, In the first path, the global pooling layer performs global average pooling on each channel of the multi-level feature map; The fully connected layer maps the pooled channel vectors to a low dimension and then back to the original number of channels, inserting non-linear activations in between. The final layer outputs a channel weight vector with values ​​between 0 and 1 via a sigmoid function. The channel weight vector is multiplied by the multi-level feature maps channel by channel along the channel direction to output a weighted channel feature map.

7. The artificial intelligence-based fracture line detection and fracture type identification method as described in claim 6, characterized in that, In the second path, the multi-level feature map first extracts multi-receptive field boundary cues through convolutional layers with progressively increasing dilatation rates, then compresses the number of channels into a single channel through 1 to 3 convolutional layers, and outputs a spatial weight map through the Sigmoid function. The spatial weight map is multiplied spatially with the multi-level feature map, and the product is multiplied element-wise with the output of the first path. The result of the multiplication is added element-wise with the multi-level feature map to form the residual, and the output is the enhanced feature representation.

8. The method for fracture line detection and fracture type identification based on artificial intelligence as described in claim 1, characterized in that, The adaptive deformation sensing classification network in step S4 consists of an encoder and a decoder. The encoder receives an enhanced feature representation, wherein at least two convolutional layers are deformable convolutional layers, and the deformable convolutional layers contain an offset prediction sub-layer, which predicts the two-dimensional spatial offset based on the enhanced feature representation. The deformable convolution kernel performs irregular offsets on the sampling grid based on the offset, and dynamically samples the direction, misalignment, and intercalation of the fracture line. The decoder performs upsampling and skip-level splicing on the output of each level of the encoder, and the end branches into a classification head and a segmentation head.

9. The method for fracture line detection and fracture type identification based on artificial intelligence as described in claim 8, characterized in that, The offset prediction sub-layer consists of one convolutional layer. The input is the enhanced feature representation of the current layer, and the number of output channels is twice the number of sampling points of the deformable convolutional kernel, which correspond to the offset of each sampling point in the height and width directions, respectively. The offset is normalized and superimposed on the regular grid coordinates to obtain the deformation sampling coordinates. The deformable convolution is performed at the deformation sampling coordinates, and the corresponding feature values ​​are extracted from the input feature map of the current layer through bilinear interpolation. The classification head performs global pooling on the features of the last layer of the decoder and then outputs the fracture type label through a fully connected layer; The segmentation head convolves the features of the last layer of the decoder pixel by pixel to output a binary segmentation mask of the same size as the input image. The foreground pixels in the mask constitute pixel-level annotations of the fracture lines.

10. A fracture line detection and fracture type identification system based on artificial intelligence, characterized in that, The system is used to implement the artificial intelligence-based fracture line detection and fracture type identification method as described in any one of claims 1 to 9, including: The image acquisition module acquires medical image data of the bone region to be detected and performs normalization processing to obtain standardized image data. The feature extraction module inputs standardized image data into a cross-stage local dense connection network, extracts trabecular texture features at different scales through recursive gated convolution, and stitches them together along the channel dimension into a multi-level feature map. The collaborative attention module inputs multi-level feature maps into the dual-path collaborative attention module. The first path filters the channel response values ​​related to the fracture line through channel attention, and the second path generates the boundary weight map of the fracture area through spatial attention. The outputs of the two paths are multiplied element-wise and then connected with the original feature residual to obtain the enhanced feature representation. The classification and segmentation module inputs the enhanced feature representation into the adaptive deformation-aware classification network. The adaptive deformation-aware classification network uses deformable convolution to dynamically sample and model the irregular direction of the fracture line and outputs the fracture type label and the pixel-level segmentation mask of the fracture line.

Images