A method, system, device, and medium for bone healing timing assessment

Abstract

The application discloses a bone healing timing evaluation method, system, device and medium, comprising the following steps: acquiring a CT image data set at multiple time points through an image preprocessing and data enhancement method; then using a convolutional neural network to extract features of the CT image at each time point to obtain spatial features of bone healing; then inputting the extracted features into an LSTM network to capture the timing changes between different periods; and finally outputting an evaluation result of the bone healing state through a fully connected layer. The application can overcome the limitations of the prior art based on single-point image analysis, provide more accurate and reliable bone healing evaluation, and provide important decision support for clinicians.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to a method, system, device and medium for assessing bone healing time. Background Technology

[0002] Following a fracture, the bone healing process is a complex physiological process that typically requires assessment using medical imaging (such as CT scans or X-rays). Currently, clinical assessment largely relies on physicians' experience and visual analysis to determine the degree of bone healing. However, this assessment method is highly subjective and susceptible to human error. With the development of medical image processing and deep learning technologies, computer-aided bone healing assessment methods offer clinicians a more objective and accurate tool.

[0003] Most existing studies assess bone healing based on single-time-point medical imaging data, neglecting the importance of time-series information during fracture healing. However, the fracture healing process is significantly time-dependent; callus formation and the degree of blurring of the fracture line change over time. Image analysis based on a single time point cannot accurately assess the dynamic changes in bone healing.

[0004] Therefore, it is urgent to solve the above problems. Summary of the Invention

[0005] Purpose of the invention: The first objective of this invention is to provide a method for assessing the timing of bone healing, which can accurately assess the bone healing status by comparing CT image data from multiple time periods and combining the timing information, thus overcoming the problem of inaccurate assessment caused by single-time-point analysis in the prior art.

[0006] A second objective of this invention is to provide a method system for assessing bone healing time.

[0007] A third objective of this invention is to provide an electronic device.

[0008] A fourth objective of this invention is to provide a computer-readable storage medium.

[0009] Technical solution: To achieve the above objectives, this invention discloses a method for assessing bone healing timing, comprising the following steps:

[0010] (1) Obtain several original CT image data at different times after fracture and preprocess them to obtain a multi-time point CT image dataset; label the degree of bone healing at each time point, where 0 represents nonunion, 1 represents bone healing, and 2 represents nonunion, forming a label file; divide the multi-time point CT image dataset into a training dataset and a validation dataset, and combine them with the label file.

[0011] (2) Construct a CNN-FPN-LSTM network, which includes a ResNet50 convolutional neural network, an FPN feature pyramid network, an average pooling layer, at least one LSTM layer, and a fully connected layer.

[0012] The training dataset obtained in step (1) is imported into the CNN-FPN-LSTM network. The CT images at multiple time points are stacked into an input tensor of shape (B,T,H,W,C) to convert the CT images into 2D slices. Here, B is the batch size, which is initially set to 1. T is the number of time steps, which represents the number of CT images at different time points. H is the height of the CT image. W is the width of the CT image. C is the number of channels, which is the single channel corresponding to grayscale.

[0013] 2D slices are input into a ResNet50 convolutional neural network for feature extraction, and multiple feature maps of different scales are output, including the maximum resolution feature map P2, high resolution feature map P3, medium resolution feature map P4 and low resolution feature map P5.

[0014] Multiple feature maps of different scales are input into the FPN feature pyramid network for fusion to obtain the fused feature map;

[0015] The fused feature map is input into an average pooling layer, which outputs a fixed-length feature vector X = [x1, x2, ..., x...] at different time points. T The feature vector extracted at each time point is input into the LSTM layer to capture the temporal relationship between features at different time points, and the temporal features in the hidden state of the last time step are output for classification.

[0016] The temporal features output by the LSTM layer are transformed into an evaluation result of the bone healing status through a fully connected layer and a softmax activation function, and the current bone healing status is output. Assuming that the bone healing status is divided into three categories, the CNN-FPN-LSTM network outputs a three-class classification, which are bone non-union 0, bone healing 1, and bone non-union 2. The predicted probability distribution is calculated based on the three-class classification results.

[0017] Repeat the training process multiple times until the evaluation accuracy of the CNN-FPN-LSTM network on the validation dataset converges. Check the evaluation performance of the CNN-FPN-LSTM network and save the trained CNN-FPN-LSTM network.

[0018] (3) Obtain the original CT image data after the fracture to be evaluated, perform preprocessing to obtain the preprocessed CT image data, and use the CNN-FPN-LSTM network trained in step S2 to evaluate the preprocessed CT image data to obtain the current bone healing status.

[0019] Optionally, the preprocessing in step (1) refers to image normalization, region of interest (ROI) cropping, and data enhancement preprocessing. This involves converting the original CT image data DICOM file into a 3D or 2D array format, normalizing the grayscale values ​​of the CT image, or converting the HU values ​​to values ​​within the standard CT range [-1000, 4000]. Based on the fracture area, CT slices containing the fracture site are cropped, redundant background information is removed, and random rotation, translation, scaling, and noise addition operations are performed.

[0020] Optionally, the formula for the Softmax activation function in step (2) is:

[0021] y pred =Softmax(FC(LSTM(X)))

[0022] Where: y pred Let X = [x1, x2, ..., xn] be the probability distribution to be predicted. T ], LSTM(X), is the result of the LSTM layer aggregating the temporal features of the feature vector; FC is a fully connected layer used for classification tasks.

[0023] Optionally, in step (3), image normalization, region of interest (ROI) cropping, and data enhancement preprocessing are performed. The DICOM file of the original CT image data is converted into a 3D or 2D array format, and the gray values ​​of the CT image are normalized or the HU values ​​are converted into values ​​within the standard CT range [-1000, 4000]. CT slices containing the fracture site are cropped according to the fracture area, redundant background information is removed, and random rotation, translation, scaling, and noise addition operations are performed.

[0024] Based on the same inventive concept, this invention discloses a bone healing timing assessment system, comprising:

[0025] The data preprocessing unit is used to acquire raw CT image data at different times after fracture and preprocess it to obtain a multi-time point CT image dataset; label the degree of bone healing at each time point, where 0 represents nonunion, 1 represents bone healing, and 2 represents nonunion, forming a label file; divide the multi-time point CT image dataset into a training dataset and a validation dataset, and combine them with the label file.

[0026] The model training unit is used to construct the CNN-FPN-LSTM network, which includes a ResNet50 convolutional neural network, an FPN feature pyramid network, an average pooling layer, at least one LSTM layer, and a fully connected layer.

[0027] The acquired training dataset is imported into the CNN-FPN-LSTM network. CT images from multiple time points are stacked into an input tensor of shape (B,T,H,W,C) to convert the CT images into 2D slices. Here, B is the batch size, initially set to 1; T is the number of time steps, representing the number of CT images at different time points; H is the height of the CT image; W is the width of the CT image; and C is the number of channels, where C is the single channel corresponding to grayscale.

[0028] 2D slices are input into a ResNet50 convolutional neural network for feature extraction, and multiple feature maps of different scales are output, including the maximum resolution feature map P2, high resolution feature map P3, medium resolution feature map P4 and low resolution feature map P5.

[0029] Multiple feature maps of different scales are input into the FPN feature pyramid network for fusion to obtain the fused feature map;

[0030] The fused feature map is input into an average pooling layer, which outputs a fixed-length feature vector X = [x1, x2, ..., x...] at different time points. T The feature vector extracted at each time point is input into the LSTM layer to capture the temporal relationship between features at different time points, and the temporal features in the hidden state of the last time step are output for classification.

[0031] The temporal features output by the LSTM layer are transformed into an evaluation result of the bone healing status through a fully connected layer and a softmax activation function, and the current bone healing status is output. Assuming that the bone healing status is divided into three categories, the CNN-FPN-LSTM network outputs a three-class classification, which are bone non-union 0, bone healing 1, and bone non-union 2. The predicted probability distribution is calculated based on the three-class classification results.

[0032] Repeat the training process multiple times until the evaluation accuracy of the CNN-FPN-LSTM network on the validation dataset converges. Check the evaluation performance of the CNN-FPN-LSTM network and save the trained CNN-FPN-LSTM network.

[0033] Bone healing assessment unit: used to acquire the original CT image data after the fracture to be assessed, perform preprocessing to obtain preprocessed CT image data, and use the CNN-FPN-LSTM network trained in step S2 to evaluate the preprocessed CT image data to obtain the current bone healing status.

[0034] Optionally, the preprocessing in the data preprocessing unit refers to image normalization, region of interest (ROI) cropping, and data enhancement preprocessing. This involves converting the original CT image data DICOM file into a 3D or 2D array format, normalizing the grayscale values ​​of the CT images, or converting the HU values ​​to values ​​within the standard CT range [-1000, 4000]. CT slices containing the fracture site are cropped based on the fracture area, redundant background information is removed, and random rotation, translation, scaling, and noise addition operations are performed.

[0035] Optionally, the formula for the Softmax activation function in the model training unit is:

[0036] y pred =Softmax(FC(LSTM(X)))

[0037] Where: y pred Let X = [x1, x2, ..., xn] be the probability distribution to be predicted. T ], LSTM(X), is the result of the LSTM layer aggregating the temporal features of the feature vector; FC is a fully connected layer used for classification tasks.

[0038] Optionally, the bone healing assessment unit performs image normalization, region of interest (ROI) cropping, and data enhancement preprocessing. It converts the DICOM file of the original CT image data into a 3D or 2D array format, normalizes the grayscale values ​​of the CT images, or converts the HU values ​​to values ​​within the standard CT range [-1000, 4000]. It also crops CT slices containing the fracture site based on the fracture area, removes redundant background information, and performs random rotation, translation, scaling, and noise addition operations.

[0039] Based on the same inventive concept, this invention discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a bone healing timing assessment method as described above.

[0040] Based on the same inventive concept, the present invention discloses a computer-readable storage medium having a computer program stored thereon, characterized in that the computer program is executed by a processor to implement a bone healing timing assessment method as described above.

[0041] Beneficial Effects: Compared with existing technologies, this invention has the following significant advantages: This invention, through multi-layer convolutional feature extraction using CNN networks, can capture important detailed features such as fracture lines and callus at different scales, providing more accurate spatial features; this invention utilizes LSTM networks to effectively process time-dependent information during fracture healing, compensating for the shortcomings of traditional single-time-point analysis; combining the advantages of CNN convolutional neural networks and LSTM, it can simultaneously consider spatial features and temporal information, significantly improving the accuracy of bone healing assessment; this invention overcomes the limitations of existing methods based on single-time-point data analysis, effectively capturing dynamic changes during the bone healing process; this invention improves the generalization ability and application performance of the assessment method through data augmentation and multi-scale feature extraction steps. Attached Figure Description

[0042] Figure 1 This is a partial structural diagram of the CNN-FPN-LSTM network in this invention. Figure 1 ;

[0043] Figure 2 This is a partial structural diagram of the CNN-FPN-LSTM network in this invention. Figure 2 . Detailed Implementation

[0044] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0045] like Figure 1 and Figure 2 As shown, the present invention provides a method for assessing bone healing timing, comprising the following steps:

[0046] (1) Obtain raw CT images of several beagle dogs at different times after fracture, or obtain multi-time point CT images from patients or animal experiments. Perform image normalization, region of interest (ROI) cropping and data enhancement preprocessing, and then label the images to obtain a multi-time point CT image dataset of several beagle dogs. Use medical image processing tools to convert the DICOM file of the raw CT image data into 3D or 2D array format, and normalize the gray values ​​of the CT images or convert the HU values ​​to values ​​within the standard CT range [-1000, 4000]. Crop CT slices containing the fracture sites are cropped according to the fracture area, remove redundant background information, and perform random rotation, translation, scaling and noise addition operations to increase the diversity of the data. Label the degree of bone healing at each time point, where 0 represents nonunion, 1 represents healing, and 2 represents nonunion. The label file is stored in text format. Divide the multi-time point CT image dataset into training dataset and validation dataset, and combine them with the label file.

[0047] (2) Construct a CNN-FPN-LSTM network. The CNN-FPN-LSTM network model includes a ResNet50 convolutional neural network, an FPN feature pyramid network, an average pooling layer, at least one LSTM layer, and a fully connected layer.

[0048] The acquired training dataset is imported into the CNN-FPN-LSTM network. CT images from multiple time points are stacked into an input tensor of shape (B, T, H, W, C), where B is the batch size (initially set to 1), T is the time step (representing the number of CT images at different time points), H is the height of the CT image, W is the width of the CT image, and C is the number of channels (C represents a single channel corresponding to grayscale). The CT images are converted into 2D slices, with each slice normalized to 512×512 pixels. If the CT image is 3D, it is cut into multiple 2D slices.

[0049] 2D slices are input into a ResNet50 convolutional neural network for feature extraction, outputting multiple feature maps at different scales. These feature maps include a maximum resolution feature map P2, a high-resolution feature map P3, a medium-resolution feature map P4, and a low-resolution feature map P5. The maximum resolution feature map P2 is the largest, providing the most detailed spatial information, especially effective in extracting the degree of blurring of the fracture line at the detail level. The high-resolution feature map P3 is a relatively high-resolution feature map, capturing local features of the fracture area with fine granularity, such as the callus formation process. The medium-resolution feature map P4 is a medium-resolution feature map, mainly used to extract medium-scale features. The low-resolution feature map P5 is a smaller feature map, containing global semantic information, suitable for extracting the overall trend of fracture healing. These multiple feature maps at different scales collectively describe the features of the bone structure at different levels and scales. The feature maps contain information crucial to the bone healing process, such as the degree of blurring of the fracture, callus growth, and possible healing status. This invention uses a convolutional neural network (CNN) to extract features from CT images at each time point, obtaining the depth features of the bone structure at that time point, such as the degree of blurring of the fracture line and the status of callus formation.

[0050] Multiple feature maps at different scales are input into the FPN feature pyramid network for fusion to obtain a fused feature map. The FPN feature pyramid network is used to extract multi-scale features to help the model capture bone features at different sizes, especially the details of callus and fracture lines. The FPN is used to combine the multi-level features output by the convolutional neural network (CNN) to help the model better capture fracture details at different sizes in bone CT images.

[0051] The fused feature map is input into a global average pooling layer, which transforms and outputs a fixed-length feature vector X = [x1, x2, ..., x...].T ], x T ∈R n x T ∈R n This is the n-dimensional feature vector extracted by the CNN at time point T; its fixed length is 512, and the size of the feature vector is (B, T, 512). Global average pooling integrates the spatial features to generate a fixed-length feature vector, which can be used as input to the LSTM layer for subsequent classification or regression tasks. The advantage of global average pooling is that by averaging the feature maps of each channel, it avoids the introduction of a large number of parameters in fully connected layers, reducing model complexity and computational overhead, making it particularly suitable for processing high-resolution inputs such as medical image data. Assuming that after the FPN convolution operation, the output feature map size is H×W×C, where H is the height of the feature map, W is the width of the feature map, and C is the number of channels in the feature map, with each channel corresponding to one feature map; global average pooling reduces the H×W feature map of each channel to a scalar value, which is the average of all elements in the feature map of that channel.

[0052] The feature vector extracted at each time point is input into an LSTM layer to capture the temporal relationship between features at different time points, learn the pattern of feature changes during bone healing, and output the temporal feature h in the hidden state (B,h) of the last time step. T Used for classification, h = 256, where h is the number of hidden layer units in the LSTM network; feature vector h T This represents the overall bone healing status of the beagle at the current point in time, h T This is a fixed-length vector containing temporal dependency information and spatial feature information captured by the LSTM layer. This invention inputs the spatial features extracted at each time point into a Long Short-Term Memory (LSTM) network. The LSTM can capture the temporal relationships between features at different time points and learn the patterns of feature changes during bone healing. This invention effectively utilizes the LSTM network to process temporal dependency information during fracture healing, overcoming the shortcomings of traditional single-time-point analysis.

[0053] The temporal features output from the LSTM layer are transformed into an evaluation result of bone healing status through a fully connected layer and a Softmax activation function, outputting the current bone healing status. Assuming bone healing status is divided into three categories, the CNN-FPN-LSTM network model outputs a three-class classification result: bone nonunion 0, bone healing 1, and bone nonunion 2. The predicted probability distribution is calculated based on the three-class classification result. The formula for the Softmax activation function is:

[0054] y pred =Softmax(FC(LSTM(X)))

[0055] Where: y pred For example, [0.1, 0.7, 0.2] represents the highest probability of bone healing; X = [x1, x2, ..., x T ], LSTM(X), is the result of LSTM aggregating the temporal features of the feature vector; FC is a fully connected layer used for classification tasks;

[0056] The CNN-FPN-LSTM network model is trained repeatedly until the classification accuracy of the validation dataset converges. The classification performance of the CNN-FPN-LSTM network model is checked, and the trained CNN-FPN-LSTM network model is saved. During the model training process, this invention uses multi-scale feature extraction, data augmentation and other techniques to optimize the model, and designs corresponding loss functions and hyperparameter tuning strategies to improve the performance of the model in practical applications.

[0057] Given the output prediction probability distribution of the model The formula for the cross-entropy loss function, along with the true label y, is:

[0058]

[0059] Where: C is the number of categories, with 3 categories: nonunion, partial union, and complete union; y i It is the i-th category of the real label, represented using One-Hot encoding; It is the model's predicted probability for the i-th class.

[0060] Cross-entropy loss measures a model’s performance in a classification task by calculating the difference between the predicted distribution and the true distribution; the smaller the loss, the closer the model’s prediction is to the true label.

[0061] This invention uses the Adam optimizer, which exhibits good convergence in most deep learning models. It employs a dynamic learning rate decay mechanism, adjusting the learning rate based on the performance on the validation set. During model training, the loss value on the validation set is monitored; training stops if the loss no longer decreases after several epochs. Model performance is monitored on the validation set, and performance on the classification task is evaluated using metrics such as accuracy, ROC curve, and confusion matrix. The correlation between the predicted healing score and the actual healing situation is assessed using mean squared error (MSE) and Pearson correlation coefficient. This invention adjusts hyperparameters such as the learning rate, number of LSTM layers, number of LSTM units, and regularization parameters to improve model performance. Transfer learning techniques are used, with initial weights assigned to a pre-trained ResNet model to improve training efficiency.

[0062] (3) Obtain the original CT image data of the fracture to be evaluated, and perform image normalization, region of interest (ROI) cropping and data enhancement preprocessing. Use medical image processing tools to convert the DICOM file of the original CT image data into a 3D or 2D array format, and normalize the gray values ​​of the CT images or convert the HU values ​​to values ​​within the standard CT range [-1000, 4000]. Crop CT slices containing the fracture site according to the fracture area, remove redundant background information, and perform random rotation, translation, scaling and noise addition operations to obtain the preprocessed CT image data. Use the CNN-FPN-LSTM network model trained in step S2 to evaluate the preprocessed CT image data to obtain the current bone healing status.

[0063] Example 2

[0064] Based on the same inventive concept, this invention discloses a bone healing timing assessment system, comprising:

[0065] The data preprocessing unit acquires raw CT images from different time points after a fracture and preprocesses them to obtain a multi-time-point CT image dataset. Each time point is labeled with a label indicating the degree of bone healing, where 0 represents nonunion, 1 represents healing, and 2 represents nonunion, forming a label file. The multi-time-point CT image dataset is divided into a training dataset and a validation dataset, and combined with the label file. Preprocessing in the data preprocessing unit includes image normalization, region of interest (ROI) cropping, and data augmentation preprocessing. This involves converting the DICOM file of the raw CT image data into a 3D or 2D array format, normalizing the grayscale values ​​of the CT images, or converting the HU values ​​to values ​​within the standard CT range [-1000, 4000]. CT slices containing the fracture site are cropped according to the fracture area, redundant background information is removed, and random rotation, translation, scaling, and noise addition operations are performed.

[0066] The model training unit is used to construct the CNN-FPN-LSTM network, which includes a ResNet50 convolutional neural network, an FPN feature pyramid network, an average pooling layer, at least one LSTM layer, and a fully connected layer.

[0067] The acquired training dataset is imported into the CNN-FPN-LSTM network. CT images from multiple time points are stacked into an input tensor of shape (B,T,H,W,C) to convert the CT images into 2D slices. Here, B is the batch size, initially set to 1; T is the number of time steps, representing the number of CT images at different time points; H is the height of the CT image; W is the width of the CT image; and C is the number of channels, where C is the single channel corresponding to grayscale.

[0068] 2D slices are input into a ResNet50 convolutional neural network for feature extraction, and multiple feature maps of different scales are output, including the maximum resolution feature map P2, high resolution feature map P3, medium resolution feature map P4 and low resolution feature map P5.

[0069] Multiple feature maps of different scales are input into the FPN feature pyramid network for fusion to obtain the fused feature map;

[0070] The fused feature map is input into an average pooling layer, which outputs a fixed-length feature vector X = [x1, x2, ..., x...] at different time points. T The feature vector extracted at each time point is input into the LSTM layer to capture the temporal relationship between features at different time points, and the temporal features in the hidden state of the last time step are output for classification.

[0071] The temporal features output by the LSTM layer are transformed into an evaluation result of the bone healing status through a fully connected layer and a softmax activation function, and the current bone healing status is output. Assuming that the bone healing status is divided into three categories, the CNN-FPN-LSTM network outputs a three-class classification, which are bone non-union 0, bone healing 1, and bone non-union 2. The predicted probability distribution is calculated based on the three-class classification results.

[0072] Optionally, the formula for the Softmax activation function in the model training unit is:

[0073] y pred =Softmax(FC(LSTM(X)))

[0074] Where: y pred Let X = [x1, x2, ..., xn] be the probability distribution to be predicted. T ], LSTM(X), is the result of the LSTM layer aggregating the temporal features of the feature vector; FC is a fully connected layer used for classification tasks;

[0075] Repeat the training process multiple times until the evaluation accuracy of the CNN-FPN-LSTM network on the validation dataset converges. Check the evaluation performance of the CNN-FPN-LSTM network and save the trained CNN-FPN-LSTM network.

[0076] The bone healing assessment unit is used to acquire the original CT image data after the fracture to be assessed, perform preprocessing to obtain preprocessed CT image data, and use the CNN-FPN-LSTM network trained in step S2 to evaluate the preprocessed CT image data to obtain the current bone healing status. The bone healing assessment unit performs image normalization, region of interest (ROI) cropping and data augmentation preprocessing, converts the DICOM file of the original CT image data into 3D or 2D array format, normalizes the gray values ​​of the CT images or converts the HU values ​​to values ​​within the standard CT range [-1000, 4000], crops CT slices containing the fracture site according to the fracture area, removes redundant background information, and performs random rotation, translation, scaling and noise addition operations.

[0077] Example 3

[0078] Another embodiment of the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a bone healing timing assessment method as described above.

[0079] The electronic device may include: a processor, a memory, a bus, and a communication interface, wherein the processor, the communication interface, and the memory are connected via the bus; the memory stores a computer program that can run on the processor, and when the processor runs the computer program, it executes a bone healing timing assessment method provided by any of the foregoing embodiments of the present invention.

[0080] The memory may include high-speed random access memory (RAM), and may also include non-volatile memory, such as at least one disk storage device. Communication between this device network element and at least one other network element is achieved through at least one communication interface (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc.

[0081] The bus can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into address bus, data bus, control bus, etc. The memory is used to store programs, and the processor executes the program after receiving execution instructions. The bone healing timing assessment method disclosed in any of the foregoing embodiments of the present invention can be applied to a processor, or implemented by a processor.

[0082] The processor may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed through integrated logic circuits in the processor's hardware or through software instructions. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application-Specific Integrated Circuit (ASIC), an Off-the-shelf Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this invention can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0083] The electronic devices and methods provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods they employ, operate, or implement.

[0084] Example 4

[0085] Another embodiment of this application provides a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the methods of any of the above embodiments. The computer-readable storage medium is an optical disc having a computer program (i.e., a program product) stored thereon, the computer program, when run by a processor, performs the methods provided in any of the foregoing embodiments.

[0086] It should be noted that examples of computer-readable storage media may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

Claims

1. A method for assessing bone healing timing, characterized in that, Includes the following steps: (1) Obtain several original CT image data at different times after fracture and preprocess them to obtain a multi-time point CT image dataset; label the degree of bone healing at each time point, where 0 represents nonunion, 1 represents bone healing, and 2 represents nonunion, forming a label file; divide the multi-time point CT image dataset into a training dataset and a validation dataset, and combine them with the label file. (2) Construct a CNN-FPN-LSTM network, which includes a ResNet50 convolutional neural network, an FPN feature pyramid network, an average pooling layer, at least one LSTM layer, and a fully connected layer. The training dataset obtained in step (1) is imported into the CNN-FPN-LSTM network. The CT images at multiple time points are stacked into an input tensor of shape (B,T,H,W,C) to convert the CT images into 2D slices. Here, B is the batch size, which is initially set to 1. T is the number of time steps, which represents the number of CT images at different time points. H is the height of the CT image. W is the width of the CT image. C is the number of channels, which is the single channel corresponding to grayscale. 2D slices are input into a ResNet50 convolutional neural network for feature extraction, and multiple feature maps of different scales are output, including the maximum resolution feature map P2, high resolution feature map P3, medium resolution feature map P4 and low resolution feature map P5. Multiple feature maps of different scales are input into the FPN feature pyramid network for fusion to obtain the fused feature map; The fused feature map is input into an average pooling layer, which outputs a fixed-length feature vector X = [x1, x2, ..., x...] at different time points. T The feature vector extracted at each time point is input into the LSTM layer to capture the temporal relationship between features at different time points, and the temporal features in the hidden state of the last time step are output for classification. The temporal features output by the LSTM layer are transformed into an evaluation result of the bone healing status through a fully connected layer and a softmax activation function, and the current bone healing status is output. Assuming that the bone healing status is divided into three categories, the CNN-FPN-LSTM network outputs a three-class classification, which are bone non-union 0, bone healing 1, and bone non-union 2. The predicted probability distribution is calculated based on the three-class classification results. Repeat the training process multiple times until the evaluation accuracy of the CNN-FPN-LSTM network on the validation dataset converges. Check the evaluation performance of the CNN-FPN-LSTM network and save the trained CNN-FPN-LSTM network. (3) Obtain the original CT image data after the fracture to be evaluated, perform preprocessing to obtain the preprocessed CT image data, and use the CNN-FPN-LSTM network trained in step S2 to evaluate the preprocessed CT image data to obtain the current bone healing status.

2. The method for assessing bone healing timing according to claim 1, characterized in that: In step (1), preprocessing refers to image normalization, region of interest (ROI) cropping, and data enhancement preprocessing. This involves converting the original CT image data DICOM file into a 3D or 2D array format, normalizing the grayscale values ​​of the CT image, or converting the HU values ​​to values ​​within the standard CT range [-1000, 4000]. Based on the fracture area, CT slices containing the fracture site are cropped, redundant background information is removed, and random rotation, translation, scaling, and noise addition operations are performed.

3. The method for assessing bone healing timing according to claim 1, characterized in that: The formula for the Softmax activation function in step (2) is: y pred =Softmax(FC(LSTM(X))) Where: y pred Let X = [x1, x2, ..., xn] be the probability distribution to be predicted. T ], LSTM(X), is the result of the LSTM layer aggregating the temporal features of the feature vector; FC is a fully connected layer used for classification tasks.

4. The method for assessing bone healing timing according to claim 1, characterized in that: In step (3), image normalization, region of interest (ROI) cropping, and data enhancement preprocessing are performed. The DICOM file of the original CT image data is converted into a 3D or 2D array format, and the gray values ​​of the CT image are normalized or the HU values ​​are converted into values ​​within the standard CT range [-1000, 4000]. CT slices containing the fracture site are cropped according to the fracture area, redundant background information is removed, and random rotation, translation, scaling, and noise addition operations are performed.

5. A bone healing timing assessment system, characterized in that, include: The data preprocessing unit is used to acquire raw CT image data at different times after fracture and preprocess it to obtain a multi-time point CT image dataset; label the degree of bone healing at each time point, where 0 represents nonunion, 1 represents bone healing, and 2 represents nonunion, forming a label file; divide the multi-time point CT image dataset into a training dataset and a validation dataset, and combine them with the label file. The model training unit is used to construct the CNN-FPN-LSTM network, which includes a ResNet50 convolutional neural network, an FPN feature pyramid network, an average pooling layer, at least one LSTM layer, and a fully connected layer. The acquired training dataset is imported into the CNN-FPN-LSTM network. CT images from multiple time points are stacked into an input tensor of shape (B,T,H,W,C) to convert the CT images into 2D slices. Here, B is the batch size, initially set to 1; T is the number of time steps, representing the number of CT images at different time points; H is the height of the CT image; W is the width of the CT image; and C is the number of channels, where C is the single channel corresponding to grayscale. 2D slices are input into a ResNet50 convolutional neural network for feature extraction, and multiple feature maps of different scales are output, including the maximum resolution feature map P2, high resolution feature map P3, medium resolution feature map P4 and low resolution feature map P5. Multiple feature maps of different scales are input into the FPN feature pyramid network for fusion to obtain the fused feature map; The fused feature map is input into an average pooling layer, which outputs a fixed-length feature vector X = [x1, x2, ..., x...] at different time points. T The feature vector extracted at each time point is input into the LSTM layer to capture the temporal relationship between features at different time points, and the temporal features in the hidden state of the last time step are output for classification. The temporal features output by the LSTM layer are transformed into an evaluation result of the bone healing status through a fully connected layer and a softmax activation function, and the current bone healing status is output. Assuming that the bone healing status is divided into three categories, the CNN-FPN-LSTM network outputs a three-class classification, which are bone non-union 0, bone healing 1, and bone non-union 2. The predicted probability distribution is calculated based on the three-class classification results. Repeat the training process multiple times until the evaluation accuracy of the CNN-FPN-LSTM network on the validation dataset converges. Check the evaluation performance of the CNN-FPN-LSTM network and save the trained CNN-FPN-LSTM network. Bone healing assessment unit: used to acquire the original CT image data after the fracture to be assessed, perform preprocessing to obtain preprocessed CT image data, and use the CNN-FPN-LSTM network trained in step S2 to evaluate the preprocessed CT image data to obtain the current bone healing status.

6. The bone healing timing assessment system according to claim 5, characterized in that: The data preprocessing unit refers to image normalization, region of interest (ROI) cropping, and data enhancement preprocessing. This involves converting the original CT image data DICOM file into a 3D or 2D array format, normalizing the grayscale values ​​of the CT images, or converting the HU values ​​to values ​​within the standard CT range [-1000, 4000]. CT slices containing the fracture sites are cropped based on the fracture area, redundant background information is removed, and random rotation, translation, scaling, and noise addition operations are performed.

7. The bone healing timing assessment system according to claim 5, characterized in that: The formula for the Softmax activation function in the model training unit is: y pred =Softmax(FC(LSTM(X))) Where: y pred Let X = [x1, x2, ..., xn] be the probability distribution to be predicted. T ], LSTM(X), is the result of the LSTM layer aggregating the temporal features of the feature vector; FC is a fully connected layer used for classification tasks.

8. The bone healing timing assessment system according to claim 5, characterized in that: The bone healing assessment unit performs image normalization, region of interest (ROI) cropping, and data enhancement preprocessing. It converts the DICOM file of the original CT image data into a 3D or 2D array format, normalizes the grayscale values ​​of the CT images, or converts the HU values ​​to values ​​within the standard CT range [-1000, 4000]. It crops CT slices containing the fracture site based on the fracture area, removes redundant background information, and performs random rotation, translation, scaling, and noise addition operations.

9. An electronic device, characterized in that, The device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a bone healing timing assessment method as described in any one of claims 1-4.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program is executed by a processor to implement a bone healing timing assessment method as described in any one of claims 1-4.

Images