Guided improved neural network ankle joint fracture classification method, system and equipment

Abstract

The invention relates to a guiding type improved neural network ankle joint fracture classification method, system and device, and belongs to the field of computer-aided medical image processing.The classification method comprises the steps that S1, database images are collected and enhanced; s2, training a segmentation network model by using the segmentation data marked with the fibula region; s3, inputting the fracture type classification data into the trained segmentation network model, predicting a fibula area, and taking a prediction result as guidance to act on the improved neural network classification model; s4, inputting the fracture type classification data into the improved neural network classification model guided by the prediction result for training; and S5, verifying a classification result by using a test set, and evaluating the model by using a statistical index. The ankle joint fracture type can be judged, the method has the advantages of being high in diagnosis accuracy, high in speed and the like, and the problem that a traditional deep learning method is prone to being interfered by other information in images in the medical field to make misjudgment is effectively solved.

Description

Technical Field

[0001] The present invention belongs to the field of computer image processing, and in particular relates to a guided improved neural network ankle fracture classification method, system and equipment. Background Art

[0002] Ankle fracture is a common orthopedic injury, mainly due to direct or indirect violence and twisting of the ankle. The diagnosis of ankle injuries often uses medical imaging methods such as taking X-rays of the patient. After the injury, the ankle joint will experience severe pain, swelling of the injured area, ankle deformity and other symptoms, and the movement of the ankle joint will be significantly restricted. Since the ankle joint is a weight-bearing joint of the human body, if the fracture is not treated promptly and effectively, it will not only affect the patient's normal walking, but also cause arthritis, the broken bone cannot heal and other problems, which will have a great impact on the patient's life.

[0003] In recent years, as classification technology in deep learning (DL) has gradually shown great potential in the medical field, compared with the traditional method of simply relying on doctors to analyze and judge fracture images, the use of convolutional neural network (CNN) technology based on deep learning to automatically identify fractures has more advantages in judging fracture types. Doctors can use this output result as a reference, which can reduce the workload of doctors and improve work efficiency.

[0004] Although CNN-driven artificial intelligence (AI) methods have achieved encouraging performance, due to the opaque learning and reasoning process of CNN, it is easy to be disturbed by other information in the image and make wrong judgments when used for clinical diagnosis, which has become an obstacle to the application of machine diagnosis in clinical settings. The task of analyzing medical images is often limited by inaccurate classification due to excessive image interference information, making its application in real-world scenarios very challenging. Therefore, it is necessary to design a model that introduces prior knowledge based on the original case image to guide CNN attention to judge the type of ankle fracture to assist doctors in treatment in order to better meet treatment needs. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention provides a guided improved neural network ankle fracture classification method, system and device, which guides the network attention by segmenting the network and then uses the improved convolutional neural network to accurately classify the patient's injury type under the theoretical model based on the ankle Danis-Weber classification, and uses various statistical indicators to evaluate the model performance, thereby completing the judgment of the type of ankle fracture. It effectively solves the problem that the traditional convolutional neural network is easily disturbed by other information in the image during classification and makes wrong judgments, resulting in a low accuracy rate.

[0006] To achieve the above object, the present invention discloses the following technical solution:

[0007] In one aspect, the present invention provides a guided improved neural network ankle fracture classification method, which comprises the following steps:

[0008] S1. Collect ankle fracture X-ray images and images with annotated ankle fibula segmentation data and fracture type classification data in the database and perform image enhancement; randomly divide the collected images into a training set and a validation set;

[0009] S2, using the segmentation data of the annotated fibula region to train the segmentation network model, so that the segmentation network model can accurately identify the size and position of the fibula on the image;

[0010] When training the segmentation network model, the improved dice coefficient loss function is used to evaluate the segmentation effect of the model and guide the modification direction of the parameters. The loss function calculation formula is:

[0011]

[0012]

[0013] Among them, t i is the predicted value of the i-th pixel, g i is the true value of the i-th pixel, ε is the term to ensure the stability of the loss function, α is the weight coefficient, and the size is automatically adjusted according to the proportion of the foreground. The larger α is, the larger the proportion of the image foreground is. H and W are the height and width of the image respectively.

[0014] S3, inputting the fracture type classification data into the trained segmentation network model, predicting the fibula region, and using the prediction results as a guide to improve the neural network classification model;

[0015] S4. Input the fracture type classification data into the improved neural network classification model guided by the prediction results for training; the improved neural network improved convolutional neural network model includes stacking of residual blocks, global average pooling and multi-layer perceptron, the stacking of residual blocks includes direct mapping part and residual part, direct mapping passes the acquired features into two convolutional layers of size 3*3, and the residual part is a connection structure from input to output; global average pooling performs global average on the entire feature map, reduces the spatial dimension of the feature map, calculates the average value of all pixel values ​​in each feature map, and generates a global feature representation; the multi-layer perceptron includes an input layer, two hidden layers and an output layer;

[0016] When training the classification model, the improved cross entropy loss function is used to evaluate the error between the model's predicted results and the actual results and guide the modification direction of the parameters. The calculation formula is:

[0017]

[0018] Among them, n c is the number of categories; p nc The sum operation of model output categories; q nc is the sum operation of the category labels, w c is the weight of the category label. The smaller the proportion of the category in the data set, the greater the weight;

[0019] S5. Use the test set to verify the classification results of the improved neural network classification model, and use statistical indicators to evaluate the improved neural network classification model.

[0020] Preferably, when acquiring the ankle joint fibula segmentation data in step S1, the area of ​​the fibula on the image is divided.

[0021] Preferably, when obtaining the fracture type classification data in step S1, the classification method adopted is the Danis-Weber classification method. According to the height of the fibula fracture, the horizontal gap between the fibula fracture and the ankle hole, and the relationship between the distal tibiofibular joint, the ankle fracture types are divided into three categories: type A fracture, type B fracture and type C fracture, and stored in different folders to obtain the fracture type classification data.

[0022] Preferably, the segmentation network model in step S2 uses a U-Net network structure, including an encoder and a decoder, and transmits information through jump connections to achieve accurate segmentation of the input image; the encoder and the decoder both include four stages, each stage of the encoder includes two 3*3 convolutional layers and a 2*2 maximum pooling layer; each stage of the decoder doubles the spatial resolution of the feature map by performing an upsampling operation through a 2*2 deconvolution.

[0023] Preferably, when predicting the fibula region in the classified data in step S3, the ratio of the intersection and union of the true label and the predicted value of the segmentation category is calculated and then averaged, that is, the average intersection-union ratio, and the calculation formula is:

[0024]

[0025] Among them, T represents the predicted value, G represents the true value, and k represents the total number of categories.

[0026] Preferably, in step S4, when calculating the cross entropy loss function, the distribution of p is used to approximate the distribution of q, and the closer p and q are, the smaller the loss is;

[0027]

[0028] Among them, s is the ankle joint image data sample; l is the true label;

[0029] The label distribution q is equivalent to a probability set; the true labels l and n in the samplec Belong to the same order of magnitude, q and l have different mathematical orders of magnitude; if the true label l and the predicted sample label n c If they are the same, the prediction is correct, which is 1, otherwise it is 0.

[0030] Preferably, in step S5, a confusion matrix is ​​used to display the amount of data correctly and incorrectly classified by the classifier, the precision, recall, Fβscore and accuracy are used to measure the robustness of the model, and the receiver operating characteristic curve is used to reflect the performance of the classifier, the horizontal axis is the false positive rate of each category, the vertical axis is the true positive rate, and the area under the ROC curve and the coordinate axis is calculated as the AUC value to characterize the classification performance.

[0031] Preferably, in step S1, the collected images are randomly divided into a training set and a validation set in a ratio of 9:1.

[0032] On the other hand, the present invention provides a guided improved neural network ankle fracture classification system, which includes a data acquisition unit, a segmentation network model training unit, an improved neural network classification model guidance unit, an improved neural network classification model training unit and an improved neural network classification model verification unit:

[0033] The data acquisition unit acquires ankle fracture X-ray images and images with annotated ankle fibula regions and fracture types in the database and performs image enhancement; the acquired images are randomly divided into a training set and a validation set;

[0034] The segmentation network model training unit uses the segmentation data of the annotated fibula region to train the segmentation network model, so that the segmentation network model can accurately identify the size and position of the fibula on the image;

[0035] The improved neural network classification model guiding unit inputs the fracture type classification data into the trained segmentation network model, predicts the fibula region, and uses the prediction result as a guide for the improved neural network classification model;

[0036] The improved neural network classification model training unit inputs the fracture type classification data into the improved neural network classification model guided by the prediction result for training;

[0037] The improved neural network classification model verification unit verifies the classification result using a test set and evaluates the model using statistical indicators.

[0038] In a third aspect, the present invention provides a guided improved neural network ankle fracture classification device, wherein a guided improved neural network ankle fracture classification system is provided inside the device.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] (1) The present invention provides a guided improved neural network ankle fracture classification method, which guides network attention by segmenting the network and then uses an improved convolutional neural network to accurately classify the patient's injury type under a theoretical model based on the ankle Danis-Weber classification, and uses various statistical indicators to evaluate the model performance, thereby completing the judgment of the type of ankle fracture.

[0041] (2) The ankle fracture classification method of the present invention effectively solves the problem that the traditional convolutional neural network is easily disturbed by other information in the image during classification and makes wrong judgments, resulting in low accuracy. It improves the accuracy of ankle fracture classification, thereby assisting in better classification of ankle fractures. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A schematic diagram of the control flow of an attention-guided improved convolutional neural network ankle fracture classification method proposed in the present invention;

[0043] Figure 2 It is a schematic diagram of the overall technical structure framework of the method implemented in the present invention;

[0044] Figure 3 A schematic diagram of the segmentation network structure provided by the present invention;

[0045] Figure 4 A schematic diagram of the residual module operation structure provided by the present invention;

[0046] Figure 5 A schematic diagram of the global average pooling operation structure provided by the present invention;

[0047] Figure 6 A schematic diagram of the structure of a multi-layer perceptron provided by the present invention;

[0048] Figure 7 It is a schematic diagram of the average intersection-over-union ratio iteration of the model on the test set provided by the present invention;

[0049] Figure 8 A schematic diagram of a receiver operating characteristic curve of the test set classification results provided by the present invention;

[0050] Fig. 9 A schematic block diagram of a classification system of the present invention. DETAILED DESCRIPTION

[0051] The exemplary embodiments, features and aspects of the present invention will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0052] On the one hand, the present invention provides a guided improved neural network ankle fracture classification method, such as Figure 1 As shown, it includes the following steps:

[0053] S1. Collect ankle fracture X-ray images and images with annotated ankle fibula segmentation data and fracture type classification data in the database and perform image enhancement; randomly divide the collected images into training sets and validation sets. When determining the ankle fibula segmentation data in step S1, the area of ​​the fibula on the image is divided. When determining the fracture type classification data, the classification method used is the Danis-Weber classification method. According to the height of the fibula fracture, the horizontal gap between the fibula fracture and the ankle hole, and the relationship between the tibiofibular joint, the ankle fracture types are divided into three categories: type A fracture, type B fracture, and type C fracture, and stored in folders to obtain fracture type classification data.

[0054] S2. Use the segmentation data of the annotated fibula area to train the segmentation network model so that the segmentation network model can accurately identify the size and position of the fibula in the image.

[0055] The segmentation network model in step S2 uses a U-Net network structure, including an encoder and a decoder, and transmits information through jump connections to achieve accurate segmentation of the input image; the encoder and decoder both include four stages, each stage of the encoder contains two 3*3 convolutional layers and a 2*2 maximum pooling layer; each stage of the decoder doubles the spatial resolution of the feature map by upsampling with a 2*2 deconvolution operation.

[0056] When training the segmentation network model, the improved dice coefficient loss function is used to evaluate the segmentation effect of the model and guide the modification direction of the parameters. The loss function calculation formula is:

[0057]

[0058]

[0059] Among them, t i is the predicted value of the i-th pixel, g i is the true value of the i-th pixel, ε is the term to ensure the stability of the loss function, α is the weight coefficient, and the size is automatically adjusted according to the proportion of the foreground. The larger α is, the larger the proportion of the image foreground is. H and W are the height and width of the image respectively.

[0060] S3. Input the fracture type classification data into the trained segmentation network model, predict the fibula area, and use the prediction results as a guide to improve the neural network classification model.

[0061] In a specific application, when predicting the fibula region in the classified data in step S3, the ratio of the intersection and union of the true label and the predicted value of the segmentation category is calculated and then the average is taken, that is, the average intersection-union ratio, and the calculation formula is:

[0062]

[0063] Among them, T represents the predicted value, G represents the true value, and k represents the total number of categories.

[0064] S4. Input the fracture type classification data into the improved neural network classification model guided by the prediction results for training; the improved neural network improved convolutional neural network model includes stacking of residual blocks, global average pooling and multi-layer perceptron. The stacking of residual blocks includes a direct mapping part and a residual part. The direct mapping passes the acquired features into two convolutional layers of size 3*3, and the residual part is a connection structure from input to output; global average pooling performs global average on the entire feature map, reduces the spatial dimension of the feature map, calculates the average of all pixel values ​​in each feature map, and generates a global feature representation; the multi-layer perceptron includes an input layer, two hidden layers and an output layer.

[0065] When training the classification model, the improved cross entropy loss function is used to evaluate the error between the model's predicted results and the actual results and guide the modification direction of the parameters. The calculation formula is:

[0066]

[0067] Among them, n c is the number of categories; p nc The sum operation of model output categories; q nc is the sum operation of the category labels, w c is the weight of the category label. The smaller the proportion of the category in the data set, the greater the weight.

[0068] When calculating the cross entropy loss function, the distribution of p is used to approximate the distribution of q. The closer p and q are, the smaller the loss.

[0069]

[0070] Among them, s is the ankle joint image data sample; l is the true label.

[0071] The label distribution q is equivalent to a probability set; the true labels l and n in the sample c Belong to the same order of magnitude, q and l have different mathematical orders of magnitude; if the true label l and the predicted sample label n c If they are the same, the prediction is correct, which is 1, otherwise it is 0.

[0072] S5. Use the test set to verify the classification results and use statistical indicators to evaluate the model.

[0073] Specifically, in step S5, a confusion matrix is ​​used to display the amount of data correctly and incorrectly classified by the classifier, precision, recall, Fβscore and accuracy are used to measure the robustness of the model, and the receiver operating characteristic curve is used to reflect the performance of the classifier. The horizontal axis is the false positive rate of each category, and the vertical axis is the true positive rate. The area under the ROC curve and the coordinate axis is calculated as the AUC value to characterize the classification performance.

[0074] On the other hand, the present invention provides a guided improved neural network ankle fracture classification system, such as Fig. 9 As shown, it includes a data acquisition unit 1, a segmentation network model training unit 2, an improved neural network classification model guiding unit 3, an improved neural network classification model training unit 4 and an improved neural network classification model verification unit 5.

[0075] The data acquisition unit 1 acquires ankle fracture X-ray images and images with annotated ankle fibula regions and fracture types in the database and performs image enhancement; the acquired images are randomly divided into a training set and a validation set.

[0076] The segmentation network model training unit 2 uses the segmentation data of the annotated fibula area to train the segmentation network model, so that the segmentation network model can accurately identify the size and position of the fibula in the image.

[0077] The improved neural network classification model guiding unit 3 inputs the fracture type classification data into the trained segmentation network model, predicts the fibula area, and uses the prediction result as a guide for the improved neural network classification model.

[0078] The improved neural network classification model training unit 4 inputs the fracture type classification data into the improved neural network classification model guided by the prediction results for training.

[0079] The improved neural network classification model verification unit 5 verifies the classification result using the test set and evaluates the model using statistical indicators.

[0080] In a third aspect, the present invention provides a guided improved neural network ankle fracture classification device, wherein a guided improved neural network ankle fracture classification system is provided inside the device.

[0081] This embodiment provides a guided improved neural network ankle fracture classification method, such as Figure 1 and Figure 2 As shown, it includes the following steps:

[0082] S1. Collect clinical ankle fracture X-ray images, then perform histogram equalization on the images to count the frequency of pixel values ​​within the grayscale range of the entire image, and generate a histogram based on this. Equalizing the histogram can redistribute the pixel values ​​in the image, making the pixel distribution of the image more uniform and enhancing the contrast of the image. Clinical orthopedic experts annotate the image data, and the annotations are divided into two categories. The first category is the annotation of segmentation, using software to frame the outer contour of the fibula area in all ankle X-ray images; the second category is to divide the original image into three categories according to the Danis-Weber classification theory: type A fracture, type B fracture, and type C fracture, and store them in folders. The Danis-Weber classification theory divides ankle fractures into different types based on the height of the fibula fracture and the relationship between the fibula fracture and the horizontal gap of the ankle hole and the tibiofibular syndesmosis. After the ankle fracture occurs, the treatment plan can be selected according to the Danis-Weber classification method. The two types of data sets produced are randomly divided into training sets and validation sets at a ratio of 9:1.

[0083] S2. Input the prepared first-category data set into the image segmentation network so that the system can autonomously predict the location of the fibula on the X-ray image. The segmentation network uses the U-Net network, and the network structure is as follows: Figure 3 As shown in the figure. The network consists of two parts, the encoder and the decoder, and transmits information through jump connections to achieve accurate segmentation of the input image. The encoder part is used to extract the features of the input image. It consists of four stages, each of which contains two 3*3 convolutional layers (Conv Layer) and a 2*2 maximum pooling (Max Pooling) layer. The decoder is used to gradually restore the feature map compressed by the encoder to the resolution of the original image and generate the segmentation result at the same time. The decoder consists of four stages, each of which doubles the spatial resolution of the feature map by upsampling through 2*2 deconvolution (UpConv). After each upsampling, the decoder performs a jump connection to splice the feature map of the corresponding encoder stage with the feature map of the current decoder stage. The jump connection is to directly transfer the low-level features in the encoder to the corresponding layer in the decoder.

[0084] When training the segmentation network, the Dice Loss is used to evaluate the segmentation effect of the model and guide the modification direction of the parameters. The calculation formula is:

[0085]

[0086]

[0087] Among them, t i is the predicted value of the i-th pixel, g iis the true value of the i-th pixel, ε is the term to ensure the stability of the loss function, α is the weight coefficient, and the size is automatically adjusted according to the proportion of the foreground. The larger α is, the larger the proportion of the image foreground is. H and W are the height and width of the image respectively.

[0088] S3. Input the prepared second type of data into the improved convolutional neural network model to obtain the classification results. The input size of the image is adjusted to 256*256 pixels, and the pixel value of each channel is mapped to a standard normal distribution with a mean of 0 and a standard deviation of 1 for normalization. Finally, the RGB channels of the image are standardized. These operations ensure that the images input to the neural network have a consistent size and a uniform value range during the training process.

[0089] S4. The new image is first passed through the segmentation prediction network to obtain the fibula area, and the size and position of the fibula on the image are obtained. Using this area as the attention guidance model can eliminate the interference of skin tissue and other bone tissues in the X-ray on the classification model. The improved convolutional neural network model consists of three parts: stacking of residual blocks, global average pooling (Global AveragePooling, GAP) and multilayer perceptron (Multilayer Perceptron, MLP). The stacking of residual blocks can solve the problems of gradient disappearance and gradient explosion during network training, and is mainly composed of two parts: direct mapping and residual. The direct mapping part passes the acquired features into two convolutional layers of size 3*3, and the residual part is a connection structure from input to output. The idea of ​​the entire residual block structure is to add the feature matrix obtained after a series of convolutional layers on the main line to the input feature matrix, and then output it through the activation function, such as Figure 4 As shown. Therefore, a residual block is defined as:

[0090] H(x)=F(x)+x

[0091] Where x is the input feature vector, H(x) is the output of the residual module, and F(x) is the output of the convolutional layer.

[0092] At the end of the convolutional layer, global average pooling is performed to globally average the entire feature map, reduce the spatial dimension of the feature map, and generate a global feature representation, such as Figure 5 As shown. The implementation method is to calculate the average value of all pixel values ​​in each feature map, generate a global feature representation, and convert the three-dimensional feature map of size H*W*C into 1*1*C. This method significantly reduces the number of parameters and computational complexity by reducing each feature map to a single value, while maintaining the integrity of global information. The calculation process is as follows:

[0093]

[0094] Where H, w, c are the height, width and number of channels of the feature map respectively. x represents the feature value at position (i, j) and channel c.

[0095] The two-layer multi-layer perceptron model is designed to receive the global feature representation from the average pooling layer as input, and perform nonlinear transformation and feature combination through multiple fully connected layers, thereby enhancing the model's ability to process complex features. The network model consists of an input layer, two hidden layers, and an output layer, such as Figure 6 shown.

[0096] The first hidden layer maps the input features to a higher-dimensional feature space, and the second hidden layer further processes the output of the first hidden layer to obtain the final output y through nonlinear mapping. The calculation process is as follows.

[0097] h=max(w1x+b1,0)

[0098] y=w2h+b2

[0099] Among them, h is the output of the first hidden layer, w1, w2, b1, b2 are the weight matrix and bias of the first and second hidden layers respectively, and y is the final output.

[0100] In order to accelerate model convergence and prevent the model from falling into local optimality, the Adam optimizer is used to adapt the learning rate and momentum. Weight decay is introduced in the optimizer, and the weight factor is set to 1e-4 to control the complexity of the model and prevent the model from over-relying on certain features to improve generalization ability.

[0101] When training the classification model, an improved cross-entropy loss function (Cross-Entropy Loss) is used to evaluate the error between the model's predicted results and the actual results and guide the modification direction of the parameters. The calculation formula is:

[0102]

[0103] Among them, n c is the number of categories; Sum the model output categories; is the sum operation of the category labels, w c is the weight of the category label. The smaller the proportion of the category in the data set, the larger the weight, and the more attention the loss function pays to the category.

[0104] When calculating the cross entropy loss function, the distribution of p is used to approximate the distribution of q. The closer p and q are, the smaller the loss.

[0105]

[0106] Among them, s is the ankle joint image data sample; l is the true label.

[0107] The label distribution q is equivalent to the probability set; the true labels l and n in the sample c Belong to the same order of magnitude, q and l have different mathematical orders of magnitude; if the true label l and the predicted sample label n c If they are the same, the prediction is correct, which is 1, otherwise it is 0.

[0108] Gradually reduce the learning rate through the learning rate scheduler, so that the parameters can be adjusted more carefully in the later stages of training. Use the StepLR scheduler, which is a method of adjusting the learning rate at equal intervals, and decay the learning rate according to a fixed number of training rounds. The size of the learning rate decay scheduler is not affected by weight decay, but only by the number of training rounds.

[0109] In step S3, in order to measure the results of the standard segmentation, the ratio of the intersection and union of the true label and the predicted value of the segmentation category is calculated and then averaged, that is, the mean intersection-union ratio (MIou), and the calculation formula is:

[0110]

[0111] Among them, T represents the predicted value, G represents the true value, and K represents the total number of categories.

[0112] The model was trained for 40 rounds and the MioU index result for segmenting the fibula in the test set of ankle fractures was 97.57%. Figure 7 The figure shows the changing curve of the MioU indicator on the test set as the number of training rounds increases.

[0113] S5. In order to quantitatively evaluate the classification model, a confusion matrix representing the classification task results is introduced, and the accuracy and sensitivity are calculated from it, and these results are used as performance indicators to measure the model. Each row of the confusion matrix represents the true number of categories in the classification model, and each column represents the number of predictions for this category. Since the classification task is divided into three categories, the size of the confusion matrix is ​​3*3.

[0114] Table 1 Confusion matrix

[0115]

[0116] The precision calculation formula for each category is:

[0117]

[0118] The calculation formula for recall is:

[0119]

[0120] The Fβscore calculation formula is:

[0121]

[0122] β is a harmonic parameter used to balance the precision and recall values. Since these two indicators cannot be high at the same time, and the increase in the value of one indicator will be accompanied by the decrease of the other indicator, this study sets β = 1 to consider that the two are equally important.

[0123] The accuracy calculation formula is:

[0124]

[0125] The performance of the proposed classification algorithm on the test set is shown in Table 2. Five independent experiments were conducted, and the final result is the average of the five experiments. According to the obtained results, the weighted average index of the classification model is calculated, and it can be obtained that on the test set, the classification accuracy of the model is 91.14%.

[0126] Table 2 Results of various indicators of the classification model on the validation set

[0127] category Precision(%) Recall rate (%) F1 score A 100 90.91 95.24 B 89.66 86.67 88.14 C 58.19 95.83 90.20

[0128] The receiver operating characteristic (ROC) curve is used to reflect the performance of the classifier. The horizontal axis is the false positive rate of each category, and the vertical axis is the true positive rate. The area under the ROC curve and the coordinate axis is calculated as the AUC value. The closer the value, the more effective the classification method.

[0129] In the model proposed in this study, the ROC curves and corresponding AUC values ​​of the effects of different fracture categories on the test set are as follows: Figure 8 As shown in Figure 2, under the guidance of the Danis-Weber classification of medical ankle fractures, the A, B, and C categories obtained AUC results of 99.54%, 97.86%, and 99.04%, respectively.

[0130] In summary, network inference results and data set experiments have proved that an attention-guided improved convolutional neural network ankle fracture classification method has a good application effect.

[0131] The embodiments described above are only descriptions of the preferred implementation modes of the present invention, and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A guided improved neural network ankle fracture classification method, characterized by: It includes the following steps: S1. Collect ankle fracture X-ray images and images with annotated ankle fibula segmentation data and fracture type classification data in the database and perform image enhancement; randomly divide the collected images into a training set and a validation set; S2, using the segmentation data of the annotated fibula region to train the segmentation network model, so that the segmentation network model can accurately identify the size and position of the fibula on the image; When training the segmentation network model, the improved dice coefficient loss function is used to evaluate the segmentation effect of the model and guide the modification direction of the parameters. The loss function calculation formula is: Among them, t i is the predicted value of the i-th pixel, g i is the true value of the i-th pixel, ε is the term to ensure the stability of the loss function, α is the weight coefficient, and the size is automatically adjusted according to the proportion of the foreground. The larger α is, the larger the proportion of the image foreground is. H and W are the height and width of the image respectively. S3, inputting the fracture type classification data into the trained segmentation network model, predicting the fibula region, and using the prediction results as a guide to improve the neural network classification model; S4. Input the fracture type classification data into the improved neural network classification model guided by the prediction results for training; the improved neural network improved convolutional neural network model includes stacking of residual blocks, global average pooling and multi-layer perceptron, the stacking of residual blocks includes direct mapping part and residual part, direct mapping passes the acquired features into two convolutional layers of size 3*3, and the residual part is a connection structure from input to output; global average pooling performs global average on the entire feature map, reduces the spatial dimension of the feature map, calculates the average value of all pixel values ​​in each feature map, and generates a global feature representation; the multi-layer perceptron includes an input layer, two hidden layers and an output layer; When improving the training of the neural network classification model, the improved cross entropy loss function is used to evaluate the error between the model's predicted results and the actual results and guide the modification direction of the parameters. The calculation formula of the cross entropy loss function is: Among them, n c is the number of categories; Sum the model output categories; is the sum operation of the category labels, w c is the weight of the category label. The smaller the proportion of the category in the data set, the greater the weight; S5. Use the test set to verify the classification results of the improved neural network classification model, and use statistical indicators to evaluate the improved neural network classification model.

2. The guided improved neural network ankle fracture classification method according to claim 1, characterized in that: When obtaining the ankle joint fibula segmentation data in step S1, the area of ​​the fibula on the image is divided.

3. The guided improved neural network ankle fracture classification method according to claim 2, characterized in that: When obtaining the fracture type classification data in step S1, the classification method used is the Danis-Weber classification method. According to the height of the fibula fracture, the horizontal gap between the fibula fracture and the ankle hole, and the relationship between the distal tibiofibular joint, the ankle fracture types are divided into three categories: type A fracture, type B fracture, and type C fracture, and stored in different folders to obtain the fracture type classification data.

4. The guided improved neural network ankle fracture classification method according to claim 1, characterized in that: The segmentation network model in step S2 uses a U-Net network structure, including an encoder and a decoder, and transmits information through jump connections to achieve accurate segmentation of the input image; the encoder and decoder both include four stages, and each stage of the encoder includes two 3*3 convolutional layers and a 2*2 maximum pooling layer; Each stage of the decoder doubles the spatial resolution of the feature map by performing an upsampling operation with a 2*2 convolution.

5. The guided improved neural network ankle fracture classification method according to claim 1, characterized in that: When predicting the fibula region in the classified data in step S3, the ratio of the intersection and union of the true label and the predicted value of the segmentation category is calculated and then the average is taken, that is, the average intersection-union ratio, and the calculation formula is: Among them, Miou is the average intersection-over-union ratio, T is the predicted value, G is the true value, and k is the total number of categories.

6. The guided improved neural network ankle fracture classification method according to claim 1, characterized in that: In step S4, the cross entropy loss function is calculated by using the distribution of p to approximate the distribution of q. The closer p and q are, the smaller the loss is. Among them, s is the ankle joint image data sample; l is the true label; The label distribution q is equivalent to a probability set; the true labels l and n in the sample c Belong to the same order of magnitude, q and l have different mathematical orders of magnitude; if the true label l and the predicted sample label n c If they are the same, the prediction is correct, which is 1, otherwise it is 0.

7. The guided improved neural network ankle fracture classification method according to claim 6, characterized in that: In step S5, the confusion matrix is ​​used to display the amount of data correctly and incorrectly classified by the classifier. The precision, recall, Fβscore and accuracy are used to measure the robustness of the model. The receiver operating characteristic curve is used to reflect the performance of the classifier. The horizontal axis is the false positive rate of each category, and the vertical axis is the true positive rate. The area under the ROC curve and the coordinate axis is calculated as the AUC value to characterize the classification performance.

8. The guided improved neural network ankle fracture classification method according to claim 1, characterized in that: In step S1, the collected images are randomly divided into a training set and a validation set in a ratio of 9:

1.

9. An ankle fracture classification system for use with the guided improved neural network ankle fracture classification method of claim 1, characterized in that: The guided improved neural network ankle fracture classification system includes a data acquisition unit, a segmentation network model training unit, an improved neural network classification model guidance unit, an improved neural network classification model training unit and an improved neural network classification model verification unit: The data acquisition unit acquires ankle fracture X-ray images and images with annotated ankle fibula regions and fracture types in the database and performs image enhancement; the acquired images are randomly divided into a training set and a validation set; The segmentation network model training unit uses the segmentation data of the annotated fibula region to train the segmentation network model, so that the segmentation network model can accurately identify the size and position of the fibula on the image; The improved neural network classification model guiding unit inputs the fracture type classification data into the trained segmentation network model, predicts the fibula region, and uses the prediction result as a guide for the improved neural network classification model; The improved neural network classification model training unit inputs the fracture type classification data into the improved neural network classification model guided by the prediction result for training; The improved neural network classification model verification unit verifies the classification result using a test set and evaluates the model using statistical indicators.

10. A guided improved neural network ankle fracture classification device, characterized in that: The device is equipped with a guided improved neural network ankle fracture classification system.

Images