Bone quantity recognition method and system based on three-dimensional convolutional neural network and CT image

Abstract

The application discloses a bone mass recognition method and system based on a three-dimensional convolutional neural network and a CT image, and comprises the following steps: acquiring an original CT image comprising a lumbar vertebra; obtaining a probability image and a lumbar vertebra part mask image according to the original CT image and a trained three-dimensional rough segmentation network model of the lumbar vertebra part; obtaining a lumbar vertebra part CT image according to the lumbar vertebra part mask image and the original CT image; obtaining a lumbar vertebra body mask image according to the probability image, the lumbar vertebra part CT image and a trained three-dimensional fine segmentation network model of the lumbar vertebra; obtaining a three-dimensional CT image of the lumbar vertebra body according to the lumbar vertebra body mask image and the original CT image; and obtaining a bone mass recognition result according to the three-dimensional CT image of the lumbar vertebra body and a trained classification network model. The application realizes accurate recognition of bone mass.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of bone mass recognition, and particularly relates to a bone mass recognition method and system based on a three-dimensional convolutional neural network and a CT image. BACKGROUND

[0002] The statements in this section merely provide background information related to the present application and do not necessarily constitute the prior art.

[0003] At present, the bone mineral density (BMD) is often evaluated based on a dual-energy X-ray absorptiometry (DXA). However, due to great differences in the acquisition of the DXA, many services fail to meet the international quality standards, so that the accuracy of the obtained bone mineral density cannot be guaranteed, and the measurement of the bone mineral density is limited due to the lack of DXA scanners in many areas.

[0004] In order to make up for the deficiency of the BMD obtained by the DXA, a method for measuring the average density of the lumbar vertebra in HU units by recognizing the CT image through a convolutional neural network to measure the BMD has appeared, for example, a diagnostic 2D CT slice at the pedicle level of the lumbar vertebra (L1) is selected in the 3D CT image, the convolutional neural network is used to segment and classify the horizontal plane CT slice to obtain the qualitative detection of the BMD; or, the convolutional neural network is used to segment and locate the sagittal plane lumbar vertebra of the CT image, and the transverse slice is extracted to regress the BMD value. It can be seen that the current CT image osteoporosis screening based on deep learning is performed on the CT slice, which may cause the loss of the three-dimensional CT image position and texture information, thereby affecting the accuracy of the prediction model and making the obtained bone mineral density result inaccurate. SUMMARY

[0005] In order to solve the above problems, the present application provides a bone mass recognition method and system based on a three-dimensional convolutional neural network and a CT image, which can obtain accurate bone mass recognition results and improve the accuracy of the bone mass recognition results.

[0006] To achieve the above object, the present application adopts the following technical solutions:

[0007] In a first aspect, a bone mass recognition method based on a three-dimensional convolutional neural network and a CT image is provided, comprising:

[0008] obtaining an original CT image including a lumbar vertebra;

[0009] obtaining a probability image and a lumbar vertebra part mask image according to the original CT image and a trained three-dimensional rough segmentation network model of the lumbar vertebra part;

[0010] obtaining a lumbar vertebra part CT image according to the lumbar vertebra part mask image and the original CT image;

[0011] obtain a lumbar vertebra body mask image according to the probability image, the lumbar vertebra part CT image and the trained lumbar vertebra three-dimensional fine segmentation network model;

[0012] obtain a lumbar vertebra body three-dimensional CT image according to the lumbar vertebra body mask image and the original CT image;

[0013] obtain a bone mass recognition result according to the lumbar vertebra body three-dimensional CT image and the trained classification network model.

[0014] In a second aspect, a bone mass recognition system based on a three-dimensional convolutional neural network and a CT image is provided, comprising:

[0015] an original CT image acquisition module configured to acquire an original CT image comprising a lumbar vertebra;

[0016] a probability image and a lumbar vertebra part mask image acquisition module configured to acquire a probability image and a lumbar vertebra part mask image according to the original CT image and a trained lumbar vertebra part three-dimensional coarse segmentation network model;

[0017] a lumbar vertebra part CT image acquisition module configured to acquire a lumbar vertebra part CT image according to the lumbar vertebra part mask image and the original CT image;

[0018] a lumbar vertebra body mask image acquisition module configured to acquire a lumbar vertebra body mask image according to the probability image, the lumbar vertebra part CT image and the trained lumbar vertebra three-dimensional fine segmentation network model;

[0019] a lumbar vertebra body three-dimensional CT image acquisition module configured to acquire a lumbar vertebra body three-dimensional CT image according to the lumbar vertebra body mask image and the original CT image;

[0020] a bone mass recognition module configured to acquire a bone mass recognition result according to the lumbar vertebra body three-dimensional CT image and the trained classification network model.

[0021] In a third aspect, an electronic device is provided, comprising a memory and a processor, and computer instructions stored in the memory and running on the processor, when the computer instructions are run by the processor, the steps of the bone mass recognition method based on a three-dimensional convolutional neural network and a CT image are completed.

[0022] In a fourth aspect, a computer readable storage medium is provided for storing computer instructions, when the computer instructions are executed by a processor, the steps of the bone mass recognition method based on a three-dimensional convolutional neural network and a CT image are completed.

[0023] Compared with the prior art, the present application has the following beneficial effects:

[0024] 1、The lumbar vertebra part in the original CT image is first roughly segmented to obtain a lumbar vertebra part mask image, and then a lumbar vertebra part CT image is obtained, the lumbar vertebra part CT image is finely segmented, and a lumbar vertebra mask image is obtained, so that accurate segmentation of the lumbar vertebra is realized, and when the lumbar vertebra three-dimensional CT image is identified, the bone mass recognition result can be accurately obtained, and the accuracy of bone mass recognition is improved.

[0025] 2、The attention module is introduced in the fine segmentation network model, the probability image obtained by rough segmentation is taken as the input of attention in fine segmentation, and when the lumbar vertebra mask image is extracted, the probability characteristics of the probability image are considered, the accuracy of the lumbar vertebra mask image segmentation is improved, and the problem of large visual field change range in the lumbar vertebra mask image extraction process can be well solved.

[0026] 3、When the bone mass recognition is carried out, the three-dimensional characteristics of the three-dimensional CT image are fully utilized, and the accuracy of the bone mass recognition is improved.

[0027] The advantages of the additional aspects of the application will be partially given in the following description, partially become obvious from the following description, or be known by the practice of the application. BRIEF DESCRIPTION OF DRAWINGS

[0028] The drawings accompanying the specification of this application form a part of this application and serve to further understand the application, the illustrative embodiments of the application and their descriptions are used to explain the application, and do not constitute improper limitation on the application.

[0029] Figure 1 The flowchart of the method disclosed in example 1 is disclosed;

[0030] Figure 2 The lumbar vertebra three-dimensional rough segmentation network model structure disclosed in example 1 is disclosed;

[0031] Figure 3 The lumbar vertebra three-dimensional fine segmentation network model structure disclosed in example 1 is disclosed;

[0032] Figure 4 The classification network model structure disclosed in example 1 is disclosed;

[0033] Figure 5 The attention module structure in the classification network model disclosed in example 1 is disclosed;

[0034] Figure 6 The segmentation result mentioned in example 1 is disclosed;

[0035] Figure 7 The ROC curve performance graph mentioned in example 1 is disclosed;

[0036] Figure 8 The pie chart schematic diagram disclosed in example 1 is disclosed. DETAILED DESCRIPTION

[0037] The application will be further described below in connection with the drawings and examples.

[0038] It should be noted that the following detailed description is illustrative only and is intended to provide further description of the application. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0039] Example 1

[0040] In this example, a bone mass recognition method based on a three-dimensional convolutional neural network and a CT image is disclosed, as shown in Figure 1 includes:

[0041] S1: Obtain an original CT image including lumbar vertebrae.

[0042] Many studies have shown that the average density of lumbar vertebrae measured in HU units obtained from CT images is significantly correlated with bone mineral density (BMD).

[0043] Therefore, the original CT image obtained in this example is an original CT image including L1-L4 lumbar vertebrae, and by recognizing the original CT image, a bone mass recognition result can be obtained.

[0044] S2: Obtain a probability image and a lumbar vertebrae mask image according to the original CT image and a trained lumbar vertebrae three-dimensional rough segmentation network model.

[0045] Because the CT image has a simple semantic and fixed structure, it is relatively simple compared to natural images, and both low-level and high-level semantic features are important, therefore in this example, a three-dimensional convolutional neural network based on U-Net architecture is used to construct a lumbar vertebrae three-dimensional rough segmentation network model, U-Net is a network model more suitable for the characteristics of medical image segmentation developed based on FCN, which is an encoder and a decoder with "symmetric" structure, the encoder and the decoder correspond to the down-sampling and up-sampling stages respectively. At the same time, the network uses the skip connection structure of the U-shaped structure to fuse shallow features and deep features, which has been proved to be able to train end-to-end with less data and achieve good results. Since the lumbar vertebrae target area only occupies a small part of the entire CT image, a spatial attention module is added in this example to suppress irrelevant areas and enhance the region of interest. At the same time, in order to better utilize the position information between the layers of the original CT image, this example uses a 3D convolution kernel for feature extraction, thereby improving the segmentation accuracy of the CT image.

[0046] The whole segmentation process is divided into two stages, first the rough segmentation of lumbar vertebrae is performed. Because the field of view of different CT images will have a large range of changes, and the input size of the three-dimensional rough segmentation network model of the lumbar vertebrae part is fixed, therefore before the segmentation of the lumbar vertebrae, the image with a large range of field of view changes is resized to a three-dimensional image with the same size and smaller resolution of 160x96x96, and the three-dimensional rough segmentation network model of the lumbar vertebrae part with a spatial attention module is used to perform rough segmentation of the lumbar vertebrae of the original CT image, and the lumbar vertebrae part mask image is obtained.

[0047] The structure of the three-dimensional rough segmentation network model of the lumbar vertebrae part is as shown in Figure 2 The original CT image is input, and the lumbar vertebrae part mask image and the probability image are output, which is obtained by constructing a three-dimensional convolutional neural network, which includes a downsampling module and an upsampling module, and a spatial attention module is added between the downsampling module and the upsampling module. The spatial attention module includes a convolution layer and a ReLu layer, the lumbar vertebrae part features output by the downsampling module are input into the convolution layer of the spatial attention module, the lumbar vertebrae part features are extracted by the convolution layer, and the feature map extracted from the lumbar vertebrae part features by the convolution layer is output after the ReLu layer. The attention feature map is multiplied by the feature map output by the downsampling module and input into the upsampling module.

[0048] The input of the downsampling module is the original CT image, which is used to extract the lumbar vertebrae part features of the lumbar vertebrae from the original CT image. The downsampling module includes four feature extraction layers, the first three feature extraction layers include two convolution layers and a max pooling layer, and the last feature extraction layer only includes two convolution layers without a pooling layer. The convolution layers of the downsampling module have 32, 64, 128 and 256 channels respectively, the convolution kernel size is 3x3x3, the convolution step is 1, and the activation function is ReLU. The pooling layer has 32, 64 and 128 channels respectively, the convolution kernel size is 2x2x2, and the convolution step is 2.

[0049] The up-sampling module aims to obtain the lumbar region mask image and the probability image from the lumbar region features obtained by the down-sampling module and output. The up-sampling module is composed of three feature extraction layers, each of which includes a transposed convolution layer and two convolution layers. The convolution layers have 32, 64 and 128 channels respectively, and the convolution kernel size is 3x3x3 and the convolution step is 1. The skip connection layer connects the corresponding encoder and decoder layer, fuses the bottom layer feature and the deep layer feature, and helps the decoder to better obtain the details of the lumbar region mask image. A spatial attention module is added between the down-sampling module and the up-sampling module. The spatial attention module includes a convolution layer and a ReLu layer. The down-sampling module outputs a lumbar region feature map of 20x12x12x256. After the spatial attention module, a convolution layer with a convolution kernel of 3x3x3 and a ReLu layer are used to obtain a spatial attention feature map. The spatial attention feature map is multiplied by the feature map output by the down-sampling module to perform spatial selection. The original CT image size is 160x96x96x1, and the output lumbar region mask image size is 160x96x96x5.

[0050] The lumbar region mask image contains L1 lumbar vertebrae, L2 lumbar vertebrae, L3 lumbar vertebrae, L4 lumbar vertebrae and background.

[0051] S3: According to the lumbar region mask image and the original CT image, the lumbar region CT image is obtained, and the process includes:

[0052] The size of the lumbar region mask image is adjusted to be the same as that of the original CT image.

[0053] The lumbar region mask image and the original CT image are cut and zero-filled together with the center of the lumbar body region in the lumbar region mask image as the center to obtain the lumbar region CT image.

[0054] In specific implementation, the lumbar region mask image obtained by rough positioning of the lumbar vertebrae is scaled to the size of the original CT image with a voxel size of 1x1x1 as the input image of the rough segmentation network model. Then, the center of the lumbar region containing L1-L4 lumbar vertebrae is taken as the center, and the lumbar region mask image and the original CT image are cut and zero-filled together to obtain a lumbar region CT image of 96x128x128.

[0055] S4: According to the probability image, the lumbar region CT image and the trained lumbar three-dimensional fine segmentation network model, a lumbar body mask image is obtained.

[0056] The lumbar vertebra three-dimensional fine segmentation network model takes a probability image and a lumbar vertebra part CT image as input, and outputs a lumbar vertebra mask image, and is obtained by constructing a three-dimensional convolutional neural network, the three-dimensional convolutional neural network includes a down-sampling module and an up-sampling module, the down-sampling module extracts lumbar vertebra body features from the lumbar vertebra part CT image, and the up-sampling module outputs the lumbar vertebra mask image according to the lumbar vertebra body features output by the up-sampling module, an attention module is added to the down-sampling module, the attention module extracts probability features from the probability image by using multiple convolutional layers, the number of convolutional layers of the attention module is one less than the number of feature extraction layers of the down-sampling module, and the output of each convolutional layer in the attention module corresponds to the input of the remaining feature extraction layers of the down-sampling module except the first layer of feature extraction layers, and each feature extraction layer of the down-sampling module except the first layer of feature extraction layers outputs the lumbar vertebra body features of the layer according to the input probability features and the output of the last layer of feature extraction layers.

[0057] The lumbar vertebra three-dimensional fine segmentation network model structure is as shown in Figure 3 The lumbar vertebra three-dimensional fine segmentation network model structure is as shown in Figure 3 The lumbar vertebra three-dimensional fine segmentation network model structure is as shown in

[0058] The number of channels of the convolutional layer in the attention module is the same as the number of channels of the feature extraction layer in the down-sampling module. Let m be the number of layers of the feature extraction layer, {F i1 ,F i2 ,…,F i() ,F in} be the features of the i-th (i∈[1,m]) channel feature extraction layer, n be the number of features of the i-th channel feature extraction layer, {A i1 ,A i2 ,…,A i() ,A in} be the probability features extracted by the i-th layer convolutional layer of the attention module. The attention module is to provide attention mechanism for feature extraction in the down-sampling module, so the output Out il of the i-th channel feature extraction layer is:

[0059] Out il =Sigmoid(A il )*F il ,l∈[1,n]

[0060] In the formula, * is a matrix dot product operation.

[0061] The lumbar vertebrae mask image output by the lumbar vertebrae three-dimensional fine segmentation network model has the same size as the input lumbar vertebrae CT image.

[0062] Both the lumbar vertebrae three-dimensional coarse segmentation network model and the lumbar vertebrae three-dimensional fine segmentation network model use the GDL loss function.

[0063] The training process of the lumbar vertebrae three-dimensional coarse segmentation network model and the lumbar vertebrae three-dimensional fine segmentation network model is as follows:

[0064] Obtain the original CT image containing the lumbar vertebrae and the corresponding DXA data for training.

[0065] The training CT image can be a CT image obtained by examining the lumbar vertebrae or the abdomen, and the training original CT image is clear in imaging, without postoperative metal or bone cement implant, the CT image and the corresponding DXA data come from the same hospital, and the time difference between obtaining the DXA data and the CT image is less than 1 week.

[0066] The DXA data is obtained by scanning L1 to L4 lumbar vertebrae by a DXA scanner. The scanning is performed by a trained technician and evaluated by an experienced doctor, and the bone parts affected by severe local structural changes or artificial products are excluded to ensure the quality of the obtained DXA data.

[0067] In order to reduce the time consumption of manual annotation, and the visibility and reliability of all relevant lumbar vertebrae (L1-L4) in CT images, a method combining machine labeling and manual labeling is used to label the lumbar region and lumbar vertebrae. Specifically, first, the SpatialConfiguration-Net and U-Net are used to locate and segment the vertebrae from coarse to fine, and the training original CT images are labeled with the lumbar region containing L1-L4 lumbar vertebrae and the L1-L4 lumbar vertebrae. Then, ITK-SNAP is used to manually modify and correct the labeled L1-L4 lumbar vertebrae, remove the lumbar arch part, and correct the lumbar vertebrae. In order to enable the network model to directly identify the four different vertebrae, in the segmentation mask of each network model, the labeled area of L1 lumbar vertebrae is filled with "1", the labeled area of L2 lumbar vertebrae is filled with "2", the labeled area of L3 lumbar vertebrae is filled with "3", and the labeled area of L4 lumbar vertebrae is filled with "4", and the other areas are filled with "0".

[0068] In order to improve the training effect of the model, the training CT images need to be preprocessed, which includes: to solve the problem that the actual space size represented by a single voxel in different images is inconsistent, all training original CT images and mask images are resampled to [1, 1, 1]; the resampled CT images are processed using window technology to enhance the features of the lumbar vertebrae, and the window level (WL) and window width (WW) used are [350, 1000]; using the min-max normalization method, all pixel values in the CT images with enhanced lumbar vertebrae features are scaled to [0, 1] to obtain preprocessed CT images and labeled images.

[0069] The preprocessed CT images and labeled images are scaled to form coarse segmentation training data.

[0070] The coarse segmentation training data is input into the constructed lumbar region three-dimensional coarse segmentation network model, and the lumbar region three-dimensional coarse segmentation network model is trained. During the training process, the training lumbar region mask image and the training probability image are output, and after the training is completed, the trained lumbar region three-dimensional coarse segmentation network model is obtained.

[0071] Taking the center of each lumbar vertebrae target region in the training lumbar region mask image as the center, the training lumbar region probability image, the preprocessed CT image and the mask image are cut and zero-filled to obtain the fine segmentation training lumbar region probability image, the CT image and the mask image.

[0072] The lumbar vertebra part CT image for training, the mask image and the training probability image are input into the constructed lumbar vertebra three-dimensional fine segmentation network model, the lumbar vertebra three-dimensional fine segmentation network model is trained, and the trained lumbar vertebra three-dimensional fine segmentation network model is obtained after the training is completed.

[0073] The lumbar vertebra three-dimensional coarse segmentation network model and the lumbar vertebra three-dimensional fine segmentation network model are both trained by using an adaptive estimated stochastic gradient descent method, in the training process, the learning rate is fixed at 0.0001, the training process has 50 epochs, the momentum value is set to 0.9, and the batch size is 2.

[0074] In the model training process, because the segmented lumbar vertebra part target region accounts for a small part of the entire CT image, the imbalance between the foreground and the background will greatly affect the performance of the model training. Therefore, the lumbar vertebra three-dimensional coarse segmentation network model and the lumbar vertebra three-dimensional fine segmentation network model both adopt a GDL loss function, and the GDL formula is as follows, wherein r ln is a standard value (GT) of the class l at the nth pixel, p ln is a corresponding predicted probability value, w l is a weight of each class, and provides invariance for different label set attributes, and is defined as follows, and the contribution between each label will be corrected by the inverse of the square of its volume.

[0075]

[0076]

[0077] The performance of the proposed segmentation network model is also evaluated by using a quantitative index. Specifically, for the segmentation result, the dice index (DICE) is selected to test the performance of the segmentation model. The definition of the DICE coefficient is as follows, wherein TPP represents the number of pixels correctly segmented into the lumbar vertebra, FPP represents the number of pixels incorrectly segmented into the lumbar vertebra, and FNP represents the number of pixels incorrectly segmented into the background.

[0078]

[0079] For the segmentation result obtained in the embodiment, the DICE of each class is calculated, and the average DICE coefficient is taken as the average DICE coefficient of L1-L4.

[0080] S5: Obtain a lumbar vertebra body three-dimensional CT image according to the lumbar vertebra body mask image and the original CT image.

[0081] The process of obtaining the lumbar vertebra body three-dimensional CT image is as follows:

[0082] Determine the center of each vertebra body in the lumbar vertebra body mask image;

[0083] Taking each vertebral body center as the center, a set size image is cut on the original CT image and the lumbar vertebral body mask image respectively;

[0084] The image cut from the lumbar vertebral body mask image is subjected to binaryzation processing;

[0085] Each binaryzation-processed image and the corresponding image cut from the original CT image are multiplied to obtain a three-dimensional vertebral body CT image of each lumbar vertebral body;

[0086] All three-dimensional vertebral body CT images are spliced to obtain a lumbar vertebral body three-dimensional CT image.

[0087] In specific implementation, the skimage.measure._regionprops.regionprops function is used to analyze the lumbar vertebral body mask image to obtain the center coordinates of the vertebral bodies of L1-L4 lumbar vertebral bodies. A 48x96x96 size image is cut on the CT original image and the lumbar vertebral body mask image after segmentation preprocessing (the resampling size is changed to [0.7, 0.5, 0.5] in the preprocessing process), the binaryzation-processed mask image is multiplied with the cut CT image to obtain a three-dimensional vertebral body CT image of L1-L4 lumbar vertebral bodies, and the L1-L4 vertebral body CT images are spliced to form a lumbar vertebral body three-dimensional CT image with a size of 192x96x96 as an input of a classification network model.

[0088] S6: obtaining a bone mass recognition result according to the lumbar vertebral body three-dimensional CT image and the trained classification network model.

[0089] The classification network model comprises, in sequence, a convolution layer, a pooling layer, four densely connected blocks, a global average pooling layer and a classifier, a conversion layer is added between adjacent densely connected blocks, a space-channel attention module is added between the last densely connected block and the conversion layer, the convolution layer takes the lumbar vertebral body three-dimensional CT image as an input, and the classifier takes the bone mass recognition result as an output.

[0090] In the image classification task, because the scene of the lumbar vertebra three-dimensional CT image is single, the structure is fixed, and the similarity between different samples is relatively high, both high-level semantic features and low-level semantic features are very important, therefore, when applying CNN to extract image semantic features, the network should be deep enough to reduce the resolution of the feature map to obtain abstract features, and the low-level semantic features and high-level semantic features need to be combined to use multi-scale information for prediction. Due to the sensitivity of private information and the demand for accurate and professional labels of radiologists, the professional lumbar profile and DXA data are relatively small compared to natural images, in order to prevent the network from overfitting, the designed CNN should have relatively few parameters. And from 2D network to 3D network, the parameters and calculation of convolutional neural network will increase exponentially, therefore, the classification network model of the embodiment adopts the dense connection mechanism of DenseNet, which can combine feature maps of different layers to realize feature reuse, thereby reducing the use of parameters. At the same time, in order to reasonably use the limited visual information processing resources, a simple channel and spatial attention module is added to achieve performance gain with a small amount of parameters. The embodiment adopts a three-dimensional dense connection convolutional neural network based on attention mechanism to perform bone mass recognition, and the structure of the classification network model is as shown in Figure 4 The classification network model is composed of a convolutional layer and a pooling layer, four dense connection blocks, three conversion layers, and a spatial-channel attention module. Because the CT value is extremely important information for bone mass recognition, and the maximum pooling reduces the sampling of the input by obtaining the maximum value of each channel of the input, which will lose a lot of CT value information, therefore, the average pooling layer is more suitable for medical image disease classification, so the embodiment uses an average pooling layer with a convolution kernel size of 3x3x3 and a convolution step of 2 after the convolution layer with a convolution kernel size of 7x7x7 and a convolution step of 2. The number of dense connection blocks is set to 6, 12, 24 and 16 respectively, and the growth rate k is 12, in order to reduce the number of training parameters, and a convolution layer with a 1x1x1 convolution kernel is added as a bottleneck layer in the dense block to optimize the calculation efficiency. The compression coefficient of the conversion layer is set to 0.5, thereby playing a role in compressing the model to reduce the parameters. A spatial-channel attention module is added before the fourth dense connection block, and the attention module includes a channel attention module and a spatial attention module, as shown in Figure 5 .

[0091] The space-channel attention module is composed of a channel attention module and a space attention module. The channel attention module adopts an ECA attention module. The input of the ECA attention module is the output feature map of the third dense connection block. The channel attention feature is obtained after the output feature map is subjected to a global average pooling layer, a convolution layer and an activation layer. The channel attention feature is multiplied by the input feature map of the ECA attention module to obtain the output feature of the ECA attention module. In the space attention module, the feature map of the input space attention module is subjected to a convolution layer and a ReLu layer to output a space attention feature. The space attention feature is multiplied by the feature map of the input space attention module to obtain the output feature of the space-channel attention module.

[0092] In the space attention module, the convolution kernel size of the convolution layer is 3x3x3, the convolution kernel size of the convolution layer in the ECA attention module is k, the convolution step and the output channel are both 1, and the activation layer is a Sigmoid activation layer. The size of the convolution kernel k is related to the channel number C, and the function is as follows, odd It is indicated that k can only take an odd number; γ and b are set to 2 and 1, respectively, to change the ratio between the channel number C and the convolution kernel size.

[0093]

[0094] The global average pooling and the softmax classifier are executed at the end of the last dense block. In this embodiment, the output bone mass recognition result can be normal bone mass, low bone mass and osteoporosis. Therefore, the output of the final classification layer is set to 3. The output of the classification network model is a probability value of three different categories. According to the probability value, the bone mass recognition result can be obtained.

[0095] The classification network model is trained by using the SGD stochastic gradient descent method, the batch size is set to 32, and the iteration number is 50. The initial learning rate is set to 0.01, and the exponential decay function learning strategy is used as follows, wherein l is the initial learning rate, n is the training number, L is the updated learning rate, and α is the decay rate set to 0.96.

[0096]

[0097] The classification network model of this embodiment is a three-classification, and a cross-entropy loss function is used, and the function formula is as follows:

[0098]

[0099] In the formula, K is the number of categories; y is the label, that is, if the category is i, y i= 1, otherwise equals 0; p is the output of the neural network, i.e. the probability that the class is i, this output is the result of the softmax.

[0100] For the final classification results, the accuracy, precision, recall and area under curve (AUC) are selected to evaluate the performance of the classification network model. The definitions are as follows, where TP represents true positive samples, FP represents false positive samples, TN represents true negative samples, and FN represents false negative samples.

[0101]

[0102]

[0103]

[0104]

[0105] Accuracy refers to the proportion of true prediction samples in the total classification samples; precision represents the proportion of true positive prediction samples in the total prediction positive samples; recall represents the proportion of true positive prediction samples in the total classification positive; ROC curve uses true positive rate (recall) as the vertical axis and false positive rate (1-specificity) as the horizontal axis. The ROC curve balances sensitivity and specificity and is a comprehensive evaluation index. AUC is the area in the lower right corner of the ROC curve, and the larger the area means the better the performance of the classifier. For this embodiment, the accuracy is measured globally, and the precision, recall and area under curve are calculated for each class. Because this embodiment uses the Macro-average method, the evaluation indexes (Precision / Recall / AUC) of different classes are directly averaged, and all classes are given the same weight.

[0106] Because the number of training classification images is small, data augmentation is needed to expand the data set to prevent overfitting of the classification network model, and to keep the size of the voxel space unchanged, random rotation, random translation, and random combination of L1-L4 single vertebra CT images of different individuals of the same class are used to augment the training data.

[0107] 618 individuals aged 22 to 98 years old, with an average of 60 years old, are selected to verify the method disclosed in this embodiment. Table 1 lists the data characteristics obtained. Among them, the number of individuals with osteoporosis is n = 204, accounting for 33.0%, the number of individuals with osteopenia is n = 188, accounting for 30.4%, and the number of individuals with normal is n = 226, accounting for 36.6%.

[0108] Table 1 Summary of characteristics of experimental data

[0109]

[0110] The two-stage coarse-to-fine lumbar vertebra segmentation framework proposed in this embodiment first uses a coarse segmentation network model to obtain an initial probability image of the lumbar vertebra five-class probability distribution, and then uses the probability image as the attention input of the second-stage fine segmentation network model to obtain the final lumbar vertebra mask image. Therefore, the coarse segmentation network is used as a benchmark network to compare with the segmentation model of this embodiment, and the results are shown in Table 2. The performance of the method proposed in this embodiment on lumbar vertebra segmentation is higher than that of the benchmark network, that is, the three-dimensional Unet segmentation network with only spatial attention. In addition, in order to better compare, a two-class classification of all lumbar vertebrae using the benchmark network is proposed, and then the lumbar vertebra region of interest is extracted using the two-class classification result, and finally the same benchmark network is used for five-class segmentation of the CT image of the region of interest. From the results in Table 2, after simply reducing the image, the L1-L4 segmentation DICE coefficient has a certain improvement compared with the single benchmark network, but the improvement effect is not as obvious as the two-stage network with probability distribution as attention input. Figure 6 The sagittal view of three-dimensional CT images (CT sagittal images), manual annotation segmentation results (Manual segmentation) and examples of two-stage coarse-to-fine automatic segmentation results proposed in this embodiment (Automated segmentation) are shown.

[0111] Table 2 Segmentation model results

[0112]

[0113] The classification network model proposed in this embodiment performs well in the test set, with a total classification accuracy (Accuracy) of 0.7915, and the classification performance of each class is shown in Table 3, and the ROC curve performance of normal bone mass, low bone mass and osteoporosis is shown in Figure 7 , wherein Normal bone mass is normal bone mass; Low bone mass is bone mass reduction; Osteoporosis is osteoporosis.

[0114] Table 3 Classification network model classification performance for different BMD categories

[0115]

[0116] The classification network model disclosed in the embodiment is compared with the original DenseNet-121 to verify the necessity of reducing network parameters. As shown in Table 4, the classification network model disclosed in the embodiment has better performance than DenseNet-121 in terms of AUC. At the same time, the influence of different pooling functions on the classification performance is also compared. As shown in Table 4, when the average pool function is used, the classification performance is improved, and the possible reason is that the maximum pooling causes the loss of CT value information. In addition, in order to further evaluate the effectiveness of the embedded attention module, the classification network model with and without the attention module is compared, and the embedded attention module can slightly improve the classification performance. Experiments show that the network combined with the attention mechanism further improves the discrimination ability of the network, which benefits from the role of the attention module in feature selection and spatial selection.

[0117] Table 4 Comparison of classification performance of different networks

[0118]

[0119] The use of only lumbar cancellous bone and the entire lumbar vertebral body as classification input images is also compared, wherein the cancellous bone mask image is obtained by performing a 5mm erosion operation on the substantial bone lumbar mask image. As shown in Table 5, the lumbar vertebral body containing cortical bone as the classification image can obtain superior performance, and the possible reason is that the deep learning feature extraction induction ability is strong, which can comprehensively utilize the cancellous bone and substantial bone features, and the osteoporotic bone also appears the features such as cortical bone thinning, morphological changes of vertebral endplate and other structures on the cortical bone. Therefore, for the lumbar vertebral body three-dimensional CT image, the complete lumbar vertebral body image can obtain better classification effect.

[0120] Table 5 Comparison of different input images

[0121]

[0122] In order to display the BMD type judgment more intuitively, the method of the embodiment also outputs the BMD type judgment and the BMD type probability pie chart, as shown in Figure 8 The results displayed above each pie chart represent the bone mass recognition result, and the pie chart represents the prediction probability. The probability of low bone mass in the second column is not much different from the probability of osteoporosis, and there is also a certain risk of osteoporosis. Therefore, the probability chart display method is more intuitive for the display of bone mass.

[0123] The method disclosed by the embodiment first performs coarse segmentation on a lumbar vertebra part in an original CT image to obtain a lumbar vertebra part mask image, and then obtains a lumbar vertebra part CT image, performs fine segmentation on the lumbar vertebra part CT image, and obtains a lumbar vertebra mask image, so that accurate segmentation of the lumbar vertebra is realized. When the lumbar vertebra three-dimensional CT image is identified, the bone mass recognition result can be accurately obtained, and the accuracy of bone mass recognition is improved. In the fine segmentation network model, the attention module is introduced, the probability image obtained by coarse segmentation is taken as the input of attention in fine segmentation, and the probability characteristics of the probability image are considered when the lumbar vertebra mask image is extracted, so that the accuracy of the lumbar vertebra mask image segmentation is improved, and the problem of large range of visual field change in the lumbar vertebra mask image extraction process can be well solved.

[0124] When the bone mass recognition is performed, the three-dimensional characteristics of the three-dimensional CT image are fully utilized, and the accuracy of the bone mass recognition is improved.

[0125] Embodiment 2

[0126] In this embodiment, a bone mass recognition system based on a three-dimensional convolutional neural network and a CT image is disclosed, comprising:

[0127] An original CT image acquisition module is configured to acquire an original CT image comprising a lumbar vertebra;

[0128] A probability image and a lumbar vertebra part mask image acquisition module is configured to acquire a probability image and a lumbar vertebra part mask image according to the original CT image and a trained lumbar vertebra part three-dimensional coarse segmentation network model;

[0129] A lumbar vertebra part CT image acquisition module is configured to acquire a lumbar vertebra part CT image according to the lumbar vertebra part mask image and the original CT image;

[0130] A lumbar vertebra mask image acquisition module is configured to acquire a probability image and a lumbar vertebra mask image according to the probability image, the lumbar vertebra part CT image and a trained lumbar vertebra three-dimensional fine segmentation network model;

[0131] A lumbar vertebra three-dimensional CT image acquisition module is configured to acquire a lumbar vertebra three-dimensional CT image according to the lumbar vertebra mask image and the original CT image;

[0132] A bone mass recognition module is configured to acquire a bone mass recognition result according to the lumbar vertebra three-dimensional CT image and a trained classification network model.

[0133] Embodiment 3

[0134] In this embodiment, an electronic device is disclosed, comprising a memory and a processor, and computer instructions stored on the memory and running on the processor, when the computer instructions are run by the processor, the steps of the bone mass recognition method based on the three-dimensional convolutional neural network and the CT image disclosed in embodiment 1 are completed.

[0135] Embodiment 4

[0136] In this embodiment, a computer readable storage medium is disclosed, for storing computer instructions, when the computer instructions are executed by the processor, the steps of the bone mass recognition method based on the three-dimensional convolutional neural network and the CT image disclosed in embodiment 1 are completed.

[0137] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application rather than limit it, although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that: the specific embodiments of the present application can still be modified or replaced by the equivalent, without departing from the spirit and scope of the present application, any modification or equivalent replacement, which should be covered within the protection scope of the claims of the present application.

Claims

1. A bone mass recognition method based on a three-dimensional convolutional neural network and a CT image, characterized by, The application relates to a method for recognizing bone mass of a lumbar vertebra. The method comprises the following steps: acquiring an original CT image of a lumbar vertebra; According to the original CT image and the trained three-dimensional rough segmentation network model of the lumbar vertebra part, a probability image and a lumbar vertebra part mask image are obtained; According to the lumbar vertebra part mask image and the original CT image, a lumbar vertebra part CT image is obtained; According to the probability image, the lumbar vertebra part CT image and the trained three-dimensional fine segmentation network model of the lumbar vertebra, a lumbar vertebra body mask image is obtained; The three-dimensional fine segmentation network model of the lumbar vertebra takes the probability image and the lumbar vertebra part CT image as input and takes the lumbar vertebra body mask image as output, is obtained by constructing a three-dimensional convolutional neural network, the three-dimensional convolutional neural network comprises a down-sampling module and an up-sampling module, the down-sampling module extracts the lumbar vertebra body features from the lumbar vertebra part CT image, an attention module is added to the down-sampling module, the attention module extracts probability features from the probability image by using multiple convolutional layers, the number of convolutional layers of the attention module is one less than that of the feature extraction layers of the down-sampling module, the output of each convolutional layer in the attention module corresponds to the input of the remaining feature extraction layers of the down-sampling module except the first layer of feature extraction layers, and each feature extraction layer of the down-sampling module except the first layer of feature extraction layers outputs the lumbar vertebra body features of the layer according to the input probability features and the output of the last layer of feature extraction layers; According to the lumbar vertebra body mask image and the original CT image, a three-dimensional CT image of the lumbar vertebra body is obtained; According to the three-dimensional CT image of the lumbar vertebra body and the trained classification network model, a bone mass recognition result is obtained. 2.The bone amount recognition method based on a three-dimensional convolutional neural network and a CT image according to claim 1, wherein, The three-dimensional rough segmentation network model of the lumbar vertebra part takes the original CT image as input and takes the lumbar vertebra part mask image and the probability image as output, is obtained by constructing a three-dimensional convolutional neural network, the three-dimensional convolutional neural network comprises a down-sampling module and an up-sampling module, and a spatial attention module is added between the down-sampling module and the up-sampling module. 3.The bone amount recognition method based on a three-dimensional convolutional neural network and a CT image according to claim 1, wherein, The process of obtaining the lumbar vertebra part CT image is as follows: The size of the lumbar vertebra part mask image is adjusted to be the same as that of the original CT image; The center of the lumbar vertebra body region in the lumbar vertebra part mask image is taken as the center, and the lumbar vertebra part mask image and the original CT image are subjected to a cut-and-zero operation together, so that the lumbar vertebra part CT image is obtained. 4.The bone amount recognition method based on a three-dimensional convolutional neural network and a CT image according to claim 1, wherein, The process of obtaining the three-dimensional CT image of the lumbar vertebra body is as follows: The center of each vertebral body in the lumbar vertebra body mask image is determined; The center of each vertebral body in the lumbar vertebra body mask image is determined; Each binary processed image and the corresponding image cut from the original CT image are multiplied, so that a three-dimensional vertebral body CT image of each lumbar vertebra body is obtained; All the three-dimensional vertebral body CT images are spliced to obtain the three-dimensional CT image of the lumbar vertebra body. The classification network model comprises convolutional layers, pooling layers, four densely connected blocks, a global average pooling layer and a classifier which are connected in sequence, a conversion layer is added between adjacent densely connected blocks, an attention module is added between the last densely connected block and the conversion layer, the convolutional layers take the three-dimensional CT image of the lumbar vertebra body as input, and the classifier takes the bone mass recognition result as output. 5.The bone amount recognition method based on a three-dimensional convolutional neural network and a CT image according to claim 1, wherein, The three-dimensional rough segmentation network model of the lumbar vertebra part and the three-dimensional fine segmentation network model of the lumbar vertebra both adopt a GDL loss function. 6.The bone amount recognition method based on a three-dimensional convolutional neural network and a CT image according to claim 1, wherein, The application relates to a method for recognizing bone mass of a lumbar vertebra.

7. A bone volume recognition system based on a three-dimensional convolutional neural network and CT images, which applies the bone volume recognition method based on a three-dimensional convolutional neural network and CT images according to any one of claims 1 to 6, characterized by, ​ An original CT image acquisition module is configured to acquire an original CT image including a lumbar vertebra; A probability image and a lumbar vertebra part mask image acquisition module is configured to acquire a probability image and a lumbar vertebra part mask image according to the original CT image and a trained three-dimensional coarse segmentation network model of a lumbar vertebra part; A lumbar vertebra part CT image acquisition module is configured to acquire a lumbar vertebra part CT image according to the lumbar vertebra part mask image and the original CT image; A lumbar vertebra body mask image acquisition module is configured to acquire a lumbar vertebra body mask image according to the probability image, the lumbar vertebra part CT image and a trained three-dimensional fine segmentation network model of a lumbar vertebra; A lumbar vertebra body three-dimensional CT image acquisition module is configured to acquire a lumbar vertebra body three-dimensional CT image according to the lumbar vertebra body mask image and the original CT image; A bone mass recognition module is configured to acquire a bone mass recognition result according to the lumbar vertebra body three-dimensional CT image and a trained classification network model.

8. An electronic device, comprising: A computer readable storage medium, comprising a memory and a processor, and computer instructions stored in the memory and run on the processor, when the computer instructions are run by the processor, the steps of the bone mass recognition method based on a three-dimensional convolutional neural network and a CT image according to any one of claims 1-6 are completed.

9. A computer-readable storage medium, characterized in that, A computer readable storage medium for storing computer instructions, when the computer instructions are executed by a processor, the steps of the bone mass recognition method based on a three-dimensional convolutional neural network and a CT image according to any one of claims 1-6 are completed.

Images