Musculoskeletal X-ray image analysis method and system and medium

Abstract

The invention provides a musculoskeletal X-ray image analysis method, which comprises the steps of converting an initial judgment result of a musculoskeletal X-ray image to be analyzed into a one-dimensional tensor, and obtaining a group of prediction probability values Prob1 through probabilistic processing; converting the pixel matrix of the image data of the musculoskeletal X-ray image to be analyzed into a three-dimensional tensor representing the number, height and width of image channels; constructing a DenseNet121 model, and inputting the three-dimensional tensor into the trained DenseNet121 model to obtain a group of first image feature representations; constructing a ResNet50 model, and inputting the three-dimensional tensor into the trained ResNet50 model to obtain a group of second image feature representations; and splicing the prediction probability value Prob1, the first image feature representation and the second image feature representation to obtain feature data for representing the musculoskeletal state. According to the method, a mixed judgment model in which human participates in deep learning is designed, and doctors can be accurately assisted in judging musculoskeletal states.

Description

Technical Field

[0001] The present invention relates to the field of medical imaging, and particularly to a method, system and medium for analyzing musculoskeletal X-ray images. Background Art

[0002] Musculoskeletal (MSK) diseases are a group of diseases involving muscles, bones, soft tissues, joints and the spine. MSK diseases can occur at different stages of the human life cycle, with risks from infancy to old age, including a variety of pathological conditions such as fractures, arthritis, ligament injuries, muscle injuries, osteoporosis, spinal degenerative diseases, etc., which have an adverse impact on the quality of life of patients, including pain, dysfunction, etc. X-ray images play an important role in MSK diagnosis. The accurate interpretation of X-ray images requires clinicians to have good film reading experience and professional knowledge, and misdiagnosis or delayed diagnosis will affect the treatment and rehabilitation process of patients, and may even endanger the lives of patients.

[0003] Traditional machine learning algorithms usually involve two steps in medical image interpretation: image feature extraction and image recognition. It is necessary to manually extract image features, and the extraction of these features depends on expert knowledge and experience, and may not be able to fully express the complex information in medical images. Limited by the finiteness and insufficiency of data, its recognition accuracy and efficiency are usually low. In contrast, deep learning algorithms can automatically learn the image feature representation in medical image data, thus improving the accuracy and efficiency of auxiliary judgment. In recent years, deep learning algorithms using deep convolutional neural networks have been widely tested and applied in medical image interpretation and have made remarkable progress. For example, Hu Jinbo, Nie Weizhi, Song Dan et al. proposed a deformable Transformer-assisted chest X-ray image disease judgment model by using the extended ResNet50 as the feature extraction network, and conducted multiple groups of experiments on the datasets ChestX-Ray14 and CheXpert. The AUC values of their models reached 0.8398 and 0.9061 respectively.

[0004] Since medical image data may be affected by various factors, such as noise, motion artifacts, class imbalance, low image fineness, etc., this has imposed some limitations and challenges on the application of deep learning models in artificial intelligence-assisted judgment, and the models of the existing technologies still have the problem of inaccurate feature extraction of image data. Summary of the Invention

[0005] Aiming at the deficiencies of the prior art, the present invention provides a method and system for analyzing musculoskeletal X-ray images. The above method helps to realize the analysis of musculoskeletal X-ray images.

[0006] The method for analyzing musculoskeletal X-ray images provided by this application includes:

[0007] Convert the preliminary judgment result of the musculoskeletal X-ray image to be analyzed into a one-dimensional tensor, and through probabilistic processing, obtain a set of predicted probability values Prob1;

[0008] Convert the pixel matrix of the image data of the musculoskeletal X-ray image to be analyzed into a three-dimensional tensor representing the number of image channels, height, and width;

[0009] Construct a DenseNet121 model, input the three-dimensional tensor into the trained DenseNet121 model, and obtain a set of first image feature representations;

[0010] Construct a ResNet50 model, input the three-dimensional tensor into the trained ResNet50 model, and obtain a set of second image feature representations;

[0011] Concatenate the predicted probability values Prob1, the first image feature representation, and the second image feature representation to obtain feature data for characterizing the musculoskeletal state.

[0012] Preferably, the probabilistic processing includes:

[0013] Obtain the one-dimensional tensor data and perform preset data processing;

[0014] Obtain the preset data processing result, calculate through the softmax function on the second dimension, and obtain a set of predicted probability values Prob1.

[0015] Preferably, the training of the DenseNet121 model includes:

[0016] Set the output of the fully connected layer of the DenseNet121 model to 2;

[0017] The forward propagation formula of DenseNet121 during training is:

[0018] x0 = input

[0019] x l = H l ([x0, x1,..., x l-1 )

[0020] where x0 is the input data, x l represents the output of the l-th layer, H i represents the non-linear transformation of the l-th layer, [x0, x1,....x l-1 represents concatenating the outputs of the 0-th layer to the l-1-th layer as the input of the l-th layer. For the first layer of DenseNet121, its input and output are:

[0021] x0 = input

[0022] x l = H l ([x0, x1, ..., x l-1 )

[0023] x i = H i ([x0, x1, ..., x i-1 ) 1≤i≤l-1

[0024] Among them, H i represents the convolutional layer and non-linear transformation in the i-th densely connected block, and [x0, x1, ..., x i-1 represents the concatenation of the outputs of the previous i - 1 layers as the input of the i-th densely connected block. The backpropagation formula of DenseNet121 is:

[0025]

[0026] Among them, L represents the loss function.

[0027] Preferably, the training of the ResNet50 model includes:

[0028] Set the output of the fully connected layer of the ResNet50 model to 2;

[0029] The forward pass of the ResNet50 model is:

[0030] Y = f n (f n-1 (...f3(f2(f1(X)))))

[0031] Among them, X is the input image, Y is the final output, and each convolutional layer, pooling layer, fully connected layer, etc. can be regarded as a function, represented by f1, f2, f3, ..., f n ;

[0032] Use cross-entropy as the loss function. Assume the training dataset is D, and the cross-entropy loss function L is:

[0033]

[0034] Among them, y i represents the actual label, represents the prediction result;

[0035] The overall calculation formula of the loss function is:

[0036]

[0037] Among them, n represents the total number of samples in the training set, Li Represents the cross-entropy loss function for each sample;

[0038] For the j-th convolutional layer of the i-th layer, the output feature map is F i，j , and the input feature map is F i-1，k , then the gradient calculation formula for this convolutional layer is:

[0039]

[0040] Among them, is the gradient of the previous layer, is the derivative of the convolutional layer. By using the above method to obtain the gradient calculation formulas for the batch normalization layer and the fully connected layer, the training model of ResNet50 is:

[0041]

[0042] Among them, θ represents the parameters of the model.

[0043] Preferably, the database used for training the DenseNet121 model or the ResNet50 model is the MURA dataset.

[0044] Preferably, the training of the DenseNet121 model or the ResNet50 model further includes minimizing the loss function using the Adam optimization algorithm. The formula of the Adam optimization algorithm is as follows:

[0045] m t =β1m t-1 +(1-β1)g t

[0046]

[0047]

[0048]

[0049]

[0050] Among them, m t and v t respectively represent the first-order and second-order momentum estimation values, β1 and β1 are the decay rates, g t is the current gradient, g t and are the bias corrections for momentum estimation, θ is the learning rate, and ∈ is an extremely small number.

[0051] Preferably, the feature data used to characterize the musculoskeletal state is input into the Naive Bayes model to obtain the recognition result of musculoskeletal X-ray images:

[0052] Obtain the predicted probability value Prob1, the first image feature representation, and the second image feature representation, and concatenate the three to form a feature vector X;

[0053] Input the feature vector X into the Naive Bayes model;

[0054] Use Bayes' formula to calculate the probability of each state classification label in the state result, and select the state classification label with the highest probability as the final state recognition result; where Bayes' formula is:

[0055]

[0056] Where represents the predicted value of class "Y", represents the value of "c" that makes the expression reach the maximum value, P(Y = c) represents the probability of the state classification label "c", represents the product of the conditional probabilities of each state classification x j under the given state classification label "c", where d is the number of feature vectors.

[0057] Preferably, before using Bayes' formula to calculate the probability of each state classification label in the state result, obtain the prior probability and conditional probability through the maximum likelihood estimation method, where the maximum likelihood estimation method includes:

[0058] According to the training data set, estimate the prior probability P(Y) of each state classification label:

[0059]

[0060] Where, N represents the size of the training data set, y i represents the true classification label of the i-th sample, I represents the indicator function, and when the condition in the parentheses holds, the function value is 1, otherwise it is 0;

[0061] Use Gaussian distribution for modeling to estimate the conditional probability P(X|Y = c) of the feature vector X under each state classification label:

[0062]

[0063] Where, x j represents the value of the j-th component of the feature vector X, μ jc and σ jc respectively represent the mean and variance of the j-th feature under the state classification label c.

[0064] This application also provides an analysis system for musculoskeletal X-ray images, and the system includes:

[0065] A processor;

[0066] A memory stores a computer program that can run on the processor. When the computer program is executed by the processor, the steps of any one of the methods for analyzing musculoskeletal X-ray images are implemented.

[0067] This application also provides a computer-readable storage medium. A data processing program is stored on the computer-readable storage medium. When the data processing program is executed by a processor, the steps of any one of the methods for analyzing musculoskeletal X-ray images are implemented.

[0068] Compared with the prior art, the method for analyzing musculoskeletal X-ray images provided by this application designs a hybrid judgment model involving human participation in deep learning. The DenseNet121 model, ResNet50 model, and human judgment module are combined to form a machine group strategy, improving the accuracy of feature extraction of X-ray images by a single model. Furthermore, it can more accurately assist doctors in judging the status of musculoskeletal images. Description of the Drawings

[0069] Figure 1 It is a schematic diagram of the prediction model of the human judgment module provided in the first embodiment of this application;

[0070] Figure 2 It is a schematic diagram of the DenseNet121 network structure involved in the first embodiment of this application;

[0071] Figure 3 It is the ResNet50 network structure diagram involved in the first embodiment of this application.

[0072] Figure 4 It is a schematic diagram of the model structure of the method for analyzing musculoskeletal X-ray images provided in the first embodiment of this application. Detailed Embodiments

[0073] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments.

[0074] The first embodiment of this application provides a method for analyzing musculoskeletal X-ray images. The following is a detailed description of the method provided in the first embodiment of this application in conjunction with Figures 1 - 4 The method provided in the first embodiment of this application will be described in detail.

[0075] Convert the preliminary judgment result of the musculoskeletal X-ray image to be analyzed into a one-dimensional tensor, and obtain a set of predicted probability values Prob1 through probabilistic processing.

[0076] This step is used to obtain the probability value data of human judgment on musculoskeletal X-ray images.

[0077] The conversion of human judgment on musculoskeletal X-ray images into one-dimensional tensors is because the human judgment on images needs to be tensorized into a language that can be processed by a computer.

[0078] The probabilistic processing includes: obtaining one-dimensional tensor data and performing preset data processing; obtaining the result of the preset data processing and calculating it through the softmax function in the second dimension to obtain a set of predicted probability values Prob1.

[0079] Please refer to Figure 1 , which is a schematic diagram of the prediction model of the human judgment module. As Figure 1 (a), the human makes a prediction on the musculoskeletal X-ray image, and there are two prediction results. Among them, 0 represents no problem, and 1 represents a problem. After converting the human judgment result into a one-dimensional vector, preset data processing is performed. At the same time, please refer to Figure 1 (b), the preset data processing here includes corresponding subtraction, dot multiplication operations, and separation matrices. The subtraction is for data centering, the dot multiplication is to apply weights, and the role of the separation matrix is to batch-convert human judgments into probability values, and different separation matrices convert human judgments into target probability values.

[0080] Calculate through the softmax function in the second dimension to obtain a set of predicted probability values Prob1. The Softmax function is a function that converts a real number vector into a probability distribution. In multi-class classification problems, it can be used to represent the predicted probability of each class. In this application, the obtained predicted probability values are the probabilities of normal and abnormal.

[0081] Convert the pixel matrix of the image data of the musculoskeletal X-ray image to be analyzed into a three-dimensional tensor representing the number of image channels, height, and width.

[0082] This step is used to facilitate the computer to process and analyze the image. An image is usually composed of a pixel matrix, and each pixel contains the values of three channels of RGB (red, green, and blue). Usually, the pixel matrix of the image is represented as a three-dimensional tensor, where the first dimension represents the number of channels, the second dimension represents the height of the image, and the third dimension represents the width of the image. If the image is grayscale, it can be represented as a three-dimensional tensor of [1, H, W], where H represents the height of the image and W represents the width of the image.

[0083] Construct a DenseNet121 model and input the three-dimensional tensor into the trained DenseNet121 model to obtain a set of first image feature representations.

[0084] Construct a ResNet50 model, input the three-dimensional tensor into the trained ResNet50 model, and obtain a set of second image feature representations.

[0085] The above steps are used to obtain the image feature representations by inputting the dataset images into the DenseNet121 model and the ResNet50 model.

[0086] The image feature representation refers to the abstract representation learned by the model from the training data. These features may be various local and global features extracted from the images during the training process. When these feature vectors are used as the input to the Naive Bayes model, each feature vector will serve as the feature vector of a sample for training or prediction. The Naive Bayes model will use these feature vectors to establish the probability distribution of the classes for classification tasks.

[0087] The training of the DenseNet121 model includes:

[0088] Set the output of the fully connected layer of the DenseNet121 model to 2;

[0089] During the training process, the forward propagation formula of DenseNet121 is:

[0090] x0 = input

[0091] x l = H l ([x0, x1,..., x l-1 )

[0092] where x0 is the input data, x l represents the output of the l-th layer, and H i represents the non-linear transformation of the l-th layer. [x0, x1,..., x l-1 means concatenating the outputs from the 0-th layer to the l-1-th layer as the input to the l-th layer. For the l-th layer of DenseNet121, its input and output are:

[0093] x0 = input

[0094] x l = H l ([x0, x1,..., x l-1 )

[0095] x i = H i ([x0, x1,..., x i-1 ) 1 ≤ i ≤ l - 1

[0096] where H i represents the convolutional layer and non-linear transformation in the i-th dense connection block, and [x0, x1,..., x i-1It means concatenating the outputs of the previous i - 1 layers as the input to the i-th dense connection block. The backpropagation formula of DenseNet121 is as follows:

[0097]

[0098] Among them, L represents the loss function, and the rest of the definitions refer to the part of training the ResNet50 model below.

[0099] The DenseNet121 model is a deep learning model and belongs to one of the densely connected convolutional networks (Densely Connected Convolutional Networks). The "121" in "DenseNet121" indicates that it is composed of a 121-layer convolutional neural network, and "Dense" here refers to the dense connection structure in the network. In traditional convolutional neural networks, the output of each layer is only passed to the next layer. However, in DenseNet, the output of each layer is passed not only to the next layer but also to all subsequent layers. This densely connected structure allows information to be transmitted and shared more fully, helping to alleviate the problem of gradient vanishing and improving the training efficiency and performance of the model. In each dense block, the input feature map will sequentially go through a series of convolutional operations and non-linear activation function processing, and then the outputs of each convolutional layer are concatenated together to form the densely connected output.

[0100] The output of the fully connected layer of the DenseNet121 model is set to 2 to meet the binary classification task of this application.

[0101] Please refer to Figure 2, This figure is a schematic diagram of the DenseNet121 network structure. In the DenseNet121 (Dense Connected Network) structure, one of the core components is the Dense Block, which contains multiple densely connected convolutional layers. Inside each Dense Block, feature extraction and processing generally follow a standard pattern, which is "BN-ReLU-Conv", that is, a combination of Batch Normalization (BN), Rectified Linear Unit (ReLU), and Convolutional layer (Conv). This combination ensures the stability and efficiency of the network during training. At the same time, by introducing non-linearity and feature normalization, it increases the adaptability of the network to input data. Usually, each Dense Block is followed by a Transition Layer, whose purpose is to downsample the feature map and reduce the dimension of the feature map in order to reduce the complexity and computational amount of the model. The transition layer is mainly composed of a convolutional layer and a pooling layer. Among them, the convolutional layer is usually used to reduce the depth of the feature map, while the pooling layer is used to reduce the spatial dimensions (height and width) of the feature map. Specifically for the BN-ReLU-Conv structure, its role in the Dense Block can be described in detail as follows: BN (Batch Normalization): Normalize the output of the previous layer, standardize these feature maps, so that they have similar distributions before activation and convolution. This helps to accelerate convergence and makes the model less sensitive to the initial weights. ReLU (Rectified Linear Unit): Following the BN layer, ReLU is applied as an activation function to introduce non-linearity into the network, enabling the network to learn and represent complex function mappings. Conv (Convolutional layer): Finally, the feature map processed by the activation function is fed into the convolutional layer. In DenseNet, the convolutional layer is used to extract features and increase the network depth at the same time. The feature map generated by each convolutional layer (also called the feature with growth rate k) will be merged with the feature map input to this layer to form the output of this Dense Block. Transition to the TransitionLayer: After each Dense Block, the Transition Layer is responsible for reducing the number and size of the feature maps to prepare for the processing of the next Dense BloCk. This design helps to control the computational complexity and the number of parameters of the network, enabling DenseNet to maintain high efficiency and performance even in a relatively deep network structure.

[0102] The training of the ResNet50 model includes:

[0103] Set the output of the fully connected layer of the ResNet50 model to 2;

[0104] The forward pass of the ResNet50 model is as follows:

[0105] Y = f n (f n-1 (...f3(f2(f1(X)))))

[0106] where X is the input image, Y is the final output, and each convolutional layer, pooling layer, fully connected layer, etc. can be regarded as a function, denoted by f1, f2, f3,..., f n ;

[0107] The cross-entropy is used as the loss function. Assuming the training dataset is D, the cross-entropy loss function L is:

[0108]

[0109] where y i represents the actual label, represents the prediction result; the calculation formula of this loss function is based on the cross-entropy concept in information theory, and its intuitive meaning is to maximize the probability of classification, that is, to minimize the gap between the prediction result and the actual label to the greatest extent.

[0110] The overall calculation formula of the loss function is:

[0111]

[0112] where n represents the total number of samples in the training set, and L i represents the cross-entropy loss function of each sample, and this loss function will be used for backpropagation to calculate the gradient and update the model parameters.

[0113] After obtaining the value Loss of the loss function, the gradient descent method is used to reduce the value of L, and the classification ability of the model is improved according to this method. For the loss function L, the gradients of each layer are calculated through the chain rule. For the jth convolutional layer of the ith layer, the output feature map is F i，j , and the input feature map is F i-1，k , then the gradient calculation formula of this convolutional layer is:

[0114]

[0115] where, is the gradient of the previous layer, is the derivative of the convolutional layer. By using the above method to obtain the gradient calculation formulas of the batch normalization layer and the fully connected layer, the training model of ResNet50 is:

[0116]

[0117] where θ represents the parameters of the model.

[0118] The ResNet50 model is a deep residual network (Deep Residual Network), which belongs to the ResNet (Residual Network) family. ResNet solves the degradation problem in the deep neural network training process by introducing the concept of residual learning (Residual Learning). The degradation problem means that as the network depth increases, the network training error increases instead, which is mainly caused by the disappearance or explosion of the gradient. By introducing the residual block (ResidualBlock), the network can directly transfer the gradient through the skip connection (Skip Connection), thereby effectively training the deep network.

[0119] Please see Figure 3 , this figure is the network structure diagram of ResNet50. ResNet50 is divided into 5 stages, among which Stage0 has a relatively simple structure and can be regarded as a preprocessing of the input data. The last 4 stages are composed of Bottleneck (residual blocks) and have similar structures. Stage1 contains 3 Bottlenecks, and the remaining 3 stages contain 4, 6, and 3 Bottlenecks respectively. In Stage0, (3, 224, 224) refers to the input data dimension size, channel, height, width. The input passes through the convolution layer, BN (Batch Normalization) layer, RELU activation function, and MaxPooling layer in turn to obtain an output with a shape of (64, 56, 56).

[0120] BTNK1: C, W, C1, S represents a residual module, and the specific meanings are as follows: BTNK1: indicates the naming of a residual module. C: represents a Convolutional Layer. This layer is used for feature extraction in the residual module. Usually, the number of input and output channels of this convolutional layer is different. W: may represent a WeightLayer, which is used to adjust the weights of the input features. This layer may be a 1x1 convolutional layer used to reduce the dimension of the features. C1: also represents a convolutional layer, usually for further feature extraction after passing through the weight layer. This convolutional layer may be a 3x3 or other-sized convolution. S: may be used to specify a Stride, which controls the moving step of the convolution. A stride of 1 means the convolutional kernel moves one pixel each time, and a stride of 2 means the convolutional kernel moves two pixels each time. BTNK1 in Stage1: 64, 56, 64, 1 represents the basic structure of a residual module in Stage1, including two convolutional layers with 64 output channels, keeping the feature map size unchanged (56x56), and a setting with a stride of 1.

[0121] The database used for training the DenseNet121 model or ResNet50 model is the MURA dataset.

[0122] The MURA dataset is the largest publicly available musculoskeletal radiography image dataset, which contains multi-view images of fingers, hands, wrists, forearms, elbows, humeri, and shoulders in the upper limb region. Each study has one or more radiography images, which are manually labeled by radiologists.

[0123] The training of the DenseNet121 model or ResNet50 model also includes minimizing the loss function using the Adam optimization algorithm.

[0124] The Adam optimization algorithm is an optimization algorithm with an adaptive learning rate, which is used to train deep neural networks. It combines the ideas of momentum gradient descent and RMSProp (Root Mean Square Propagation) and introduces some additional corrections. Specifically, the Adam algorithm maintains two dynamically adjusted learning rates: Momentum term: This is a moving average, similar to momentum gradient descent. It reduces the variance of the gradient update and helps for more stable training. RMSProp term: This is the moving average of the squared gradient, similar to the RMSProp algorithm. It adjusts the learning rate of each parameter so that for parameters with large gradients, the learning rate decreases faster.

[0125] The formula of the Adam optimization algorithm is as follows:

[0126] mt = β1m t-1 + (1 - β1)g t

[0127]

[0128]

[0129]

[0130]

[0131] where m t and v t represent the first - order and second - order momentum estimates respectively, β1 and β1 are decay rates, g t is the current gradient, g t and are the bias corrections for momentum estimation, θ is the learning rate, and ∈ is an extremely small number.

[0132] Concatenate the predicted probability value Prob1, the first image feature representation, and the second image feature representation to obtain the feature data for characterizing the musculoskeletal state.

[0133] This step is used to obtain the feature data for characterizing the musculoskeletal state.

[0134] Furthermore, the feature data for characterizing the musculoskeletal state can be input into the Naive Bayes model to obtain the recognition result of the musculoskeletal X - ray image.

[0135] The Naive Bayes model is a probabilistic statistical classification model based on Bayes' theorem. The ″naive″ in the name of this model is because it assumes that the features are independent of each other, that is, given a certain class, there is no correlation between the features. The basis of the model is Bayes' theorem, which describes how to update the probability estimate of an event based on new observed data given prior information.

[0136] Obtain the predicted probability value Prob1, the first image feature representation, and the second image feature representation, and concatenate the three as the feature vector X; input the feature vector X into the Naive Bayes model; use Bayes' formula to calculate the probability of each state classification label in the state result, and select the state classification label with the highest probability as the final state recognition result; where Bayes' formula is:

[0137]

[0138] where represents the predicted value of class ″Y″, denotes the value of "c" that maximizes the expression, and P(Y = c) represents the probability of the state classification label "c". represents the product of the conditional probabilities of each state classification x given the state classification label "c" j where d is the number of feature vectors.

[0139] The input of the Naive Bayes model is usually a feature vector, rather than a single predicted probability value. The input values predicted probability value Prob1, the first image feature representation, and the second image feature representation need to be transformed into a feature vector, that is, the three are concatenated as the feature vector X.

[0140] The training phase of the Naive Bayes model involves calculating the prior probability of the state classification label and the conditional probability of each feature vector under each category. In the prediction phase, the model will use the conditional probabilities and prior probabilities of these feature vectors to calculate the posterior probabilities of each state classification label, and then select the category with the highest posterior probability as the final prediction result.

[0141] Before calculating the probability of each state classification label in the state result using Bayes' formula, the prior probability and conditional probability are obtained by the maximum likelihood estimation method, where the maximum likelihood estimation method includes:

[0142] Estimate the prior probability P(Y) of each state classification label according to the training data set:

[0143]

[0144] where N represents the size of the training data set, y i represents the true classification label of the i-th sample, and I represents the indicator function, whose function value is 1 when the condition in the parentheses holds, otherwise 0;

[0145] Modeling is carried out using the Gaussian distribution to estimate the conditional probability P(X|Y = c) of the feature vector X under each state classification label:

[0146]

[0147] where x j represents the value of the j-th component of the feature vector X, μ jc and σ jc represent the mean and variance of the j-th feature under the state classification label c, respectively.

[0148] The maximum likelihood method is a method for estimating the parameters of a statistical model. It seeks the parameter values that maximize the probability of the observed data in a given data set under the model. The maximum likelihood estimation method is used to estimate the prior probability and conditional probability in the Naive Bayes model because the maximum likelihood estimation method is a common parameter estimation method in statistics and machine learning. Especially in the absence of prior knowledge, it provides a reasonable way to estimate parameters.

[0149] Please refer to Figure 4 , which is a schematic diagram of the model structure of the analysis method for musculoskeletal X-ray images. Obtain the images of the MURA data set, convert the image data of the MURA data set images into a three-dimensional tensor A, input the three-dimensional tensor data into the trained DenseNet121 model B and ResNet50 model C to obtain two corresponding sets of image feature representations; input the MURA data set images into the model D participated by humans for prediction (convert the human judgment on the MURA data set images into a one-dimensional tensor and perform probability processing) to obtain a set of predicted probability values, and input the predicted probability values and the two sets of image feature representations into the Naive Bayes model E to obtain the analysis result of the musculoskeletal disease state.

[0150] The second embodiment of the present application provides an effect evaluation method for the analysis method of musculoskeletal X-ray images. The accuracy rate, recall rate, precision rate, F1-Score, ROC curve and AUC value are used as evaluation indicators to evaluate the performance of the analysis method of musculoskeletal X-ray images, and compare with a single ResNet50 model and a single DenseNet121169 model.

[0151] The hybrid model provided by the present application and a single ResNet50 model and a single DenseNet121169 model are evaluated with the same settings. All models are trained using the same strategy, and the model with the smallest cross-entropy loss after training participates in the evaluation. The confusion matrix can better understand the performance of the model in each category when testing the hybrid model and the baseline model. The accuracy rate, recall rate, precision rate and other indicators of the hybrid model and the baseline model are obtained through the confusion matrix, so as to compare the performance differences between the two models.

[0152] Through comparison, the hybrid model provided by the present application performs better than the single model in most aspects, and has good generalization performance and classification accuracy.

[0153] The third embodiment of the present application provides an analysis system for musculoskeletal X-ray images, and the system includes:

[0154] A processor;

[0155] A memory stores a computer program that can run on the processor. When the computer program is executed by the processor, the steps of the method for analyzing musculoskeletal X-ray images described in any one of the above are implemented.

[0156] The fourth embodiment of the present application provides a computer-readable storage medium. A data processing program is stored on the computer-readable storage medium. When the data processing program is executed by the processor, the steps of the method for analyzing musculoskeletal X-ray images described in any one of the above are implemented.

[0157] As described above, the above are only the preferred specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, makes equivalent substitutions or changes, and all should be covered by the protection scope of the present invention.

Claims

1. A method for analyzing musculoskeletal X-ray images, characterized in that, Including: Convert the preliminary judgment result of the musculoskeletal X-ray image to be analyzed into a one-dimensional tensor, and through probabilistic processing, obtain a set of predicted probability values Prob1; Convert the pixel matrix of the image data of the musculoskeletal X-ray image to be analyzed into a three-dimensional tensor representing the number of image channels, height, and width; Construct a DenseNet121 model, input the three-dimensional tensor into the trained DenseNet121 model, and obtain a set of first image feature representations; Construct a ResNet50 model, input the three-dimensional tensor into the trained ResNet50 model, and obtain a set of second image feature representations; Concatenate the predicted probability values Prob1, the first image feature representation, and the second image feature representation to obtain feature data for characterizing the musculoskeletal state.

2. The analysis method of musculoskeletal X-ray images according to claim 1, characterized in that, The probabilistic processing includes: Obtain one-dimensional tensor data and perform preset data processing; Obtain the preset data processing result, calculate through the softmax function on the second dimension, and obtain a set of predicted probability values Prob1.

3. The analysis method of musculoskeletal X-ray images according to claim 1, characterized in that, The training of the DenseNet121 model includes: Set the output of the fully connected layer of the DenseNet121 model to 2; During the training process, the forward propagation formula of DenseNet121 is: x0 = input x l = H l ([x0, x1,..., x l-1 ) where, x0 is the input data, x l represents the output of the l-th layer, H i represents the non-linear transformation of the l-th layer, [x0, x1,..., x l-1 represents concatenating the outputs from the 0-th layer to the (l-1)-th layer as the input of the l-th layer. For the l-th layer of DenseNet121, its input and output are as follows: x0 = input x l = H l ([x0, x1,..., x l-1 ) x i = H i ([x0, x1,..., x i-1 )1≤i≤l-1 Among them, H i represents the convolutional layer and non-linear transformation in the i-th densely connected block, and [x0, x1,..., x i-1 represents concatenating the outputs of the previous i - 1 layers as the input to the i-th densely connected block. The backpropagation formula of DenseNet121 is as follows: where L represents the loss function.

4. The analysis method of musculoskeletal X-ray images according to claim 1, characterized in that The training of the ResNet50 model includes: Set the output of the fully connected layer of the ResNet50 model to 2; The forward propagation of the ResNet50 model is: Y = f n (f n-1 (...f3(f2(f1(X))))) Among them, X is the input image, and Y is the final output. Each convolutional layer, pooling layer, fully connected layer, etc. can be regarded as a function, denoted by f1, f2, f3, ..., f n ; Use cross-entropy as the loss function. Assume the training dataset is D, and the cross-entropy loss function L is: Among them, y i represents the actual label, and represents the prediction result; The overall calculation formula of the loss function is: where n represents the total number of samples in the training set, and L i represents the cross-entropy loss function for each sample; For the j-th convolutional layer in the i-th layer, the output feature map is F i，j , and the input feature map is F i-1，k . Then the gradient calculation formula for this convolutional layer is: Among them, is the gradient of the previous layer, is the derivative of the convolutional layer. By using the above method to obtain the gradient calculation formulas of the batch normalization layer and the fully connected layer, the training model of ResNet50 is as follows: where θ represents the parameters of the model.

5. The analysis method of musculoskeletal X-ray images according to claim 3 or 4, characterized in that The database used for the training of the DenseNet121 model or the ResNet50 model is the MURA dataset.

6. The method for analyzing musculoskeletal X-ray images according to claim 3 or 4, characterized in that, The training of the DenseNet121 model or the ResNet50 model also includes using the Adam optimization algorithm to minimize the loss function. The formula of the Adam optimization algorithm is as follows: m t = β1m t-1 + (1 - β1)g t where m t and v t represent the first - order and second - order momentum estimates respectively, β1 and β1 are decay rates, g t is the current gradient, g t and are the bias corrections for momentum estimation, θ is the learning rate, and ∈ is a very small number.

7. The analysis method of musculoskeletal X-ray images according to claim 1, characterized in that, Also including: Input the feature data for characterizing the musculoskeletal state into the Naive Bayes model to obtain the recognition result of the musculoskeletal X-ray image: Obtain the predicted probability values Prob1, the first image feature representation, and the second image feature representation, and concatenate the three as the feature vector X; Input the feature vector X into the Naive Bayes model; Use Bayes' formula to calculate the probability of each state classification label in the state result, and select the state classification label with the highest probability as the final state recognition result; where Bayes' formula is: wherein represents the predicted value of class "Y", represents the value of "c" that maximizes the expression, and P(Y = c) represents the probability of the state classification label "c", represents the product of the conditional probabilities of each state classification x j under the given state classification label "c", where d is the number of feature vectors.

8. The analysis method of musculoskeletal X-ray images according to claim 7, characterized in that, Before using Bayes' formula to calculate the probability of each state classification label in the state result, obtain the prior probability and conditional probability through the maximum likelihood estimation method. Among them, the maximum likelihood estimation method includes: According to the training dataset, estimate the prior probability P(Y) of each state classification label: where N represents the size of the training dataset, y i represents the true classification label of the i-th sample, and I represents the indicator function, whose function value is 1 when the condition in the parentheses holds, and 0 otherwise; Use the Gaussian distribution for modeling and estimate the conditional probability P(X|Y = c) of the feature vector X under each state classification label: where x j represents the value of the j-th component of the feature vector X, μ jc and σ jc represent the mean and variance of the j-th feature under the state classification label c, respectively.

9. An analysis system for musculoskeletal X-ray images, characterized in that, The system includes: A processor; A memory stores a computer program that can run on the processor. When the computer program is executed by the processor, it implements the steps of the method for analyzing musculoskeletal X-ray images according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, A data processing program is stored on the computer-readable storage medium. When the data processing program is executed by the processor, it implements the steps of the method for analyzing musculoskeletal X-ray images according to any one of claims 1 to 8.

Images