Patellofemoral joint cartilage injury automatic grading method and system
By combining radiomics feature extraction and classification models of magnetic resonance images, the problem of the inability to identify patellofemoral joint cartilage damage in existing technologies has been solved, achieving higher accuracy in identifying cartilage damage levels and preventing misjudgments.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- THE FIFTH AFFILIATED HOSPITAL SUN YAT SEN UNIV
- Filing Date
- 2026-01-30
- Publication Date
- 2026-06-19
Smart Images
Figure CN122244492A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of medical technology, and in particular to an automatic grading method and system for patellofemoral joint cartilage injury. Background Technology
[0002] Current technologies typically utilize deep learning models for image recognition in medical images to aid in diagnosis and treatment. For example, in skeletal medical images, existing deep learning models often identify and automatically grade injuries to bones and hard bones such as the meniscus to assist in treatment. However, existing deep learning models cannot identify cartilage injuries in medical images. Therefore, there is an urgent need for a method that can identify cartilage injuries based on medical images. Summary of the Invention
[0003] The main objective of this application is to propose an automatic grading method and system for patellofemoral joint cartilage damage, which can achieve higher accuracy in identifying cartilage damage based on magnetic resonance images.
[0004] To achieve the above objectives, a first aspect of this application proposes an automatic grading method for patellofemoral joint cartilage damage, the automatic grading method for patellofemoral joint cartilage damage comprising:
[0005] Image omics features are extracted from the target magnetic resonance image to obtain the target texture transformation feature set and the target statistical change feature set. The target texture transformation feature group and the target statistical change feature group are classified by a preset target classification model to obtain the predicted probability of each preset damage level. Based on the damage level corresponding to the two largest predicted probabilities and the probability difference between the two largest predicted probabilities, determine whether the largest predicted probability meets the preset prediction accuracy condition. When the maximum predicted probability is determined to satisfy the preset prediction accuracy condition, the damage level corresponding to the maximum predicted probability is output. If the maximum predicted probability is determined to be insufficient to meet the prediction accuracy condition, a preset prompt message is output.
[0006] To achieve the above objectives, a second aspect of this application provides an automatic grading system for patellofemoral joint cartilage damage, comprising: The display terminal is used to provide an interactive interface; An execution module, in response to a target magnetic resonance image input through the interactive interface, executes the automatic grading method for patellofemoral cartilage injury as described in any of the first aspects.
[0007] The automatic grading method and system for patellofemoral cartilage injury proposed in this application extracts target texture transformation feature groups and target statistical change feature groups from the target magnetic resonance image. These features contain characteristics that distinguish patellofemoral cartilage from other bones, enabling the target classification model to more accurately identify the cartilage injury level. Furthermore, by adding a judgment condition for prediction accuracy, it effectively prevents misjudgments of adjacent cartilage injury levels, thus ensuring more reliable output results. Therefore, compared with related technologies, this application embodiment can achieve higher accuracy in identifying cartilage injury based on magnetic resonance images. Attached Figure Description
[0008] Figure 1 This is a schematic flowchart of an embodiment of the automatic grading method for patellofemoral joint cartilage injury provided in this application; Figure 2 A schematic diagram of the training and application process in another embodiment of the automatic grading method for patellofemoral cartilage injury provided in this application; Figure 3a A schematic diagram of the ROC curve of the LR model used in the model validation stage of the automatic grading method for patellofemoral cartilage injury provided in this application; Figure 3b A schematic diagram of the ROC curve of the RF model used in the model validation stage of the automatic grading method for patellofemoral cartilage injury provided in this application; Figure 3c A schematic diagram of the ROC curve of the SVM model used in the model validation stage of the automatic grading method for patellofemoral cartilage injury provided in this application. Figure 4 This is a schematic diagram of the system framework of the automatic grading system for patellofemoral cartilage injury provided in this application; Figure 5 This is a schematic diagram of the hardware structure of the device corresponding to the automatic grading method for patellofemoral cartilage injury provided in this application. Detailed Implementation
[0009] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0010] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.
[0011] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.
[0012] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the proper functioning of these embodiments acquired.
[0013] The following is a description of the terminology used in the embodiments of this application: Magnetic resonance imaging, or MRI for short, is a technique used to describe the process of achieving high levels of magnetic resonance imaging.
[0014] nnU-Net, short for no-new-Net, is an adaptive medical image segmentation deep learning framework. Its core is based on the U-Net architecture, which adapts to different medical image data through a fully automated process and can stably output high-performance segmentation results without manual parameter tuning.
[0015] Current technologies typically utilize deep learning models for image recognition in medical images to aid in diagnosis and treatment. For example, in skeletal medical images, existing deep learning models often focus on identifying and automatically grading damage to bones and hard bones like the meniscus to assist in treatment. Bones and menisci are typically regular or thick hard bones, while cartilage is often irregular and thin. For knee joints, MRI is often used for image acquisition. Each voxel in an MRI image is a volume-weighted average of all tissue signals. When a voxel contains both cartilage and another tissue (such as synovial fluid or bone), its signal is a mixture of these two tissue signals. Due to the unique characteristics of cartilage, its voxel is often superimposed with either bone or synovial fluid. Therefore, identifying the level of cartilage damage requires the model to be able to identify the signal contribution of pure cartilage tissue from the cartilage-synovial fluid and cartilage-bone boundary voxels. However, existing deep learning models cannot identify the signal contribution related to pure cartilage signals; that is, they cannot perform medical image recognition of cartilage damage. Therefore, there is an urgent need for a method to identify cartilage damage based on medical images. Based on this, this application proposes an automatic grading method and system for patellofemoral joint cartilage damage, which can achieve higher accuracy in identifying cartilage damage based on magnetic resonance imaging.
[0016] Understandably, referring to Figure 1 As shown in the embodiment of this application, the automatic grading method for patellofemoral joint cartilage damage includes: Step S100: Extract radiomics features from the target magnetic resonance image to obtain the target texture transformation feature group and the target statistical change feature group; Step S200: Classify the target texture transformation feature group and the target statistical change feature group using a preset target classification model to obtain the predicted probability of each preset damage level. Step S300: Based on the damage level corresponding to the two largest predicted probabilities and the probability difference between the two largest predicted probabilities, determine whether the largest predicted probability meets the preset prediction accuracy condition. Step S400: When the maximum prediction probability is determined to meet the preset prediction accuracy condition, the damage level corresponding to the maximum prediction probability is output. Step S500: When the maximum prediction probability is determined to be insufficient to meet the prediction accuracy condition, a preset prompt message is output.
[0017] By extracting target texture transformation feature groups and target statistical change feature groups from the target magnetic resonance image, and utilizing the features of each target texture transformation feature group and target statistical change feature group that contain characteristics that distinguish the patellofemoral joint cartilage from other bones, the target classification model can more accurately identify the damage level of the cartilage when classifying and recognizing it. At the same time, by increasing the judgment of prediction accuracy conditions, the misjudgment of adjacent damage levels of cartilage can be effectively prevented, thereby ensuring that the output results are more reliable. Therefore, the embodiments of this application can achieve higher accuracy in the identification of cartilage damage based on magnetic resonance images.
[0018] The target magnetic resonance image is the image required for damage level prediction in practical applications. This application does not limit the type of target magnetic resonance image. In some embodiments, the target magnetic resonance image is an image obtained using MRI.
[0019] This application does not limit the type of model used for radiomics feature extraction.
[0020] Both the target texture change feature group and the target statistical change feature group include at least one feature. In practical applications, radiomics features are divided into four categories: first-order statistical features, shape features, texture features, and wavelet transform features. First-order statistical features describe features related to voxel intensity distribution, reflecting lesion density information, regional disorder, etc. Shape features describe the geometric characteristics of lesions, reflecting lesion size, shape, surface roughness, etc. Texture features describe the spatial distribution information of lesion voxels, reflecting spatial heterogeneity features such as image grayscale changes, granularity, and roughness. Wavelet transform features are used to obtain multi-resolution image description information through wavelet transformation of the original image. The target texture transformation feature group includes texture-related features under wavelet transform features, as well as spatial heterogeneity features under texture features after LBP-2D transformation, logarithmic transformation, and square transformation.
[0021] This application does not limit the target classification model. In some embodiments, the target classification model can be a logistic regression model, and in other embodiments, the target classification model can be a support vector machine model.
[0022] This application does not limit the number of preset damage levels. In some embodiments, the damage level is a classification of lower-level damage, such as levels 0 to 3, and other methods are used to identify levels above level 3.
[0023] In some embodiments, the prediction accuracy condition is set to the probability difference between the predicted probabilities of the damage level with the highest predicted probability and its adjacent damage levels being greater than or equal to a preset probability threshold. In other embodiments, the prediction accuracy condition is the probability difference between the predicted probabilities of two specified adjacent damage levels being greater than or equal to a preset probability threshold. For example, if the probability of misclassifying level 2 as 1 is high only for level 2, then level 2 and level 1 are designated as the two adjacent damage levels. If the probability of misclassifying level 2 as 3 is high only for level 2, then level 2 and level 3 are designated as the two adjacent damage levels. In other words, the damage level with the highest misclassification probability during the training and validation phase of the target classification model and its adjacent levels are designated as the two adjacent damage levels.
[0024] This application does not limit how the prompt information is set. In some embodiments, the prompt information includes the damage levels corresponding to the two highest predicted probabilities. For example, when the difference between the predicted probabilities of level 1 and level 2 is less than 0.1, the message "Suspected damage, manual review recommended" is output. This approach allows for real-time combination of manual and automated methods, improving the reliability of the prediction results.
[0025] Understandably, based on the damage levels corresponding to the two highest predicted probabilities and the probability difference between the two highest predicted probabilities, it is determined whether the highest predicted probability meets the preset prediction accuracy conditions, including: When the damage levels corresponding to the two largest predicted probabilities are adjacent damage levels and the probability difference between the two largest predicted probabilities is less than the preset probability threshold, it is determined that the largest predicted probability does not meet the prediction accuracy condition. When the two highest predicted probabilities correspond to non-adjacent damage levels, the maximum prediction accuracy is determined to satisfy the prediction accuracy condition. When the probability difference between the two largest predicted probabilities is greater than or equal to the probability threshold, the largest predicted probability is determined to satisfy the prediction accuracy condition.
[0026] The embodiments of this application do not impose any restrictions on the probability threshold, which can be determined by those skilled in the art based on the verification results of the actual model during the verification phase.
[0027] In some embodiments, the probability threshold can be dynamically adjusted by combining the prompts from the target classification model's output with the results of actual manual review, thereby further improving the degree of automated coverage of the target classification model.
[0028] The embodiments of this application do not limit the adjacent damage levels. In some embodiments, it can be any two adjacent damage levels among a plurality of preset damage levels. In other embodiments, it can be configured adjacent damage levels.
[0029] Understandably, performing radiomics feature extraction on the target magnetic resonance image yields a target texture transformation feature set and a target statistical change feature set, including: The patellofemoral joint cartilage mask is obtained by performing pixel-level segmentation of the target magnetic resonance image using a pre-defined target nnU-Net. Based on the preset feature extraction terms, radiomics features are extracted from the region of interest corresponding to the patellofemoral joint cartilage mask in the target magnetic resonance image to obtain the target texture transformation feature group and the target statistical change feature group.
[0030] By performing pixel-level segmentation of target magnetic resonance images using nnU-Net, the training process of the target nnU-Net can be simplified, thereby improving the deployment efficiency of the entire process of automated recognition of target magnetic resonance images.
[0031] In some embodiments, the target nnU-Net may take the following steps: The initial nnU-Net is an nnU-Net constructed using a 3D weighted convolutional neural network architecture. The initial nnU-Net is configured with stochastic gradient descent (SGD) as the optimizer and poly learning rate decay strategy as the learning rate decay strategy. The magnetic resonance image of the target sample is obtained by data augmentation of the sample magnetic resonance image; The configured initial nnU-Net is iteratively trained based on the target sample magnetic resonance images.
[0032] Data augmentation includes at least one of rotation, scaling, Gaussian noise, and Gamma correction to improve the generalization ability of images from different devices.
[0033] Understandably, the feature extraction terms are obtained through the following steps: The target nnU-Net is used to perform pixel-level segmentation on multiple sample magnetic resonance images to obtain the patellofemoral joint cartilage sample mask for each sample magnetic resonance image; Based on the patellofemoral joint cartilage sample mask, high-dimensional radiomics features are extracted from the corresponding sample magnetic resonance images to obtain multiple radiomics sample features corresponding to each sample magnetic resonance image; Variance thresholding is performed on the features of each radiomics sample to obtain the features of the first candidate radiomics sample. Univariate statistical tests were performed on the characteristics of the first candidate radiomics sample to obtain the characteristics of the second candidate radiomics sample. The features of the second candidate image omics samples are screened by a preset minimum absolute shrinkage and selection operator model to obtain texture transformation sample features and statistical change sample features. Each texture transformation sample feature and the feature term corresponding to the statistical change sample feature are used as feature extraction terms.
[0034] The variance threshold is used to characterize the content of cartilage features. Those skilled in the art can selectively set it according to the actual situation. For example, in some embodiments, the variance threshold is set to 0.8, then less than 0.8 indicates that the content of cartilage-related features is low and needs to be removed.
[0035] The minimum absolute contraction and selection operator model is also known as the LASSO regression model.
[0036] For example, taking high-dimensional impact omics feature extraction based on PyRadiomics as an example, the specific details are as follows: Features of patellofemoral cartilage sample masks within corresponding regions of interest (ROIs) of the MRI images were extracted using PyRadiomics. The results included: first-order statistical features related to pixel grayscale energy, entropy, and kurtosis; shape features related to the surface area-to-volume ratio and sphericity of the patellofemoral cartilage sample mask; texture features related to the spatial arrangement of pixel grayscale values within ROIs such as GLCM, GLRLM, GLSZM, GLDM, and NGTDM; and wavelet transform features related to the spatial arrangement of pixel grayscale values within ROIs such as GLCM, GLRLM, GLSZM, GLDM, and NGTDM after wavelet transformation. The wavelet transform involved wavelet decomposition of the ROIs in the MRI images (including LLL, LLH, LHL, LHH, HLL, HLH, HHL, and HHH). This yielded 1409 radiomics sample features for each MRI image. At this point, the variance of radiomics features with the same characteristic item in multiple sample MRI images is calculated to obtain the feature variance. This feature variance is compared with a variance threshold, and those greater than or equal to the variance threshold are selected as first-candidate radiomics features (e.g., features with a variance < 0.8 are deleted). Univariate statistical tests are performed on the first-candidate radiomics features corresponding to the same characteristic item in each sample MRI image to determine the distribution state of each feature item. The distribution state value is compared with a preset significance value. If it is less than or equal to a significance threshold, it is retained; otherwise, it is deleted. For example, if the significance threshold is set to 0.05, a distribution state value less than 0.05 indicates that the feature item has a significant feature that can extract cartilage and should be retained, thus obtaining second-candidate radiomics features. Finally, a LASSO regression model is used to perform a final screening of the second-candidate radiomics features. In some embodiments, a 10-fold cross-validation method can be used to split the second candidate radiomics sample features corresponding to each sample magnetic resonance image into a validation set and a training set, thereby obtaining feature terms corresponding to the target texture transformation feature group and the target statistical change feature group that can be extracted.
[0037] Therefore, by using this multi-layered cascaded funnel algorithm to filter the feature items of the target texture transformation feature group and the target statistical change feature group, it can be further ensured that the extracted features are the features most correlated with the patellofemoral joint cartilage.
[0038] Understandably, performing radiomics feature extraction on the target magnetic resonance image yields a target texture transformation feature set and a target statistical change feature set, including: The target magnetic resonance image is resampled to obtain a first processed image of a preset target size; The grayscale values of the first processed image are normalized to obtain the second processed image; Image omics features are extracted from the second processed image to obtain target texture transformation feature set and target statistical change feature set.
[0039] Unifying the target MRI image to the target size so that it matches the size during training makes the spatial resolution independent of the device used to capture the target MRI image. Furthermore, normalizing the image grayscale values can make the final extracted radiomics features more accurate.
[0040] Normalization can be normalized to a specified interval, such as the interval [0,1].
[0041] This application does not limit the target size in its embodiments; in some embodiments, it can be set to 1.0mm × 1.0mm × 1.0mm. During the training phase, the sample magnetic resonance images are converted to the target size and then the image grayscale values are normalized. The target size ensures consistent spatial resolution for images from different sources during the training phase, thereby enhancing the generalization ability of the target nnU-Net for extracting radiomics features. This ensures that the extracted target texture transformation feature set and target statistical change feature set are close to the expected values in practical applications.
[0042] Understandably, the target texture transformation feature set includes: gray-level non-uniformity index, wavelet transform gray-level uniformity index, gray-level dependence index, area gray-level index, gray-level contrast index, region index, and correlation index.
[0043] Gray-level non-uniformity indexes characterize the degree of non-uniformity in the distribution of connected regions at different gray levels in the region of interest of an image after wavelet transform and LBP-2D transform, including lbp-2D_glszm_GrayLevelNonUniformity, wavelet-LLH_glrlm_GrayLevelNonUniformity, wavelet-LHL_glrlm_GrayLevelNonUniformity, logarithm_glszm_GrayLevelNonUniformity, and wavelet-HHL_glszm_GrayLevelNonUniformityNormalized.
[0044] Wavelet transform grayscale uniformity indices characterize the grayscale uniformity of regions of interest in an image, including wavelet-HHL_glcm_Idmn and wavelet-LHH_glcm_Idmn.
[0045] Gray-level dependence metrics characterize the relationship between texture looseness and gray-level values in an image, including square_gldm_LargeDependenceHighGrayLevelEmphasis, wavelet-HLL_gldm_LargeDependenceHighGrayLevelEmphasis, and wavelet-LHL_gldm_SmallDependenceLowGrayLevelEmphasis.
[0046] Area grayscale indices characterize the proportion of high and low grayscale values in connected regions of an image, including wavelet-HHL_glszm_LargeAreaHighGrayLevelEmphasis and wavelet-LHH_glszm_LargeAreaHighGrayLevelEmphasis.
[0047] Gray-scale contrast indexes characterize the degree of spatial change and contrast of gray-scale in local areas of an image after transformation compared to the original image. They include wavelet-HLL_ngtdm_Contrast and wavelet-HHH_ngtdm_Contrast.
[0048] Regional metrics are used for morphological evaluation of local regions in an image, including wavelet-HHL_glszm_LargeAreaHighGrayLevelEmphasis, wavelet-HLL_glszm_ZoneEntropy, and wavelet-LHH_glszm_ZonePercentage.
[0049] Correlation indices characterize the gray-level correlation patterns of pixels in an image, including wavelet-LLH_glcm_MCC, wavelet-LLH_glcm_Correlation, wavelet-LLH_glcm_Imc2, and lbp-2D_glszm_LargeAreaEmphasis.
[0050] Understandably, the target statistical change feature group includes: 2D Local Binary Mode (LDP)-2D gray mean index, HLL subband gray distribution index, exponential transform gray median index, and square transform gray value kurtosis index.
[0051] The LDP-2D grayscale mean index is also known as lbp-2D_firstorder_Mean.
[0052] The HLL subband grayscale distribution index is also known as wavelet-HLL_firstorder_90Percentile.
[0053] The exponential transformation gray median index is also known as exponential_firstorder_Median.
[0054] The kurtosis index of grayscale values after square transformation is also known as square_firstorder_Kurtosis.
[0055] Understandably, the target classification model is determined through the following steps: The process involves obtaining multiple pre-defined models to be validated, as well as texture transformation sample features and statistical change sample features of magnetic resonance images of each sample. The multiple models to be validated include logistic regression model, support vector machine model, and random forest model. The model was validated by examining the texture transformation sample features and statistical change sample features of each sample magnetic resonance image, and the classification results corresponding to the sample magnetic resonance images were obtained. Evaluation metrics are determined based on the classification results of each model to be validated. These metrics include the area under the curve (AUC) for a single class, recall, macro-average AUC, micro-average AUC, and precision. The target classification model is determined from a large number of models to be validated based on the evaluation metrics.
[0056] The model to be validated is a pre-trained model. The area under the receiver operating characteristic (ROC) curve for each damage level, represented by the single-class AUC metric, is shown below. Figure 3a As shown, the damage levels are set from 0 to 3. The ROC curves include ROC curves for each damage level from 0 to 3. For example, the level 0 ROC is the ROC curve for damage level 0. There are multiple AUC metrics for a single class: the area under the curve for level 0 ROC, level 1 ROC, level 2 ROC, and level 3 ROC. The micro-average AUC represents the result of "breaking down" all class / label samples into a unified binary classification problem and then calculating the global FPR and TPR. The macro-average AUC first calculates the AUC for each class independently, and then takes the arithmetic mean of the AUCs for all classes. Recall represents the probability that a sample that is actually positive is predicted as positive by the model to be validated. Precision represents the probability that the model to be validated predicts the sample accurately. In some embodiments, the evaluation metrics also include the F1 score, which is the harmonic mean of precision and recall.
[0057] Therefore, by using evaluation metrics from multiple dimensions, we can make a more comprehensive selection of the model to be validated, ensuring that the target classification model has higher accuracy in cartilage identification.
[0058] For example, refer to the evaluation metrics of the logistic regression model, support vector machine model, and random forest model on different test datasets as shown in Tables 1 and 2:
[0059] Table 1. AUC of LR, SVM, and RF on three training sets. In this table, LR stands for Logistic Regression, SVM for Support Vector Machine, and RF for Random Forest. micro represents the micro-average AUC, and macro represents the macro-average AUC. Table 1 shows 745 cases in the training set, 100 cases in the external test set, and 342 cases in the internal test set. The training set, external test set, and internal test set are independent of age and gender.
[0060]
[0061] Table 2 shows the accuracy, recall, F1 score, and precision metrics of LR, SVM, and RF on the three training sets. As shown in Tables 1 and 2 above, in the internal test set, the three radiomics models can effectively distinguish between grade 0 and grade 3 cartilage lesions. The AUCs of the LR radiomics model were 0.81 (95% CI: 0.75, 0.78) and 0.83 (95% CI: 0.78, 0.88), respectively, while the AUCs of the RF radiomics model were 0.74 (95% CI: 0.69, 0.80) and 0.79 (95% CI: 0.73, 0.84). The SVM radiomics model had an AUC of 0.79 (95% CI: 0.74, 0.84) and 0.84 (95% CI: 0.79, 0.89), respectively. In the external test set, both the LR and SVM radiomics models showed good ability to identify grade 3 cartilage lesions, with AUCs of 0.78 (95% CI: 0.68, 0.88) and 0.76 (95% CI: 0.66, 0.86), respectively. However, their diagnostic efficacy for other grades was generally low. Therefore, LR or SVM was selected as the candidate model for the target classification. In practical applications, LR can be used as the default target classification model, or the user can modify the target classification model to SVM. Alternatively, both models can be used separately for prediction, and the lesion grade can be determined based on the combined prediction results. Those skilled in the art can selectively configure these models according to the specific circumstances.
[0062] Understandably, the probability threshold is determined through the following steps: Obtain the evaluation metrics for the target classification model during the validation phase; Based on the evaluation indicators, the first candidate damage level is determined when the target classification model has an error rate higher than a preset threshold for different damage levels. The probability threshold is determined based on the false prediction probability and the correct prediction probability of the first candidate damage level.
[0063] The validation phase refers to the process of testing each model to be validated on the training set, internal test set, and external test set after training.
[0064] For example, if the error rate of identifying A1 in sample data of damage level A1 is higher than a preset threshold, then A1 is a candidate damage level. Assume the predicted probability of identifying it as A1 is B1%, and the predicted probability of identifying it as A2 is B2%, meaning B1% is the correct prediction probability and B2% is the incorrect prediction probability. This can be determined based on the first difference between B1% and B2%. In some embodiments, a second difference between the probability of A2 being incorrectly identified as A1 and the probability of correct identification is also determined, and then the probability is determined jointly based on the first and second differences. In this case, by identifying the first candidate damage level with a high error rate and the second candidate damage level that is incorrectly identified by the first candidate damage level, the probability threshold is determined based on the first difference between the predicted probabilities of the first and second candidate damage levels when the first candidate damage level is incorrectly identified during the verification stage, and the second difference between the second candidate level and the first candidate damage level when the second candidate level is incorrectly identified during the verification stage, and the mean or root mean square deviation of the first and second differences.
[0065] For example, see below. Figure 2 The illustration shows the entire automated process from model training to application involved in the automatic grading of patellofemoral joint cartilage damage according to embodiments of this application. The specific steps are as follows: Step S1: Multicenter data standardization preprocessing: The acquired raw DICOM data is resampled, interpolated and normalized to eliminate the differences in imaging parameters caused by different MRI devices (such as 1.5T / 3.0T, GE / Siemens). At this time, the sagittal PDWI-FS image of the knee joint corresponding to the magnetic resonance image of each sample of the raw DICOM data can be obtained.
[0066] Step S2: Anatomical structure segmentation based on adaptive deep network: Construct an initial nnU-Net model and train it to obtain a target nnU-Net model. Perform pixel-level automatic segmentation of PFJ cartilage in sagittal PDWI-FS images of the knee joint to generate a patellofemoral cartilage sample mask. The training of the initial nnU-Net model includes the following settings: Optimizer configuration: SGD (Stochastic Gradient Descent) is used; Configure learning rate decay measurement: Set the initial learning rate to 0.01 and use a poly learning rate decay strategy; Set training batches: 4; Configure the number of rounds per batch: 500; Configure online data augmentation: Before training, the sagittal PDWI-FS images of the knee joint are augmented by rotation, scaling, Gaussian noise, and Gamma correction and then used as sample data for training. At this point, the target nnU-Net model can be automatically trained by starting the configured initial nnU-Net model.
[0067] Step S3: High-dimensional radiomics feature extraction: Based on the patellofemoral joint cartilage sample mask, high-dimensional features including first-order statistics, shape, texture (GLCM, GLRLM, etc.) and wavelet transform are extracted from the sagittal PDWI-FS image of the knee joint.
[0068] Step S4: Cascaded Feature Dimensionality Reduction and Screening: A three-level cascaded screening strategy of "variance thresholding method -> univariate selection method -> LASSO regression" is adopted to identify the key feature combination most relevant to cartilage damage, determine the feature extraction items, and identify the features of texture transformation samples and statistical change samples.
[0069] Step S5: Determine the target classification model: Input the texture transformation sample features and statistical change sample features into multiple trained models to be validated for classification validation, obtain evaluation indicators, and use LR and SVM as candidate models for the target classification model based on the evaluation indicators. The following example uses LR as the target classification model. For example, such as Figures 3a-3c As shown, Figure 3a The ROC curve for LR is shown. Figure 3b The ROC curve for RF is shown. Figure 3c The ROC curve for SVM is given below. Figures 3a-3cThe single-class AUC, macro-average AUC, and micro-average AUC can be obtained. Combining the data in Table 1 with Tables 1 and 2, it can be seen that for cartilage injury levels 0-3, all three models significantly misclassify levels 1 and 2. In the external test set, the three models can relatively well identify level 3 cartilage injury, but their diagnostic efficacy for other levels is not ideal. In the internal test set, the three radiomics models can relatively well identify levels 0 and 3 cartilage injury. Furthermore, combining Tables 1 and 2, it can be seen that LR and SVM have higher recognition accuracy than RF, but their accuracy for cartilage injury levels 1 and 2 is insufficient. Therefore, prediction accuracy conditions can be set based on Tables 1 and 2 to address the insufficient accuracy for cartilage injury levels 1 and 2, thereby improving the reliability of the predicted cartilage injury level.
[0070] Step S6: Radiomics feature extraction of the target magnetic resonance image: The target magnetic resonance image is segmented using the target nnU-Net model to obtain the patellofemoral joint cartilage mask; Radiomics feature extraction is performed on the target magnetic resonance image based on the patellofemoral joint cartilage mask to obtain the target texture transformation feature group and the target statistical change feature group; Step S7: Multi-class damage grading prediction: Input the filtered features into the target classification model to obtain the predicted probabilities of each preset damage level (such as the predicted probabilities of level 0, level 1, level 2, and level 3).
[0071] Step S8: Weighted decision mechanism: To address the issue of low differentiation between Level 1 and Level 2 in the verification phase, when the highest predicted probability among the predicted probabilities of each damage level is Level 1 or Level 2, and the difference in predicted probabilities between Level 1 and Level 2 is less than 0.1, a prompt message is output; otherwise, the damage level corresponding to the highest predicted probability is output.
[0072] Therefore, this application combines nnU-Net with high-dimensional radiomics analysis and applies it to the patellofemoral joint cartilage, a specific anatomical structure. By using 25 selected specific radiomics feature combinations (i.e., target texture transformation feature group and target statistical change feature group) for classification training, it can achieve high-precision recognition for software with high damage levels, while providing information prompts for damage levels with low recognition accuracy, thereby improving the reliability of automated magnetic resonance image recognition output results.
[0073] Understandably, referring to Figure 4 As shown, an automatic grading system for patellofemoral joint cartilage injury provided according to an embodiment of this application includes: Display terminal, which is used to provide an interactive interface; The execution module, in response to the target magnetic resonance image input through the interactive interface, performs the following steps: Image omics features are extracted from the target magnetic resonance image to obtain the target texture transformation feature set and the target statistical change feature set. The target texture transformation feature group and the target statistical change feature group are classified by a preset target classification model to obtain the predicted probability of each preset damage level. Based on the damage level corresponding to the two highest predicted probabilities and the probability difference between the two highest predicted probabilities, determine whether the predicted probabilities meet the preset prediction accuracy conditions. When the predicted probability is determined to meet the preset prediction accuracy condition, the damage level corresponding to the highest predicted probability will be output. If the predicted probability does not meet the prediction accuracy requirements, a preset prompt message will be output.
[0074] The display terminal and the execution module can be integrated into a single device. In other embodiments, the execution module can be deployed in the cloud or on a server. This application does not impose limitations on this; for example, refer to... Figure 4 As shown, the execution module is deployed in the cloud, and the display terminal is set up with multiple terminals, namely terminal 1 to terminal N. At this time, terminals 1 to N access the execution module through the interconnection network.
[0075] Please see Figure 5 , Figure 5 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 601 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 602 can be a NAND flash memory. The relevant program code is stored in the memory 602 and is called by the processor 601 to execute the methods described in the embodiments of this application. The input / output interface 603 is used to implement information input and output; The communication interface 604 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 605 transmits information between various components of the device (e.g., processor 601, memory 602, input / output interface 603, and communication interface 604); The processor 601, memory 602, input / output interface 603, and communication interface 604 are connected to each other within the device via bus 605.
[0076] This application also provides a computer-readable storage medium that stores a computer program that, when executed by a processor, implements the above-described method.
[0077] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.
[0078] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.
[0079] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.
[0080] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.
[0081] The terms “comprising” and “having”, and any variations thereof, in the specification and accompanying drawings of this application are intended to cover non-exclusive inclusion, such that a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units that are expressly listed, but may include other steps or units that are not expressly listed or that are inherent to such process, method, product, or apparatus.
[0082] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.
[0083] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.
[0084] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0085] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0086] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0087] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.
Claims
1. An automatic grading method for patellofemoral joint cartilage damage, characterized in that, The method includes: Image omics features are extracted from the target magnetic resonance image to obtain the target texture transformation feature set and the target statistical change feature set. The target texture transformation feature group and the target statistical change feature group are classified by a preset target classification model to obtain the predicted probability of each preset damage level. Based on the damage level corresponding to the two largest predicted probabilities and the probability difference between the two largest predicted probabilities, determine whether the largest predicted probability meets the preset prediction accuracy condition. When the maximum predicted probability is determined to satisfy the preset prediction accuracy condition, the damage level corresponding to the maximum predicted probability is output. If the maximum predicted probability is determined to be insufficient to meet the prediction accuracy condition, a preset prompt message is output.
2. The automatic grading method for patellofemoral joint cartilage injury according to claim 1, characterized in that, The step of determining whether the largest predicted probability meets a preset prediction accuracy condition based on the damage level corresponding to the two largest predicted probabilities and the probability difference between the two largest predicted probabilities includes: If the damage levels corresponding to the two largest predicted probabilities are adjacent damage levels and the probability difference between the two largest predicted probabilities is less than a preset probability threshold, it is determined that the largest predicted probability does not meet the prediction accuracy condition. When the two largest predicted probabilities correspond to non-adjacent damage levels, the largest predicted accuracy is determined to satisfy the prediction accuracy condition. When the probability difference between the two largest predicted probabilities is greater than or equal to the probability threshold, the largest predicted probability is determined to satisfy the prediction accuracy condition.
3. The automatic grading method for patellofemoral joint cartilage injury according to claim 1, characterized in that, The process of extracting radiomics features from the target magnetic resonance image to obtain a target texture transformation feature set and a target statistical change feature set includes: The target magnetic resonance image is segmented at the pixel level using a preset target nnU-Net to obtain a patellofemoral joint cartilage mask; Based on preset feature extraction terms, radiomics features are extracted from the region of interest corresponding to the patellofemoral joint cartilage mask in the target magnetic resonance image to obtain target texture transformation feature group and target statistical change feature group.
4. The automatic grading method for patellofemoral joint cartilage injury according to claim 3, characterized in that, The feature extraction terms are obtained through the following steps: The target nnU-Net is used to perform pixel-level segmentation on multiple sample magnetic resonance images to obtain the patellofemoral joint cartilage sample mask for each sample magnetic resonance image; Based on the patellofemoral joint cartilage sample mask, high-dimensional radiomics features are extracted from the corresponding magnetic resonance images of the samples to obtain multiple radiomics sample features corresponding to each magnetic resonance image of the samples; Variance thresholding is performed on each of the radiomics sample features to obtain the first candidate radiomics sample features; Univariate statistical tests were performed on the features of the first candidate radiomics sample to obtain the features of the second candidate radiomics sample. The second candidate image omics sample features are filtered by a preset minimum absolute shrinkage and selection operator model to obtain texture transformation sample features and statistical change sample features. Each feature of the texture transformation sample and the feature corresponding to the statistical change sample are used as the feature extraction term.
5. The automatic grading method for patellofemoral joint cartilage injury according to claim 1, characterized in that, The process of extracting radiomics features from the target magnetic resonance image to obtain a target texture transformation feature set and a target statistical change feature set includes: The target magnetic resonance image is resampled to obtain a first processed image of a preset target size; The grayscale values of the first processed image are normalized to obtain the second processed image; The second processed image is subjected to image omics feature extraction to obtain target texture transformation feature group and target statistical change feature group.
6. The automatic grading method for patellofemoral joint cartilage injury according to claim 1, characterized in that, The target texture transformation feature group includes: gray-level non-uniformity index, wavelet transform gray-level uniformity index, gray-level dependence index, area gray-level index, gray-level contrast index, region index, and correlation index.
7. The automatic grading method for patellofemoral joint cartilage injury according to claim 1, characterized in that, The target statistical change feature group includes: 2D Local Binary Mode (LDP)-2D gray-level mean index, HLL subband gray-level distribution index, exponential transformation gray-level median index, and square transformation gray-level kurtosis index.
8. The automatic grading method for patellofemoral joint cartilage injury according to claim 1, characterized in that, The target classification model is determined through the following steps: Acquire multiple preset models to be verified, as well as texture transformation sample features and statistical change sample features of magnetic resonance images of each sample. The multiple models to be verified include logistic regression model, support vector machine model and random forest model. The texture transformation sample features and statistical change sample features of each sample magnetic resonance image are validated by each of the models to be validated, and the classification results corresponding to the sample magnetic resonance images are obtained. Evaluation metrics are determined based on the classification results of each model to be validated. These evaluation metrics include the area under the curve (AUC) for a single class, recall, macro-average AUC, micro-average AUC, and accuracy. The target classification model is determined from a majority of models to be validated based on the evaluation metrics.
9. The automatic grading method for patellofemoral joint cartilage injury according to claim 2, characterized in that, The probability threshold is determined through the following steps: Obtain the evaluation metrics of the target classification model during the validation phase; Based on the evaluation index, the first candidate damage level is determined to have a recognition error rate for different damage levels that is higher than a preset threshold. A probability threshold is determined based on the error prediction probability and the correct prediction probability of the first candidate damage level.
10. An automatic grading system for patellofemoral joint cartilage injury, characterized in that, include: The display terminal is used to provide an interactive interface; An execution module, in response to a target magnetic resonance image input through the interactive interface, executes the automatic grading method for patellofemoral cartilage injury as described in any one of claims 1 to 9.