Bone age assessment methods and systems based on MRI and deep neural networks
By employing a bone age assessment method based on MRI and deep neural networks, and utilizing multimodal feature extraction and fusion, the problems of radiation risk and subjectivity in existing bone age assessments have been solved. This method achieves radiation-free, rapid, and accurate bone age assessment, promoting green and precise bone age identification in adolescents.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SICHUAN UNIV
- Filing Date
- 2023-05-11
- Publication Date
- 2026-06-30
AI Technical Summary
Existing bone age assessment methods suffer from problems such as radiation exposure risks, subjective nature of artificial characterization, high workload, and poor assessment performance, especially in the assessment of adolescent bone age, where ethical controversies are serious.
A bone age assessment method based on MRI and deep neural networks was adopted. An automated model for multimodal feature extraction, fusion and bone age output was constructed. Three-dimensional convolutional neural networks were used to extract and fuse T1WI, T2WI and PDWI image features, and the softmax-mean-variane loss function was combined for bone age assessment.
It achieves radiation-free, rapid, and accurate bone age assessment, improves the accuracy and precision of bone age assessment, reduces the subjectivity and time cost of manual assessment, and promotes green and healthy examinations and precise assessments for adolescent bone age identification.
Smart Images
Figure CN116807398B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of bone age assessment technology, and in particular relates to a bone age assessment method and system based on MRI and deep neural networks. Background Technology
[0002] Bone age is of great significance in criminal investigations, sentencing, eligibility verification for competitive sports, and adult height prediction. Current bone age assessment methods are mostly based on radiological examinations such as X-rays or CT scans of the long bones of the limbs. However, adolescents are highly sensitive to X-rays, and conducting radiological examinations for non-clinical purposes does not comply with modern medical ethics. In vivo bone age estimation based on radiological examinations has been a source of ongoing ethical controversy in the field of international forensic science. Furthermore, existing methods often rely on expert-designed features and manual interpretation of X-rays to assess skeletal development and infer age. These designed features often fail to fully reflect age-related characteristics, and the entire interpretation process is time-consuming and labor-intensive, resulting in significant subjective errors and limiting the universality of such methods in bone age assessment practice.
[0003] Magnetic resonance imaging (MRI) possesses superior cartilage imaging capabilities, clearly displaying the growth and development of long bone epiphyses and epiphyseal plates, as well as the process of epiphyseal closure. It is an ideal radiation-free imaging method for assessing immature skeletal structures. Furthermore, MRI can image in any plane, containing three-dimensional information, providing new and more promising research techniques and imaging strategies for bone age assessment. Multimodal MRI images of the knee joint, such as T1-weighted images (T1WI), T2-weighted images (T2WI), and proton density-weighted images (PDWI), contain rich and diverse age-related information, demonstrating good bone age assessment performance. However, the multidimensional and multiplanar characteristics of MRI limit the application of traditional manual methods in bone age MRI assessment. Deep learning methods can automatically learn multi-layer features, making them well-suited for interpreting the complex relationships between medical images and their mappings. Previous studies on automated MRI bone age inference have had relatively small sample sizes, not including younger age groups, and such sample data is severely scarce in my country. Moreover, most studies use two-dimensional convolutional kernels, making it difficult to fully utilize the data correlations between MRI slices. Three-dimensional convolutional neural networks (3D-CNN) can encode 3D data in three directions (x, y, z) to extract continuous information between slices, making good use of the spatial and temporal features of MRI and improving the performance of bone age assessment.
[0004] In summary, the inventors have developed a bone age assessment method based on multimodal MRI of the knee joint and 3D-CNN that integrates multimodal feature extraction, feature fusion and bone age output, achieving a green, convenient and accurate integrated bone age assessment for adolescents.
[0005] Based on the above analysis, the problems and shortcomings of the existing technology are as follows:
[0006] (1) Existing bone age assessment methods based on CT or X-ray image features pose a risk of radiation exposure;
[0007] (2) Existing artificial feature grading methods for bone age assessment are labor-intensive, highly subjective, rely on experience in reading images, and have unclear feature patterns, resulting in poor bone age assessment performance. Summary of the Invention
[0008] In view of the problems existing in the prior art, the present invention provides a bone age assessment method and system based on MRI and deep neural networks.
[0009] This invention is implemented as follows: a bone age assessment method based on MRI and deep neural networks, wherein the bone age assessment method based on MRI and deep neural networks includes:
[0010] By using deep learning algorithms to comprehensively analyze three types of MRI weighted images, an automated, radiation-free bone age assessment model is constructed that integrates multimodal feature recognition, extraction, fusion, and bone age output.
[0011] Furthermore, the bone age assessment method based on MRI and deep neural networks includes the following steps:
[0012] Step 1: Resample, perform bias field correction, and normalize the MRI scans of users whose bone age is to be assessed;
[0013] Step 2: Automatically extract T1WI, T2WI and PDWI image features using three feature extraction networks respectively;
[0014] Step 3: Automatically fuse T1WI, T2WI, and PDWI image features using the multimodal multiscale feature fusion module;
[0015] Step 4: Use the bone age assessment subnetwork to perform bone age assessment based on the fused feature map.
[0016] Furthermore, the automated, radiation-free bone age assessment model based on multi-feature fusion includes:
[0017] Feature extraction networks were designed for T1WI, T2WI, and PDWI of MRI. Each feature extraction network has one 1×7×7 convolutional layer and four R-P3D modules. Each convolutional layer is followed by a normalization layer and a CELU activation function. Each R-P3D module is followed by a max pooling layer to obtain the feature map of each modality.
[0018] The multimodal and multiscale feature fusion module includes two selective feature fusion (SFF) modules. The first SFF module is used to fuse the feature maps of the second-to-last maxpooling layer in the three feature extraction networks. The second SFF module is used to fuse the feature maps of the first SFF and the last maxpooling layer in the three feature extraction networks, and outputs the final multimodal and multiscale fused feature map.
[0019] The bone age assessment subnetwork consists of an average pooling layer, a fully connected layer, CELU, a Dropout layer, and another fully connected layer. The loss function used is the Softmax-Mean-Variant loss function. It is used to generate the final predicted age from the fused features of T1WI, T2WI, and PDWI.
[0020] Another object of the present invention is to provide an MRI-based bone age assessment system for implementing the aforementioned MRI-based bone age assessment method, the MRI-based bone age assessment system comprising:
[0021] The image preprocessing module is used to resample, perform bias field correction, and normalize the T1WI, T2WI, and PDWI images of users whose bone age is to be assessed.
[0022] The bone age assessment module is used to assess bone age based on processed user MRI T1WI, T2WI, and PDWI images of the bone age to be assessed using a trained, multi-feature fusion-based automated, radiation-free bone age assessment model.
[0023] Based on the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solution to be protected by this invention are as follows:
[0024] First, addressing the technical problems existing in the prior art and the difficulty in solving them, this paper closely analyzes, in conjunction with the technical solution to be protected by this invention and the results and data obtained during the research and development process, how the technical solution of this invention solves the technical problems, and the inventive technical effects brought about by solving these problems. The specific description is as follows:
[0025] This invention utilizes artificial intelligence combined with multimodal MRI 3D information to optimize bone age estimation strategies. MRI contains 3D information on epiphyseal development, and multimodal MRI contains even richer epiphyseal development features, thus serving as a superior carrier of epiphyseal development information. This invention employs a 3D convolutional neural network to automatically identify and extract effective features from the input MRI 3D image, and automatically fuses multimodal, multi-scale features, achieving automated, radiation-free bone age assessment. This improves the accuracy of bone age estimation, significantly shortens assessment time, and makes the method more readily applicable.
[0026] Second, considering the technical solution as a whole or from a product perspective, the technical effects and advantages of the technical solution to be protected by this invention are specifically described as follows:
[0027] This invention employs multimodal knee MRI observation metrics to construct an automated MRI method for estimating bone age in adolescents, overcoming the ethical dilemmas of in vivo age estimation. The deep learning model used overcomes the limitations of traditional deep learning methods for bone age assessment, which suffer from insufficient feature representation and learn only local features. It can comprehensively extract local, global, and inter-slice information from 3D MRI, potentially discovering effective features that experts might overlook, thus improving the network's feature learning ability. The SFF module adaptively and weightedly fuses multimodal and multi-scale MRI features, fully leveraging the synergistic effects between different modalities, and combines this with the Softmax-Mean-Variance loss function to improve the accuracy and precision of bone age assessment.
[0028] Third, as supplementary evidence of the inventive step of the claims of this invention, it is also reflected in the following important aspects:
[0029] The technical solution of this invention solves a technical problem that people have long desired to solve but have been unable to achieve:
[0030] This invention uses multimodal MRI, which provides richer bone age information while avoiding ionizing radiation;
[0031] This invention incorporates a feature extraction network and a multimodal, multiscale feature fusion module into an end-to-end bone age assessment subnetwork, which can efficiently and comprehensively extract age-related features from MRI, thereby making bone age assessment faster, more accurate, and more reliable.
[0032] This invention is expected to address the shortcomings in radiation exposure for in vivo bone age estimation, safeguard the legitimate rights and health of adolescents, and actively promote the integrated configuration of green health examination, accurate bone age estimation, and efficient automatic assessment in the practice of bone age identification for adolescents in my country.
[0033] The model in this invention has low requirements for the operator's professional skills for the already trained model, can quickly assess bone age, save manpower and resources for primary testing, and facilitate relevant testing in grassroots units. Attached Figure Description
[0034] Figure 1 This is a flowchart of the MRI-based bone age assessment method provided in an embodiment of the present invention;
[0035] Figure 2 This is a schematic diagram of the bone age assessment method based on MRI provided in this embodiment of the invention. Detailed Implementation
[0036] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.
[0037] like Figure 1 As shown, the bone age assessment method based on MRI and deep neural networks provided in this embodiment of the invention includes the following steps:
[0038] S101, resampling, bias field correction and normalization of MRI scans of users whose bone age is to be assessed;
[0039] S102, using three feature extraction networks to automatically extract T1WI, T2WI and PDWI image features respectively;
[0040] S103 automatically fuses T1WI, T2WI, and PDWI image features using a multimodal and multiscale feature fusion module;
[0041] S104 utilizes a bone age assessment subnetwork to perform bone age assessment based on fused feature maps.
[0042] The automated, radiation-free bone age assessment model with multi-feature fusion provided in this invention includes:
[0043] Feature extraction networks were designed for T1WI, T2WI, and PDWI of MRI. Each feature extraction network has one 1×7×7 convolutional layer and four R-P3D modules. Each convolutional layer is followed by a normalization layer and a CELU activation function. Each R-P3D module is followed by a max pooling layer to obtain the feature map of each modality.
[0044] The multimodal and multiscale feature fusion module includes two selective feature fusion (SFF) modules. The first SFF module is used to fuse the feature maps of the second-to-last maxpooling layer in the three feature extraction networks. The second SFF module is used to fuse the feature maps of the first SFF and the last maxpooling layer in the three feature extraction networks, and outputs the final multimodal and multiscale fused feature map.
[0045] The bone age assessment subnetwork consists of an average pooling layer, a fully connected layer, CELU, a Dropout layer, and another fully connected layer. The loss function used is the Softmax-Mean-Variant loss function. It is used to generate the final predicted age from the fused features of T1WI, T2WI, and PDWI.
[0046] The embodiments of this invention provide an automated, radiation-free bone age assessment model that integrates multiple features and is trained, including:
[0047] The image preprocessing module is used to resample, perform bias field correction, and normalize the T1WI, T2WI, and PDWI images of users whose bone age is to be assessed.
[0048] The bone age assessment module is used to assess bone age based on processed user MRI T1WI, T2WI, and PDWI images of the bone age to be assessed using a trained, multi-feature fusion-based automated, radiation-free bone age assessment model.
[0049] To demonstrate the inventiveness and technical value of the technical solution of this invention, this section provides specific product or related technology application examples of the technical solution claimed.
[0050] It should be noted that embodiments of the present invention can be implemented in hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by a suitable instruction execution system, such as a microprocessor or dedicated-design hardware. Those skilled in the art will understand that the above-described devices and methods can be implemented using computer-executable instructions and / or included in processor control code, for example, such code provided on a carrier medium such as a disk, CD, or DVD-ROM, a programmable memory such as read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuitry such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field-programmable gate arrays, programmable logic devices, etc., or by software executed by various types of processors, or by a combination of the above-described hardware circuitry and software, such as firmware.
[0051] The inclusion criteria for this invention are: no history of disease, trauma, or surgery affecting knee joint development, and knee MRI examinations including T1WI, T2WI, and PDWI. All images were obtained using FSE sequences; sagittal T1WI, sagittal T2WI, and coronal PDWI.
[0052] The deep learning model framework provided in this embodiment of the invention is named AMKM-Net3D. AMKM-Net3D consists of three parts: a feature extraction network, a multimodal multiscale feature fusion module, and a bone age assessment subnetwork.
[0053] The feature extraction network provided in this embodiment of the invention is designed for T1WI, T2WI and PDWI of MRI respectively. Each feature extraction network has one 1×7×7 convolutional layer and four R-P3D modules. Each convolutional layer is followed by a normalization layer and a CELU activation function. Each R-P3D module is followed by a max pooling layer to obtain the feature map of each modality.
[0054] The multimodal and multiscale feature fusion module includes two selective feature fusion (SFF) modules. The first SFF module is used to fuse the feature maps of the second-to-last maxpooling layer in the three feature extraction networks. The second SFF module is used to fuse the feature maps of the first SFF and the last maxpooling layer in the three feature extraction networks, and outputs the final multimodal and multiscale fused feature map.
[0055] The bone age assessment subnetwork consists of an average pooling layer, a fully connected layer, CELU, a Dropout layer, and another fully connected layer. The loss function used is the Softmax-Mean-Variant loss function. It is used to generate the final predicted age from the fused features of T1WI, T2WI, and PDWI.
[0056] For model training, since knee MRI is a three-dimensional image composed of multiple stacked two-dimensional images, and the images at both ends contain soft tissue structures without effective bone information, 12 intermediate slices from the knee MRI were selected and resampled to 12×384×384. Then, all MRI images underwent N4ITK bias field correction. Furthermore, this embodiment of the invention uses min-max normalization for image normalization. After normalization, the intensity in each slice is normalized to [0; 1], which is beneficial for the training process of deep learning methods.
[0057] To obtain robust age estimates, a 10-fold cross-validation strategy was employed. During the generation of each fold, the training set was stratified by age and gender using different random seeds. Meanwhile, the test set remained independent and invariant. The average age of the 10-fold cross-validation results was used as the final predicted age.
[0058] This embodiment of the invention implements AMKM-Net3D in PyTorch and trains it on a server using an NVIDIA ARTX 2060 super GPU. By default, the AdamW optimization algorithm is used during network training, with a weight decay rate of 0.001. The maximum number of training iterations in this embodiment is 60 epochs. The initial learning rate is set to 0.0001.
[0059] Network evaluation was performed using an independent test set that did not overlap with the training set. Mean absolute error (MAE) and the correlation between actual age and predicted age were calculated.
[0060] result
[0061] Sample features
[0062] A total of 1794 MRI images from 598 patients were included (mean age 20.51 ± 5.64 years, 245 females). Each sample underwent multimodal knee MRI examination, including T1WI, T2WI, and PDWI. The training sample consisted of 478 patients (mean age 20.53 ± 5.63 years, 196 females). The test sample consisted of 120 patients (mean age 20.44 ± 5.68 years, 49 females). Sample characteristics are detailed in Table 1.
[0063] Table 1: Sample Characteristics
[0064] Total (n=598) Training set (n=478) Test set (n=120) gender male 353 282 71 female 245 196 49 Age (y)* 20.51±5.64 20.53±5.63 20.44±5.68 10-13 years old 102 80 22 14-17 years old 113 92 21 18-21 years old 124 99 25 22-25 years old 133 106 27 26-29 years old 126 101 25
[0065] *Data are expressed as mean ± standard deviation
[0066] Table 2 shows the evaluation performance of different modalities on the test set. In the 10-29 year old test group, T2WI performed best in single modality. Compared with various MRI methods, multimodal 3D MRI with T1WI, T2WI, and PDWI performed best, with a MAE value of 1.61±1.23 years for 10-29 year olds and 1.32±1.01 years for 10-25 year olds. The results indicate that AMKM-Net3D based on multimodal MRI has superior performance and high potential for age estimation.
[0067] Table 2. Performance comparison of different models and MRI modalities on the test set.
[0068]
[0069] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. A method for bone age assessment based on MRI and deep neural networks, characterized in that, The bone age assessment method based on MRI and deep neural networks includes: By using deep learning algorithms to comprehensively analyze three types of weighted MRI images, an automated, radiation-free bone age assessment model with multi-feature fusion is constructed. The bone age assessment model is then used to automatically identify, extract, and fuse effective features of MRI three-dimensional images to obtain bone age assessment results. The MRI-based bone age assessment method includes the following steps: Step 1: Resample, perform bias field correction, and normalize the MRI scans of users whose bone age is to be assessed; Step 2: Automatically extract T1WI, T2WI and PDWI image features using three feature extraction networks respectively; Step 3: Automatically fuse T1WI, T2WI, and PDWI image features using the multimodal multiscale feature fusion module; Step 4: Use the bone age assessment subnetwork to perform bone age assessment based on the fused feature map; The automated, radiation-free bone age assessment model based on multi-feature fusion includes: Feature extraction networks were designed for T1WI, T2WI, and PDWI of MRI. Each feature extraction network consisted of one 1×7×7 convolutional layer and four R-P3D modules. Each convolutional layer was followed by a normalization layer and a CELU activation function. Each R-P3D module was followed by a max pooling layer to obtain the feature map of each modality. The multimodal and multiscale feature fusion module includes two selective feature fusion (SFF) modules. The first SFF module is used to fuse the feature maps of the second-to-last maxpooling layer in the three feature extraction networks. The second SFF module is used to fuse the feature maps of the first SFF and the last maxpooling layer in the three feature extraction networks, and outputs the final multimodal and multiscale fused feature map. The bone age assessment subnetwork consists of an average pooling layer, a fully connected layer, CELU, a Dropout layer, and a fully connected layer. The loss function adopted is the Softmax-Mean-Variane loss function. It is used to generate the final predicted age from the fused features of T1WI, T2WI, and PDWI.
2. An MRI-based bone age assessment system implementing the MRI and deep neural network-based bone age assessment method of claim 1, characterized by, The MRI-based bone age assessment system includes: (1) Image preprocessing module, used to resample, bias field correct and normalize the T1WI, T2WI and PDWI of the user's MRI to be evaluated for bone age; (2) Bone age assessment module, which uses a trained multi-feature fusion automated radiation-free bone age assessment model to assess bone age based on processed user MRI T1WI, T2WI and PDWI images of the bone age to be assessed.