A knowledge distillation-based bone density prediction method and system
By using a knowledge distillation-based method to map and fuse features from X-ray and CT images, a bone density prediction model is trained. This solves the problem of existing technologies being unable to effectively utilize hospital data, enabling accurate prediction of bone density using only X-ray data, thus reducing screening costs and fracture risk.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHEJIANG UNIV
- Filing Date
- 2023-08-03
- Publication Date
- 2026-05-12
AI Technical Summary
Current technology lacks an effective method to predict bone mineral density from X-ray plain film data, and cannot make full use of the existing CT data resources in hospitals.
A knowledge distillation-based approach is adopted, which uses computer vision and deep learning technologies to map the features of X-ray plain films and CT images to the same feature space and then stitches and fuses them. The knowledge distillation technique is used to train the bone density prediction regressors of teacher and student models, so as to predict bone density from X-ray plain film data.
It enables accurate prediction of bone density using only X-ray plain film data, reducing the cost of osteoporosis screening, lowering the risk of fractures, and improving the efficiency of early screening.
Smart Images
Figure CN117058082B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer vision, and more specifically to a bone density prediction method and system based on knowledge distillation. Background Technology
[0002] Osteoporosis is a systemic skeletal disease characterized by decreased bone density and quality, and damage to bone microstructure, leading to increased bone fragility and a greater susceptibility to fractures, as well as an increased incidence of lung infections. Bone mineral density (BMD) is the preferred method for diagnosing osteoporosis; its measurement alone can confirm the diagnosis. Plain X-rays are a cost-effective and widely applicable imaging modality. Predicting BMD values from X-ray data can aid in early screening, reducing the cost of osteoporosis screening and lowering the risk of fractures.
[0003] Currently, there is a lack of existing technology for predicting bone mineral density values from X-ray films, and it is also impossible to fully utilize the existing data in hospitals. Summary of the Invention
[0004] The present invention aims to overcome the above-mentioned shortcomings of the prior art and provide a bone mineral density prediction method and system based on knowledge distillation.
[0005] To address the aforementioned technical problems, this invention proposes a bone mineral density prediction method based on knowledge distillation, the method comprising:
[0006] S110 uses computer vision technology to extract image features from X-ray plain films and CT (computed tomography) scans respectively. The teacher model contains two types of image features, while the student model only has X-ray plain film image features.
[0007] S120 uses deep learning technology to map the 2D image features of X-ray films and the 3D image features of CT scans in the teacher model to the same feature space.
[0008] S130 uses the 2D image features of X-ray films and the 3D image features of CT scans in the same feature space of the teacher model as the fused features.
[0009] S140 uses the methods of fusion features and knowledge distillation to distill the fusion features of the teacher model and the 2D image features of the X-ray plain film of the student model and calculates the distillation loss.
[0010] S150, through online distillation, jointly trains the bone density prediction regressor of the teacher model and the student model.
[0011] Furthermore, in step S110, image features of X-ray plain films and CT (computed tomography) scans are extracted using computer vision technology. The teacher model contains both types of image features, while the student model only contains X-ray plain film image features. Specifically, these features include:
[0012] S1101, using a 2D-ResNet network model pre-trained on a natural image dataset to extract image feature maps g from X-ray films. x .
[0013] S1102, using a 3D-ResNet network model pre-trained on a medical image dataset to extract image feature maps g from CT data. ct .
[0014] S1103, the extracted image feature maps are processed using 2D global average pooling and 3D global average pooling techniques to obtain feature vectors v. x and v ct .
[0015] Furthermore, step S120, which describes using deep learning technology to map the 2D image features of X-ray films and the 3D image features of CT scans in the teacher model to the same feature space, specifically includes:
[0016] S1201, through the design of a multilayer perceptron f x In the teacher model, the feature vector v x Mapping to a certain feature space yields the feature vector v fx That is, v fr =f x (v x ).
[0017] S1202, by designing a multilayer perceptron f ct In the teacher model, the feature vector v ct Mapping to v fx The feature space yields the feature vector v fct That is, v fct =f ct (v ct ).
[0018] Furthermore, step S130, which involves stitching and fusing the 2D image features of X-ray films and the 3D image features of CT scans in the same feature space of the teacher model as the fused features, specifically includes:
[0019] S1301, the feature vector v of the X-ray film fx With the feature vector v of CT fct The feature vector v is obtained by concatenating the features. r =[v fx ,vfct ].
[0020] S1302, through the design of a multilayer perceptron f r v fx and v fct The feature vector v after simple concatenation r Further fusion yields the feature vector v fr That is, v fr =f r (v r ).
[0021] Furthermore, step S140, which utilizes fusion features and knowledge distillation to perform distillation between the fusion features of the teacher model and the 2D image features of the student model's X-ray film and calculates the distillation loss, specifically includes:
[0022] S1401, through the design of a multilayer perceptron f sx The 2D image feature vector v of the X-ray film in the student model x Mapping to the fused feature vector v fr In the same feature space, and with the same dimensions, we obtain feature vectors v. sx That is, v sx =f sx (v x ).
[0023] S1402, using knowledge distillation to extract the fusion features v of the teacher model. fr distilled to the student model v sx In this way, the student model only needs to use X-ray film data for prediction, and calculates the distillation loss using the mean squared error loss function (MSE). z loss z =MSE(v sx v fr ).
[0024] Furthermore, step S150, which describes jointly training the bone density prediction regressor of the teacher model and the student model through online distillation, specifically includes:
[0025] S1501, In designing the teacher model, the feature vector v is fused. fr A multilayer perceptron regressor m serves as input for bone density prediction. t ;
[0026] S1502, utilizing this multilayer perceptron regressor m t Based on the fusion feature vector v fr The predicted value of bone mineral density, y1, is obtained, i.e., y1 = m. t (v fr ).
[0027] S1503, Design the student model using v processed from X-ray film feature vectors. sx A multilayer perceptron regressor m serves as input for bone density prediction. s ;
[0028] S1504, utilizing this multilayer perceptron regressor m t v after processing the feature vector of X-ray flat film sx The predicted value of bone mineral density, y2, is obtained, i.e., y2 = m. s (v sx ).
[0029] S1505 calculates the loss between the true bone mineral density y and y1 and y2 using the mean squared error loss function (MSE). t loss x loss t =MSE(y1, y), loss s =MSE(y2, y).
[0030] S1506, designed a weighted loss method combining three loss methods. z loss t loss s The model is trained using this as the final loss.
[0031] Furthermore, step S1506 describes the design of a weighted loss method that combines three loss methods. z loss t loss s The model is trained using the final loss, specifically including:
[0032] (1) Loss t loss x Apply equal weights to learning.
[0033] (2) Assign loss through learnable parameter a z The weights, parameter 'a', will be normalized to between 0 and 1, and the remaining weights will be assigned to (loss). t +loss s ).
[0034] (3) Final loss last =a*loss z +(1-a)*(loss t +loss s ).
[0035] A bone mineral density prediction system based on knowledge distillation, comprising:
[0036] The image feature extraction module uses computer vision technology to extract image features from X-ray plain films and computed tomography (CT) scans. The teacher model contains two types of image features, while the student model only contains X-ray plain film image features.
[0037] The image feature mapping module uses deep learning technology to map the 2D image features of X-ray films and the 3D image features of CT scans in the teacher model to the same feature space.
[0038] The image feature stitching and fusion module stitches and fuses the 2D image features of X-ray films and the 3D image features of CT scans in the same feature space in the teacher model as the fused features;
[0039] The distillation module uses fusion feature and knowledge distillation methods to distill the fusion features of the teacher model and the 2D image features of the X-ray plain film of the student model and calculate the distillation loss.
[0040] The bone mineral density prediction regressor training module trains both the teacher and student models' bone mineral density prediction regressors through online distillation.
[0041] The beneficial effects of this invention are as follows: The bone mineral density prediction method based on knowledge distillation provided by this invention involves extracting image features from X-ray films and CT scans (computed tomography) using computer vision technology. The teacher model contains both types of image features, while the student model only contains X-ray film image features. Deep learning technology is used to map the 2D image features of X-ray films and the 3D image features of CT scans in the teacher model to the same feature space. The 2D image features of X-ray films and the 3D image features of CT scans in the same feature space of the teacher model are then concatenated and fused to form a fused feature. The fused feature and knowledge distillation method are used to distill the fused feature of the teacher model and the 2D image features of X-ray films in the student model and calculate the distillation loss. The bone mineral density prediction regressors of the teacher model and the student model are jointly trained through online distillation. This invention helps to distill a teacher model that utilizes existing CT data (which can be flexibly replaced with any other available data in the hospital) and X-ray plain film data into a student model that only uses X-ray plain film data. This helps the model to accurately predict bone density values using only X-ray data during the prediction phase, enabling patients to screen for osteoporosis earlier, reducing the cost of osteoporosis screening, and lowering the risk of fractures. Attached Figure Description
[0042] The present invention will be further described below with reference to the accompanying drawings and embodiments.
[0043] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation
[0044] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.
[0045] Example 1
[0046] like Figure 1 As shown in the figure, this embodiment 1 provides a bone mineral density prediction method based on knowledge distillation. The method includes: proposing a solution for predicting bone mineral density from X-ray films using existing hospital data.
[0047] Specifically, the method includes:
[0048] S110 uses computer vision technology to extract image features from X-ray plain films and CT (computed tomography) scans. The teacher model contains both types of image features, while the student model only contains X-ray plain film image features.
[0049] Specifically, a 2D-ResNet network model pre-trained on a natural image dataset is used to extract image feature maps from X-ray films, and a 3D-ResNet network model pre-trained on a medical image dataset is used to extract image feature maps from CT data. The extracted image feature maps are then subjected to 2D global average pooling and 3D global average pooling techniques to obtain feature vectors v. x and v ct .
[0050] S120 uses deep learning technology to map the 2D image features of X-ray films and the 3D image features of CT scans in the teacher model to the same feature space.
[0051] Specifically, by designing a multilayer perceptron, the feature vector v in the teacher model is... x Mapping to a certain feature space yields the feature vector v fx By designing a multilayer perceptron, the feature vector v is incorporated into the teacher model. ct Mapping to v fx The feature space yields the feature vector v fct .
[0052] S130 uses the 2D image features of X-ray films and the 3D image features of CT scans in the same feature space of the teacher model as the fused features.
[0053] Specifically, the feature vector v of the X-ray film fx With the feature vector v of CT fct The feature vector v is obtained by concatenating the features. r By designing a multilayer perceptron, v fx and v fctThe feature vector v after simple concatenation r Further fusion yields the feature vector v fr .
[0054] S140 uses the methods of fusion features and knowledge distillation to distill the fusion features of the teacher model and the 2D image features of the X-ray plain film of the student model and calculates the distillation loss.
[0055] Specifically, by designing a multilayer perceptron f sx The 2D image feature vector v of the X-ray film in the student model x Mapping to the fused feature vector v fr In the same feature space, and with the same dimensions, we obtain feature vectors v. sx The fusion features v of the teacher model are obtained through knowledge distillation. fr distilled to the student model v sx In this way, the student model only needs to use X-ray film data for prediction, and calculates the distillation loss using the mean squared error loss function (MSE). z .
[0056] S150, through online distillation, jointly trains the bone density prediction regressor of the teacher model and the student model.
[0057] Specifically, in designing the teacher model, the feature vector v is fused. fr A multilayer perceptron regressor is used as input for bone density prediction, and this multilayer perceptron regressor is used based on the fused feature vector v fr The predicted value of bone mineral density y1 is obtained, and the v value after processing with the feature vector of X-ray film is designed in the student model. sx A multilayer perceptron regressor is used as input for bone density prediction; this multilayer perceptron regressor is used to process the v based on the feature vector of the X-ray film. sx The predicted bone mineral density value y2 is obtained, and the loss between the true bone mineral density y and y1 and y2 is calculated using the mean squared error loss function MSE. t loss x Design a weighted loss method to calculate the loss. t loss x Equal-weighted learning is applied, with the loss parameter 'a' assigned to each learnable parameter. z The weights, parameter 'a', will be normalized to between 0 and 1, and the remaining weights will be assigned to (loss). t +loss s ), final loss last =a*loss z +(1-a)*(loss t +loss s ).
[0058] In summary, the bone mineral density prediction method based on knowledge distillation provided by this invention involves extracting image features from X-ray films and CT scans using computer vision technology. The teacher model contains both types of image features, while the student model only contains X-ray film image features. Deep learning technology is used to map the 2D image features of X-ray films and the 3D image features of CT scans in the teacher model to the same feature space. The 2D image features of X-ray films and the 3D image features of CT scans in the same feature space of the teacher model are then concatenated and fused to form the fused features. The fused features and knowledge distillation are used to distill the fused features of the teacher model and the 2D image features of X-ray films in the student model, and the distillation loss is calculated. The bone mineral density prediction regressors of the teacher model and the student model are trained together through online distillation. This invention helps to distill a teacher model that utilizes existing CT data (which can be flexibly replaced with any other available data in the hospital) and X-ray plain film data into a student model that only uses X-ray plain film data. This helps the model to accurately predict bone density values using only X-ray data during the prediction phase, enabling patients to screen for osteoporosis earlier, reducing the cost of osteoporosis screening, and lowering the risk of fractures.
[0059] Example 2
[0060] This embodiment relates to a bone mineral density prediction system based on knowledge distillation, used to implement the method of Embodiment 1, including:
[0061] The image feature extraction module uses computer vision technology to extract image features from X-ray plain films and computed tomography (CT) scans. The teacher model contains two types of image features, while the student model only contains X-ray plain film image features.
[0062] The image feature mapping module uses deep learning technology to map the 2D image features of X-ray films and the 3D image features of CT scans in the teacher model to the same feature space.
[0063] The image feature stitching and fusion module stitches and fuses the 2D image features of X-ray films and the 3D image features of CT scans in the same feature space in the teacher model as the fused features;
[0064] The distillation module uses fusion feature and knowledge distillation methods to distill the fusion features of the teacher model and the 2D image features of the X-ray plain film of the student model and calculate the distillation loss.
[0065] The bone mineral density prediction regressor training module trains both the teacher and student models' bone mineral density prediction regressors through online distillation.
[0066] This invention helps predict bone mineral density values using only X-ray plain film data during the prediction stage, enabling patients to screen for osteoporosis earlier, reducing the cost of osteoporosis screening, and lowering the risk of fractures.
[0067] Based on the above-described preferred embodiments of the present invention, and through the foregoing description, those skilled in the art can make various changes and modifications without departing from the inventive concept. The technical scope of this invention is not limited to the contents of the specification, but must be determined according to the scope of the claims.
Claims
1. A bone mineral density prediction method based on knowledge distillation, characterized in that, Includes the following steps: S110 uses computer vision technology to extract image features from X-ray plain film and electronic computed tomography CT scan. The teacher model contains two types of image features, while the student model only has X-ray plain film image features. S120 uses deep learning technology to map the 2D image features of X-ray films and the 3D image features of CT scans in the teacher model to the same feature space. S130, the 2D image features of X-ray films and the 3D image features of CT scans in the same feature space of the teacher model are stitched together and fused as the fusion features; S140, using fusion feature and knowledge distillation methods, distills the fusion features of the teacher model and the 2D image features of the student model's X-ray plain film, and calculates the distillation loss; including: S1401, through the design of a multilayer perceptron The 2D image feature vector of the X-ray film in the student model Mapping to fused feature vector Feature vectors are obtained in the same feature space and with the same dimensions. ,Right now S1402, using knowledge distillation to fuse the features of the teacher model. Distillation to the student model In this way, the student model only needs to use X-ray film data for prediction, and calculates the distillation loss using the mean squared error loss function (MSE). ,Right now ; S150, through online distillation, jointly trains the bone mineral density prediction regressor for both the teacher and student models; including: S1501, Designing a teacher model using fused feature vectors A multilayer perceptron regressor used as input for bone density prediction ; S1502, utilizing this multilayer perceptron regressor Based on the fusion feature vector Obtain predicted values of bone mineral density. ,Right now ; S1503, Design student model after processing with X-ray film feature vectors A multilayer perceptron regressor used as input for bone density prediction ; S1504, utilizing this multilayer perceptron regressor Based on the processed feature vector of X-ray film Obtain predicted values of bone mineral density. ,Right now ; S1505 calculates the true bone mineral density y using the mean squared error loss function MSE. Losses between ,Right now ( ), ( ); S1506, designed a weighted loss method combining three... The model is trained using this as the final loss.
2. The bone mineral density prediction method based on knowledge distillation as described in claim 1, characterized in that, Step S110 includes: S1101, using a 2D-ResNet network model pre-trained on a natural image dataset to extract image feature maps from X-ray films. ; S1102, using a 3D-ResNet network model pre-trained on a medical image dataset to extract image feature maps from CT data. ; S1103, the extracted image feature maps are processed using 2D global average pooling and 3D global average pooling techniques to obtain feature vectors. and .
3. The bone mineral density prediction method based on knowledge distillation as described in claim 1, characterized in that, Step S120 includes: S1201, through the design of a multilayer perceptron In the teacher model, feature vectors Mapping to a certain feature space yields feature vectors. ,Right now S1202, through the design of a multilayer perceptron In the teacher model, feature vectors Mapped to Feature space yields feature vectors ,Right now 4. The bone mineral density prediction method based on knowledge distillation as described in claim 1, characterized in that, Step S130 includes: S1301, converts the feature vector of the X-ray film... With the feature vector of CT The feature vector is obtained by concatenating the features. =[ ]; S1302, through the design of a multilayer perceptron Will and Simple concatenation of feature vectors Further fusion yields the feature vector. ,Right now 5. The bone mineral density prediction method based on knowledge distillation as described in claim 1, characterized in that, Step S1506 includes: (1) Apply equal weights to learning; (2) Through learnable parameters To give Weights, parameters It will be normalized to between 0 and 1, and the remaining weights will be assigned to ( ); (3) Final loss .
6. A bone mineral density prediction system based on knowledge distillation, characterized in that, include: The image feature extraction module uses computer vision technology to extract image features from X-ray plain films and computed tomography (CT) scans. The teacher model contains two types of image features, while the student model only contains X-ray plain film image features. The image feature mapping module uses deep learning technology to map the 2D image features of X-ray films and the 3D image features of CT scans in the teacher model to the same feature space. The image feature stitching and fusion module stitches and fuses the 2D image features of X-ray films and the 3D image features of CT scans in the same feature space of the teacher model as the fused features; including: S1401, through the design of a multilayer perceptron The 2D image feature vector of the X-ray film in the student model Mapping to fused feature vector Feature vectors are obtained in the same feature space and with the same dimensions. ,Right now S1402, using knowledge distillation to fuse the features of the teacher model. Distillation to the student model In this way, the student model only needs to use X-ray film data for prediction, and calculates the distillation loss using the mean squared error loss function (MSE). ,Right now ; The distillation module utilizes fusion feature and knowledge distillation methods to distill the fused features of the teacher model and the 2D image features of the student model's X-ray plain film, and calculates the distillation loss; including: S1501, Designing a teacher model using fused feature vectors A multilayer perceptron regressor used as input for bone density prediction ; S1502, utilizing this multilayer perceptron regressor Based on the fusion feature vector Obtain predicted values of bone mineral density. ,Right now ; S1503, Design student model after processing with X-ray film feature vectors A multilayer perceptron regressor used as input for bone density prediction ; S1504, utilizing this multilayer perceptron regressor Based on the processed feature vector of X-ray film Obtain predicted values of bone mineral density. ,Right now ; S1505 calculates the true bone mineral density y using the mean squared error loss function MSE. Losses between ,Right now ( ), ( ); S1506, designed a weighted loss method combining three... The model is trained using this as the final loss. The bone mineral density prediction regressor training module trains both the teacher and student models' bone mineral density prediction regressors through online distillation.