On-line bone age and bone mineral density detection method based on hand normal position X-ray image
Through the online detection method of bone age bone density based on hand orthotopic X-ray image, the U-net and UNet++ network combined with openCV and elastic energy algorithm is used to solve the problem of low accuracy, low efficiency and cost detection of bone age and bone density in the prior art. High problems achieve automated simultaneous detection and efficient and reliable detection results.
Patent Information
- Application Number
- CN202311326643.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-13
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2043-10-13
AI Technical Summary
The existing bone age and bone density detection methods have problems such as low detection accuracy, low efficiency, high cost and poor interpretability of the detection results. In particular, the feature extraction and recognition algorithm based on palm X-ray images requires strict and robust sample data. Insufficient and unavailable to detect bone age and bone density simultaneously.
The bone age bone density online detection method based on hand orthotopic X-ray image is adopted. The hand recognition and bone recognition model is built through the streamlined U-net network and UNet++ network, and combined with the openCV contour search algorithm and elastic energy algorithm to achieve automated Bone age and bone density detection, using a pre-constructed epiphyseal grade score table and bone maturity standard function, SMS score values and bone density values were calculated.
Simultaneous detection of bone age and bone density is achieved, detection efficiency is improved, detection cost is reduced, and detection accuracy and reliability are improved by using the model generated by Chinese children's sample data.
Smart Images

Figure CN119090794A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of medical detection equipment, and in particular relates to an online detection method for bone age and bone density based on hand X-ray images, and a corresponding online detection system and equipment for bone age and bone density. Background Art
[0002] Bone age and bone density are important indicators of the user's bone development status, and they play a guiding role in early detection of developmental delays in adolescents and children and providing corresponding solutions. There are two main methods for detecting bone age and bone density. One is manual analysis based on the user's X-ray images by experienced orthopedic doctors, and the other is artificial intelligence algorithm recognition based on machine learning.
[0003] Of the above two detection schemes, the former has higher detection accuracy, but is limited by the number and energy of experienced medical personnel, so the efficiency of this detection scheme is relatively low. Although the latter can achieve large-scale rapid detection, it still has the following problems: (1) The algorithm training stage uses labeled data of samples from a specific region, which is not suitable for bone age assessment of Chinese children. In some regions, there are large errors in the assessment results. (2) Most of the existing bone age detection algorithms and bone density detection algorithms directly extract and identify features based on medical images of the palm, and output the corresponding detection results. The interpretability of the evaluation results is poor and cannot be verified. (3) The existing various bone age detection algorithms and bone density detection algorithms have strict requirements on sample data, the robustness of the algorithms is insufficient, and the detection accuracy is easily affected by the image quality. In the presence of interference information, the accuracy of the detection results will be significantly reduced. (4) In the existing schemes, the two indicators of bone age and bone density are usually measured separately, and it is impossible to perform a one-time detection through the same medical image, which will increase the detection cost of users. Summary of the invention
[0004] In order to solve the problem that the existing bone age and bone density detection schemes cannot meet the requirements in terms of detection accuracy, detection efficiency and detection cost, the present invention provides an online bone age and bone density detection method based on hand frontal X-ray images, and a corresponding online bone age and bone density detection system and equipment.
[0005] The present invention is implemented by the following technical solutions:
[0006] A method for online detection of bone age and bone density based on hand X-ray images, comprising the following steps:
[0007] S1: Obtain the gender information of the current user and the corresponding X-ray image of the hand.
[0008] S2: A hand recognition model for identifying and segmenting the palm and forearm is built and trained in advance using a streamlined U-net network; and a hand image with the background removed is generated based on the hand X-ray image.
[0009] S3: A hand bone recognition model for identifying 29 types of hand bones is built and trained in advance using a four-layer full-node UNet++ network; and the hand bone recognition model is used to identify and locate various types of bones in the hand image.
[0010] S4: Based on the recognition and positioning results of the previous step, the contour search algorithm in openCV is used to generate the minimum circumscribed rectangular area of the 13 types of characteristic bones contained therein, and image segmentation is performed to obtain the corresponding 13 characteristic bone images Pc i , i=1…13.
[0011] S5: Preliminarily use the three-layer full-node UNet++ network to build and train an epiphysis classification model for classifying the epiphysis grade of the characteristic bone; and use the epiphysis classification model to generate each characteristic bone image Pc i Corresponding epiphyseal grade Rank i .
[0012] S6: Combine the current user's gender and the 13 epiphyseal levels to query a pre-built "epiphyseal level score table" to obtain the score corresponding to each bone level. i , and then use the following formula to calculate the corresponding SMS score:
[0013]
[0014] S7: Input the SMS score value into a pre-fitted 50 percentile bone maturity standard function and output the corresponding bone age value.
[0015] S8: Select the second metacarpal bone image from the 13 characteristic bone images in step S4; analyze the second metacarpal bone image and generate a bone density value, the process is as follows:
[0016] S81: Using an elastic energy algorithm to correct the contour of the second metacarpal bone image.
[0017] S82: Perform rotation correction on the image corrected in the previous step.
[0018] S83: Differentiate the regions corresponding to the cortical bone and the cancellous bone in the second metacarpal bone image according to a preset grayscale threshold.
[0019] S84: Calculate the average thickness of cortical bone and cancellous bone D 皮 and D 松 .
[0020] S85: Calculate the metacarpal index MCI of the second metacarpal of the current user: MCI=D 皮 / (D 松 +D 皮 );.
[0021] S86: Based on the mapping relationship between the metacarpal index and the bone density, the corresponding bone density value is calculated according to the metacarpal index.
[0022] As a further improvement of the present invention, the hand recognition model is a fully convolutional neural network using a UNet network as the backbone network. The backbone network includes 4 downsampling layers and 4 upsampling layers. Each downsampling layer uses a fully convolutional relu activation sublayer with a kernel of 3, a batch normalization sublayer, and a maximum pooling sublayer. Each upsampling layer uses a fusion layer, a fully convolutional relu activation sublayer, and an upsampling sublayer. Then it passes through a 2-channel feature fully convolutional layer with a kernel of 1, and then outputs a 2-classification feature image through softmax. The last layer output of the fully convolutional neural network will be compared with the pre-labeled supervision data t n The cross entropy error calculation is performed, and then the network back propagation is performed to learn and correct the parameters of each layer. The network of the hand recognition model can learn to obtain the binary classification image features of the background and the hand contour edge.
[0023] As a further improvement of the present invention, the hand bone recognition model is a fully convolutional neural network using the UNet++ network as the backbone network; the backbone network includes 4 downsampling layers and multiple upsampling layers. Each downsampling layer uses a fully convolutional relu activation sublayer with a kernel of 3, a batch normalization sublayer, and a maximum pooling sublayer. Each upsampling layer uses a fusion layer, a fully convolutional relu activation sublayer, and an upsampling sublayer. Then it passes through a 10-channel feature fully convolutional layer with a kernel of 1, and then outputs a 29-classification feature image through softmax. The last layer output of the fully convolutional neural network will be compared with the pre-labeled supervision data t n The cross entropy error calculation is performed, and then the back propagation of the network is performed to learn and correct the parameters of each layer. The network of the hand recognition model can learn 29 classification image features of 29 different characteristic bones.
[0024] As a further improvement of the present invention, in step S5, the epiphyseal classification model adopts a streamlined 3-layer UNet++ network; its backbone network is divided into two paths for back propagation. One path uses the same full convolutional neural network as the hand bone recognition model to perform cross-entropy error operation learning supervision on the feature layer output and label, and then performs Flatten dimensionality reduction operation on the output image, extracts feature information through the Dense1024 fully connected layer and the Dense256 fully connected layer, and then fuses with the feature information of another path. The other path extracts features from each final layer X30, X21, X12, and X03 and processes them through the Dense256 fully connected layer, and then performs feature fusion through the Concatenate layer; then fuses the feature information of the previous path through the Desne256 fully connected layer. Finally, it is output as 15 epiphyseal grade classifications through Dense15; and the cross-entropy operation and back propagation with the label complete the learning and training.
[0025] As a further improvement of the present invention, the 29 types of bones identified by the hand bone recognition model in step S3 include: ulna, radius, hamate, capitate, triquetrum, lunate, trapezium, trapezium, scaphoid, pisiform, metacarpals I to V, proximal phalanges I to V, distal phalanges I to V, and middle phalanges II to V;
[0026] In step S4, the 13 types of characteristic bones segmented include: radius, ulna, metacarpal bone I, metacarpal bone III, metacarpal bone V, proximal phalanx I, proximal phalanx III, proximal phalanx V, middle phalanx III, middle phalanx V, distal phalanx I, distal phalanx III, distal phalanx V.
[0027] As a further improvement of the present invention, in step S6, the "epiphyseal grade score table" is a first data comparison table established with reference to the "RUS-CHN method bone development grade score table" in the "Chinese adolescent and child wrist bone maturity and evaluation method" TY / T 3001-2006.
[0028] The first data comparison table establishes score values corresponding to the epiphyseal grades of 13 characteristic bones; the first data comparison table is divided into two types according to gender; namely, "the first data comparison table (male)" and "the first data comparison table (female)", which are respectively suitable for epiphyseal grade scoring for males and females.
[0029] As a further improvement of the present invention, in step S7, the 50 percentile bone maturity standard function is a function fitted according to the 50 percentile curve of the "RUS-CHN Bone Maturity Percentile Standard Curve" in the "Chinese Adolescent and Children's Wrist Bone Maturity and Evaluation Method" TY / T 3001-2006, whose independent variable is the SMS score value and the dependent variable is the corresponding bone age test result.
[0030] Among them, the 50 percentile bone maturity standard function is also divided into two types according to gender, namely "50 percentile bone maturity standard function (male)" and "50 percentile bone maturity standard function (female)". The two are respectively suitable for predicting bone age for males and females.
[0031] As a further improvement of the present invention, in step S81, the energy function E used in the elastic energy algorithm is total as follows:
[0032]
[0033] In the above formula, represents a contour line connected by a set of coordinate points; δ is the symbol for obtaining the derivative; d is the differential symbol; s represents the independent variable describing the boundary in the form of Fourier transform, s∈[0,1]; for The modulus of the first derivative of represents the elastic energy; for The modulus of the second derivative, the bending energy; α represents the weighting coefficient of elastic energy; β represents the weighting coefficient of bending energy; It is a function that calculates the degree of fit between the deformation curve and the original curve, and the function value represents the external energy.
[0034] The value of the energy function is minimized through iteration to obtain the corresponding second metacarpal bone image after contour correction.
[0035] The present invention also includes an online detection system for bone age and bone density, which adopts the online detection method for bone age and bone density based on the hand frontal X-ray image as described above, and outputs the detection results of the current user's bone age and bone density based on the input hand frontal X-ray image and gender information of any user.
[0036] The online detection system of bone age and bone density includes: data input module, image preprocessing module, hand bone classification module, feature bone extraction module, epiphyseal grade recognition module, SMS score generation module, bone age output module, contour correction module, metacarpal index generation module, and bone density output module.
[0037] The data input module is used to obtain the gender information of the user to be detected and the X-ray image of the hand.
[0038] The image preprocessing module uses a hand recognition model pre-created based on a streamlined U-net network to perform feature recognition on the hand X-ray image, identify the hand bones and soft tissue parts, and generate a hand image after removing the background.
[0039] The hand bone classification module uses a hand bone recognition model created based on a four-layer full-node UNet++ network to perform bone recognition and classification on the hand image output by the image preprocessing module, and then obtains feature images corresponding to 29 types of hand bones.
[0040] The feature bone extraction module first obtains the minimum circumscribed rectangle of the outer contours of 13 feature bones through the contour search algorithm in openCV. A template image corresponding to the inner area of the circumscribed rectangle is generated; then the template image and the original hand X-ray image are used for mask operation to obtain 13 feature bone images.
[0041] The epiphysis grade recognition module is used to classify the input feature bone image using an epiphysis classification model created based on a three-layer full-node UNet++ network to obtain the corresponding epiphysis grade classification result. During network training, Gaussian noise, salt and pepper noise, and logarithmic compression methods are used for data enhancement to solve the problem that the output structure is easily disturbed by noise.
[0042] The SMS score generation module is used to select the corresponding epiphyseal grade score table according to gender, and then query the bone grade score table and assign points according to the epiphyseal grades of the 13 input feature bone images. Finally, the scoring results are added up to obtain the SMS score value of the current user.
[0043] The bone age output module is used to input the SMS score value of the current user into a pre-fitted 50 percentile bone maturity standard function and output the corresponding bone age value.
[0044] The contour correction module is used to obtain the second metacarpal bone image from the 13 characteristic bone images, and use the elastic energy algorithm to correct its contour, and then perform rotation correction on the corrected image.
[0045] The metacarpal index generation module processes the corrected second metacarpal image using a sampling threshold method, identifies the cortical bone and cancellous bone therein, and then calculates the ratio of the average thickness of the cortical bone to the cancellous bone, which is the required metacarpal index of the second metacarpal.
[0046] The bone density output module is based on the mapping relationship between the metacarpal index and bone density, and calculates the corresponding bone density value according to the metacarpal index.
[0047] The present invention also includes an online detection device for bone age and bone density, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, an online detection system for bone age and bone density as described above is created, and then the detection results of the bone age and bone density of the current user are automatically generated according to the gender information of the input detection user and the X-ray image of the hand in the positive position.
[0048] The technical solution provided by the present invention has the following beneficial effects:
[0049] The present invention classifies and detects the bones in the user's hand anteroposterior image through the created network, extracts images of 13 types of characteristic bones, and uses the images of characteristic bones to detect bone age and bone density. The method provided by the present invention is a purely digital solution, and the processes of medical image recognition, bone classification, epiphyseal grade, epiphyseal grade scoring and bone age value prediction, as well as metacarpal index extreme and bone density value prediction can all be automatically processed online. The user only needs to provide an anteroposterior X-ray image of the hand and gender information to obtain the test results of bone age and bone density at the same time, which greatly improves the detection efficiency of medical institutions and reduces the detection cost of users.
[0050] In addition, in the detection scheme provided by the present invention, the network model used and various comparison tables generated by statistical data are all generated through sample data of Chinese children, so the detection accuracy is higher and the reliability is stronger. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] Figure 1 This is a flowchart of the steps of an online detection method for bone age and bone density based on hand frontal X-ray images provided in Example 1 of the present invention.
[0052] Figure 2 This is a method principle diagram of an online detection method for bone age and bone density based on hand frontal X-ray images provided in Example 1 of the present invention.
[0053] Figure 3 This is a typical frontal X-ray image of the hand.
[0054] Figure 4 This is the network architecture of the hand recognition model designed in Example 1 of the present invention.
[0055] Figure 5 This is the network architecture of the hand bone recognition model designed in Example 1 of the present invention.
[0056] Figure 6 This is a schematic diagram of generating various characteristic bone image segmentation results according to the output of the hand bone recognition model in Example 1 of the present invention.
[0057] Figure 7 This is the network architecture of the hand recognition model designed in Example 1 of the present invention.
[0058] Figure 8 This is the function graph of the 50 percentile bone maturity standard function (male) fitted in Example 1 of the present invention.
[0059] Fig. 9This is the function graph of the 50 percentile bone maturity standard function (female) fitted in Example 1 of the present invention.
[0060] Fig.10 This is a module architecture diagram of the online detection system for bone age and bone density provided in Example 1 of the present invention.
[0061] Fig.11 Schematic diagram of the training strategy for the hand bone recognition model in the performance testing experiment.
[0062] Fig.12 This is the loss function image of the hand bone recognition model during the training phase in the performance testing experiment.
[0063] Figure 13-Figure 15 Shown are the results of bone recognition and positioning of three hand X-ray images in the performance testing experiment.
[0064] Fig.16 This is a typical feature bone recognition result obtained by using the online detection system of bone age and bone density in the performance testing experiment.
[0065] Fig.17 This is the image of the second metacarpal bone corrected using the elastic energy algorithm in the performance test experiment.
[0066] Fig.18 Loss function image of the epiphysis classification model in the performance testing experiment.
[0067] Fig.19 A bone age examination report for a female user produced by an online bone age and bone density detection system in a performance test experiment.
[0068] Fig. 20 A bone age examination report for a male user produced by an online bone age and bone density detection system in a performance test experiment. DETAILED DESCRIPTION
[0069] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0070] Example 1
[0071] This embodiment provides a method for online detection of bone age and bone density based on X-ray images of the hand. Figure 1 and Figure 2 As shown, it includes the following steps:
[0072] S1: Obtain the gender information of the current user and the corresponding X-ray image of the hand.
[0073] Hand X-ray examination is a routine standardized hand bone examination item; a typical hand X-ray image is a medical image taken by a medical X-ray machine with the user's palm facing down. Figure 3 As shown in the figure, the X-ray image of the hand in the anteroposterior position contains 29 types of bones of the human hand, namely: ulna, radius, hamate, capitate, triquetrum, lunate, trapezium, trapezium, scaphoid, pisiform, metacarpals I to V, proximal phalanges I to V, distal phalanges I to V, middle phalanges II to V. And the soft tissues around the bones. In addition, the X-ray image of the hand in the anteroposterior position also includes inevitable background images.
[0074] S2: A hand recognition model for identifying and segmenting the palm and forearm is built and trained in advance using a streamlined U-net network; and a hand image with the background removed is generated based on the hand X-ray image.
[0075] In this embodiment, Figure 4 As shown in the figure, the hand recognition model is a fully convolutional neural network with a UNet network as the backbone network. The backbone network includes 4 downsampling layers and 4 upsampling layers. Each downsampling layer uses a fully convolutional relu activation sublayer with a kernel of 3, a batch normalization sublayer, and a maximum pooling sublayer. Each upsampling layer uses a fusion layer, a fully convolutional relu activation sublayer, and an upsampling sublayer. Then it passes through a 2-channel feature fully convolutional layer with a kernel of 1, and then outputs a 2-class feature image through softmax. The output of the last layer of the fully convolutional neural network will be compared with the pre-labeled supervision data t n Perform cross entropy error calculation, and then perform back propagation of the network to learn and correct the parameters of each layer. The network of the hand recognition model can learn to obtain binary classification image features of the background and hand contour edges.
[0076] In the training phase of the hand recognition model, the input used is an 8-bit image dataset in the format of (n, w, h, c). Among them, n represents the number of samples in the sample dataset in the training phase; w is the sample width, which is set to 288 in this embodiment; h is the sample height, which is set to 320 in this embodiment; c represents the number of sample channels, and this embodiment only uses the segmented hand images as training set samples for training, so c is set to 1.
[0077] S3: A hand bone recognition model for identifying 29 types of hand bones is built and trained in advance using a four-layer full-node UNet++ network; and the hand bone recognition model is used to identify and locate various types of bones in the hand image.
[0078] In this embodiment, Figure 5As shown in the figure, the hand bone recognition model is a fully convolutional neural network with UNet++ network as the backbone network; the backbone network includes 4 downsampling layers and multiple upsampling layers. Each downsampling layer uses a fully convolutional relu activation sublayer with a kernel of 3, a batch normalization sublayer, and a maximum pooling sublayer. Each upsampling layer uses a fusion layer, a fully convolutional relu activation sublayer, and an upsampling sublayer. Then it passes through a 10-channel feature fully convolutional layer with a kernel of 1, and then outputs a 29-classification feature image through softmax. The output of the last layer of the fully convolutional neural network will be compared with the pre-labeled supervision data t n The cross entropy error calculation is performed, and then the back propagation of the network is performed to learn and correct the parameters of each layer. The network of the hand recognition model can learn 29 classification image features of 29 different characteristic bones.
[0079] In the training phase of the hand bone recognition model, the input used is an 8-bit image dataset in the format of (n, w, h, c). Among them, n represents the number of samples in the sample dataset in the training phase; w is the sample width, which is set to 288 in this embodiment; h is the sample height, which is set to 320 in this embodiment; c represents the number of sample channels. This embodiment uses 29 different types of hand bones and sample images of soft tissues after segmentation as training set samples to train the hand bone recognition model, so c in the hand bone recognition model is set to 30.
[0080] S4: Based on the recognition and positioning results of the previous step, the contour search algorithm in openCV is used to generate the minimum circumscribed rectangular area of the 13 types of characteristic bones contained therein, and image segmentation is performed to obtain the corresponding 13 characteristic bone images Pc i , i = 1 ... 13. The 13 types of characteristic bones segmented include: radius, ulna, metacarpal Ⅰ, metacarpal Ⅲ, metacarpal Ⅴ, proximal phalanx Ⅰ, proximal phalanx Ⅲ, proximal phalanx Ⅴ, middle phalanx Ⅲ, middle phalanx Ⅴ, distal phalanx Ⅰ, distal phalanx Ⅲ, distal phalanx Ⅴ.
[0081] In this embodiment, Figure 6As shown, the detailed steps for generating the characteristic bone image are as follows: First, obtain the hand image A output by step S2 and the recognition and positioning results of the 29 characteristic bones generated by step S3. Then, select the recognition and positioning results corresponding to 13 types of characteristic bones, and use the contour search algorithm in openCV to generate the minimum circumscribed rectangular area of the 13 types of characteristic bones contained therein. Next, generate a template image M corresponding to each characteristic bone based on the minimum circumscribed rectangle of the 13 types of characteristic bones. Among them, the template image is a binary image, and the area inside the image corresponding to the minimum circumscribed rectangle is assigned a value of 1, and the area outside the minimum circumscribed rectangle is assigned a value of 0. Finally, the template image is used to perform a mask operation on the original hand image A to obtain 13 images that only include local information of the area inside the corresponding minimum external rectangle in the hand image, that is, the characteristic bone image Pc. i , i=1…13.
[0082] S5: Preliminarily use the three-layer full-node UNet++ network to build and train an epiphysis classification model for classifying the epiphysis grade of the characteristic bone; and use the epiphysis classification model to generate each characteristic bone image Pc i Corresponding epiphyseal grade Rank i .
[0083] like Figure 7 As shown, the epiphyseal classification model provided in this embodiment adopts a streamlined 3-layer UNet++ network; its backbone network is divided into two paths for back propagation. One path uses the same full convolutional neural network as the hand bone recognition model to perform cross-entropy error operation learning supervision on the feature layer output and label, and then performs Flatten dimensionality reduction operation on the output image, extracts feature information through the Dense1024 fully connected layer and the Dense256 fully connected layer, and then fuses with the feature information of another path. The other path extracts features from each final layer X30, X21, X12, and X03 and processes them through the Dense256 fully connected layer, and then performs feature fusion through the Concatenate layer; then fuses the feature information of the previous path through the Desne256 fully connected layer. Finally, the output is 15 epiphyseal grade classifications through Dense15; and the cross-entropy operation and back propagation with the label complete the learning and training.
[0084] The strategy of training epiphysis classification model in the present embodiment is: first, a large number of medical images of 13 characteristic bones are obtained as sample images, and the data of each characteristic bone in the sample image is guaranteed to contain samples of corresponding 15 bone grades (0-14 grades). Then, the bone grade in the sample image is manually identified by professional medical personnel, and a label of the bone grade is added. Then, in order to improve the robustness of the network model, Gaussian noise, salt and pepper noise and logarithmic compression method are also used in the sample image for data enhancement, which solves the problem that the output structure is easily disturbed by noise. And the epiphysis grade label identical with the original image is added to the sample image after amplification. Finally, the designed epiphysis classification model is trained using the sample data set with labels, and the direct branch reaches the preset training target or iteration termination condition.
[0085] It should be emphasized that the hand recognition model, hand bone recognition model and epiphyseal classification model designed in this embodiment all use the following cross entropy error E as the optimization function during the training stage.
[0086]
[0087] In the above formula, t nk Represents the label information of the kth feature element of the nth sample data image input, y nk Represents the classification result of the kth feature element of the output nth sample data image.
[0088] S6: Combine the current user's gender and the 13 epiphyseal levels to query a pre-built "epiphyseal level score table" to obtain the score corresponding to each bone level. i , and then use the following formula to calculate the corresponding SMS score:
[0089]
[0090] In this embodiment, the "epiphyseal grade score table" is a first data comparison table established with reference to the "RUS-CHN method bone development grade score table" in the "Chinese adolescent and child wrist bone maturity and evaluation method" TY / T 3001-2006.
[0091] The first data comparison table establishes score values corresponding to the epiphyseal grades of 13 characteristic bones; the first data comparison table is divided into two types according to gender; namely, the "first data comparison table (male)" as shown in Table 1 and the "first data comparison table (female)" as shown in Table 2, which are respectively suitable for epiphyseal grade scoring for males and females.
[0092] Table 1: First data comparison table (male)
[0093]
[0094] Table 2: First data comparison table (female)
[0095]
[0096]
[0097] The usage of the first data comparison table (male) and the first data comparison table (female) in this embodiment is as follows: first determine the type of comparison table to be queried according to the gender of the user, and then select the epiphyseal grade Rank of the 13 characteristic bones in turn. i Query the corresponding first data comparison table to obtain the corresponding score value Score i Finally, the 13 score values Score1~Score 13 The corresponding SMS score is obtained by adding them up.
[0098] In actual application, this embodiment encodes the 13 characteristic bone categories as a, a=1~13; encodes the 15 epiphyseal grades as b, b=0~14; then establishes a matrix representing the first data comparison table (male) and the first data comparison table (female), namely Form1(male) and Form1(female); wherein the rows of the matrix are encoded according to the bone grade, the columns of the matrix are encoded according to the characteristic bone type, and each element in the matrix is the corresponding score value. Next, the matrix Form1(male) and the matrix Form1(female) are respectively converted into vector representations to obtain a vector table. The format of each vector in the vector table is (a, b, Score a,b ), where Score a,b It indicates the score value corresponding to the bone level of the a-th characteristic bone when it is b. Finally, use a and b as the search conditions to look up the table and get the required score value.
[0099] S7: Input the SMS score into a pre-fitted 50th percentile bone maturity standard function f stard , output the corresponding bone age value.
[0100] Bone age = f stard (SMS)
[0101] In the present embodiment, the 50 percentile bone maturity standard function is a function fitted by the 50 percentile curve of the "RUS-CHN bone maturity percentile standard curve" in the "Chinese adolescent and child wrist bone maturity and evaluation method" TY / T 3001-2006, and its independent variable is the SMS score value, and the dependent variable is the corresponding bone age detection result. The obtained 50 percentile bone maturity standard function is also divided into two types according to gender, namely "50 percentile bone maturity standard function (male)" and "50 percentile bone maturity standard function (female)". The two are respectively applicable to bone age prediction for males and females.
[0102] In the design process of this embodiment, Matlab is used as a tool for function fitting. The function images of the 50th percentile bone maturity standard functions of males and females are shown as follows: Figure 8 and Fig. 9 shown.
[0103] S8: Select the second metacarpal bone image from the 13 characteristic bone images in step S4; analyze the second metacarpal bone image and generate a bone density value, the process is as follows:
[0104] S81: Use the elastic energy algorithm to correct the contour of the second metacarpal bone image. The energy function E used in the elastic energy algorithm of this embodiment total as follows:
[0105]
[0106] In the above formula, represents a contour line connected by a set of coordinate points; δ is the symbol for obtaining the derivative; d is the differential symbol; s represents the independent variable describing the boundary in the form of Fourier transform, s∈[0,1]; for The modulus of the first derivative of represents the elastic energy; for The modulus of the second derivative, the bending energy; α represents the weighting coefficient of elastic energy; β represents the weighting coefficient of bending energy; It is a function that calculates the degree of fit between the deformation curve and the original curve, and the function value represents the external energy.
[0107] The value of the energy function is minimized through iteration to obtain the corresponding second metacarpal bone image after contour correction.
[0108] S82: Perform rotation correction on the image corrected in the previous step.
[0109] S83: Differentiate the regions corresponding to the cortical bone and the cancellous bone in the second metacarpal bone image according to a preset grayscale threshold.
[0110] S84: Calculate the average thickness of cortical bone and cancellous bone D皮 and D 松 .
[0111] S85: Calculate the metacarpal index MCI of the second metacarpal of the current user: MCI=D 皮 / (D 松 +D 皮 );.
[0112] S86: Based on the mapping relationship between the metacarpal index and the bone density, the corresponding bone density value is calculated according to the metacarpal index. In this embodiment, the functional relationship between the bone density value and the metacarpal index is as follows:
[0113] Bone density value = (MCI+4) / 46.5.
[0114] Thus, in the online detection method of bone age and bone density based on the hand anteroposterior X-ray image provided by this embodiment, only the anteroposterior X-ray image of the hand and the gender of any user need to be obtained to simultaneously realize the dual detection of bone age and bone density, and the detection efficiency is higher. At the same time, the solution of this embodiment can also avoid the problem that the user needs to take two X-rays (hand and second metacarpal bone), so that the detection cost is significantly reduced.
[0115] Example 2
[0116] This embodiment provides an online detection system for bone age and bone density, which adopts the online detection method for bone age and bone density based on the frontal X-ray image of the hand as in Embodiment 1, and outputs the detection results of the bone age and bone density of the current user according to the input frontal X-ray image of the hand and gender information of any user.
[0117] like Fig.10 As shown, the online detection system for bone age and bone density includes: a data input module, an image preprocessing module, a hand bone classification module, a feature bone extraction module, an epiphyseal grade recognition module, an SMS score generation module, a bone age output module, a contour correction module, a metacarpal index generation module, and a bone density output module.
[0118] The data input module is used to obtain the gender information of the user to be detected and the X-ray image of the hand.
[0119] The image preprocessing module uses a hand recognition model pre-created based on a streamlined U-net network to perform feature recognition on the hand X-ray image, identify the hand bones and soft tissue parts, and generate a hand image after removing the background.
[0120] The hand bone classification module uses a hand bone recognition model created based on a four-layer full-node UNet++ network to perform bone recognition and classification on the hand image output by the image preprocessing module, and then obtains feature images corresponding to 29 types of hand bones.
[0121] The feature bone extraction module first obtains the minimum circumscribed rectangle of the outer contours of 13 feature bones through the contour search algorithm in openCV. A template image corresponding to the inner area of the circumscribed rectangle is generated; then the template image and the original hand X-ray image are used for mask operation to obtain 13 feature bone images.
[0122] The epiphysis grade recognition module is used to classify the input feature bone image using an epiphysis classification model created based on a three-layer full-node UNet++ network to obtain the corresponding epiphysis grade classification result.
[0123] The SMS score generation module is used to select the corresponding epiphyseal grade score table according to gender, and then query the bone grade score table and assign points according to the epiphyseal grades of the 13 input feature bone images. Finally, the scoring results are added up to obtain the SMS score value of the current user.
[0124] The bone age output module is used to input the SMS score value of the current user into a pre-fitted 50 percentile bone maturity standard function and output the corresponding bone age value.
[0125] The contour correction module is used to obtain the second metacarpal bone image from the 13 characteristic bone images, and use the elastic energy algorithm to correct its contour, and then perform rotation correction on the corrected image.
[0126] The metacarpal index generation module processes the corrected second metacarpal image using a sampling threshold method, identifies the cortical bone and cancellous bone therein, and then calculates the ratio of the average thickness of the cortical bone to the cancellous bone, which is the required metacarpal index of the second metacarpal.
[0127] The bone density output module is based on the mapping relationship between the metacarpal index and bone density, and calculates the corresponding bone density value according to the metacarpal index.
[0128] Example 3
[0129] This embodiment provides an online detection device for bone age and bone density, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, an online detection system for bone age and bone density as in Embodiment 2 is created, and then the detection results of the bone age and bone density of the current user are automatically generated according to the input detection user's gender information and the X-ray image of the hand in the positive position.
[0130] The online detection device for bone age and bone density in this embodiment is essentially a computer device, which can be connected to a medical X-ray fluoroscopy device as a back-end device for analyzing and processing the acquired X-ray images of the hand in the anteroposterior position. The computer device can be a smart phone, tablet computer, laptop computer, desktop computer, rack server, blade server, tower server or cabinet server (including an independent server, or a server cluster composed of multiple servers) that can execute programs. The computer device of this embodiment includes at least but is not limited to: a memory and a processor that can communicate with each other through a system bus.
[0131] In this embodiment, the memory (i.e., readable storage medium) includes flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory may be an internal storage unit of a computer device, such as a hard disk or memory of the computer device. In other embodiments, the memory may also be an external storage device of a computer device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the computer device. Of course, the memory may also include both the internal storage unit of the computer device and its external storage device. In this embodiment, the memory is generally used to store an operating system and various application software installed on the computer device. In addition, the memory may also be used to temporarily store various types of data that have been output or are to be output.
[0132] The processor may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip in some embodiments. The processor is generally used to control the overall operation of a computer device.
[0133] Performance Testing
[0134] According to the schemes of Examples 1 and 2, the inventors created the corresponding online detection system for bone age and bone density, and trained and tested it. Among them, the hand bone recognition model is one of the key points of the design of the present invention. In order to achieve the goal of multi-classification, the present invention uses the UNet++ network as the backbone network and adopts Fig.11 The created network model is trained using the training method. The training results are as follows: Fig.12As shown in the figure. It can be seen from the data in the figure that after about 100 rounds of training, the performance average index of the hand bone recognition model designed by the present invention has converged, and the recognition accuracy can meet the requirements. Furthermore, this experiment also selected three samples to verify the performance of the hand bone recognition model. The verification results are as follows: Figure 13-Figure 15 As shown, according to the data in the figure, it can be seen that the solution of the present invention can accurately identify 29 types of hand bones in the hand X-ray image after removing the background information.
[0135] In the online bone age and bone density detection system created in this experiment, the position distribution of the 13 characteristic bones identified in the hand image is as follows Fig.16 The image of the second metacarpal bone corrected by the elastic energy algorithm is shown in Fig.17 The loss curve of the epiphyseal grade classification model during the training phase is shown in Fig.18 shown. Fig.19 and Fig. 20 These are examination reports of two users, one female and one male, generated by the system designed by the present invention.
[0136] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A method for online detection of bone age and bone density based on X-ray images of the hand, characterized in that: It includes the following steps: S1: Obtain the gender information of the current user and the corresponding X-ray image of the hand; S2: Preliminarily using a simplified U-net network to build and train a hand recognition model for identifying and segmenting the palm and forearm; and using the hand recognition model to generate a hand image with the background removed based on the hand X-ray image; S3: Preliminarily using a four-layer full-node UNet++ network to build and train a hand bone recognition model for identifying 29 types of hand bones; and using the hand bone recognition model to identify and locate various types of bones in the hand image; S4: Based on the recognition and positioning results of the previous step, the contour search algorithm in openCV is used to generate the minimum circumscribed rectangular area of the 13 types of characteristic bones contained therein, and image segmentation is performed to obtain the corresponding 13 characteristic bone images Pc i , i=1……13; S5: Preliminarily build and train an epiphysis classification model for classifying the epiphysis grade of the characteristic bone using a three-layer full-node UNet++ network; and generate each characteristic bone image Pc using the epiphysis classification model i Corresponding epiphyseal grade Rank i ; S6: Combine the current user's gender and the 13 epiphyseal levels to query a pre-built "epiphyseal level score table" to obtain the score corresponding to each bone level. i , and then use the following formula to calculate the corresponding SMS score: S7: inputting the SMS score value into a pre-fitted 50 percentile bone maturity standard function, and outputting the corresponding bone age value; S8: Select the second metacarpal bone image from the 13 characteristic bone images in step S4; analyze the second metacarpal bone image and generate a bone density value, the process is as follows: S81: performing contour correction on the second metacarpal bone image using an elastic energy algorithm; S82: performing rotation correction on the image corrected in the previous step; S83: differentiating the regions corresponding to the cortical bone and the cancellous bone in the second metacarpal bone image according to a preset grayscale threshold; S84: Calculate the average thickness D of the cortical bone and cancellous bone 皮 and D 松 ; S85: Calculate the metacarpal index MCI of the second metacarpal of the current user: MCI=D 皮 / (D 松 +D 皮 ); S86: Based on the following mapping relationship between metacarpal index and bone density, the corresponding bone density value is calculated: Bone density value = (MCI+4) / 46.
5.
2. The method for online detection of bone age and bone density based on hand X-ray images as claimed in claim 1, characterized in that: The hand recognition model is a fully convolutional neural network using the UNet network as the backbone network; the backbone network includes 4 downsampling layers and 4 upsampling layers, each downsampling layer uses a fully convolutional relu activation sublayer with a kernel of 3, a batch normalization sublayer, and a maximum pooling sublayer; each upsampling layer uses a fusion layer, a fully convolutional relu activation sublayer, and an upsampling sublayer; then it passes through a 2-channel feature fully convolutional layer with a kernel of 1, and then outputs a 2-classification feature image through softmax; the output of the last layer of the fully convolutional neural network will be compared with the pre-labeled supervision data t n Perform cross entropy error calculation, and then perform back propagation of the network to learn and correct the parameters of each layer.
3. The method for online detection of bone age and bone density based on hand X-ray images as claimed in claim 1, characterized in that: The hand bone recognition model is a fully convolutional neural network using the UNet++ network as the backbone network; the backbone network includes 4 downsampling layers and multiple upsampling layers, each downsampling layer uses a fully convolutional relu activation sublayer with a kernel of 3, a batch normalization sublayer, and a maximum pooling sublayer; each upsampling layer uses a fusion layer, a fully convolutional relu activation sublayer, and an upsampling sublayer; then it passes through a 10-channel feature fully convolutional layer with a kernel of 1, and then outputs a 29-classification feature image through softmax; the output of the last layer of the fully convolutional neural network will be compared with the pre-labeled supervision data t n Perform cross entropy error calculation, and then perform back propagation of the network to learn and correct the parameters of each layer.
4. The method for online detection of bone age and bone density based on hand X-ray images as claimed in claim 1, characterized in that: In step S5, the epiphyseal classification model adopts a streamlined 3-layer UNet++ network; its backbone network is divided into two paths for back propagation; one path adopts the same full convolutional neural network as the hand bone recognition model to perform cross-entropy error operation learning supervision on the feature layer output and the label, and then performs Flatten dimensionality reduction operation on the output image, extracts feature information through Dense1024 fully connected layer and Dense256 fully connected layer, and then fuses it with the feature information of another path; the other path extracts features from each final layer X30, X21, X12, and X03 and processes it through the Dense256 fully connected layer, and then performs feature fusion through the Concatenate layer; then it is fused with the feature information of the previous path through the Desne256 fully connected layer; finally, it is output as 15 epiphyseal grade classification through Dense15, and cross-entropy operation and back propagation with the label are performed to complete learning and training.
5. The method for online detection of bone age and bone density based on hand X-ray images as claimed in claim 3, characterized in that: The 29 types of bones identified by the hand bone recognition model in step S3 include: ulna, radius, hamate, capitate, triquetrum, lunate, trapezium, trapezium, scaphoid, pisiform, metacarpals I to V, proximal phalanges I to V, distal phalanges I to V, and middle phalanges II to V; In step S4, the 13 types of characteristic bones segmented include: radius, ulna, metacarpal bone I, metacarpal bone III, metacarpal bone V, proximal phalanx I, proximal phalanx III, proximal phalanx V, middle phalanx III, middle phalanx V, distal phalanx I, distal phalanx III, distal phalanx V.
6. The method for online detection of bone age and bone density based on hand X-ray images as claimed in claim 1, characterized in that: In step S6, the "epiphyseal grade score table" is a first data comparison table established with reference to the "RUS-CHN method bone development grade score table" in the "Chinese adolescent and child wrist bone maturity and evaluation method" TY / T 3001-2006. The first data comparison table establishes score values corresponding to the epiphyseal grade of 13 characteristic bones; the first data comparison table is divided into two types according to gender.
7. The method for online detection of bone age and bone density based on hand X-ray images as claimed in claim 1, characterized in that: In step S7, the 50 percentile bone maturity standard function is a function fitted according to the 50 percentile curve of the "RUS-CHN Bone Maturity Percentile Standard Curve" in the "Chinese Adolescent and Children's Wrist Bone Maturity and Evaluation Method" TY / T 3001-2006, whose independent variable is the SMS score value and the dependent variable is the corresponding bone age test result; the 50 percentile bone maturity standard function is also divided into two types according to gender.
8. The method for online detection of bone age and bone density based on hand X-ray images as claimed in claim 1, characterized in that: In step S81, the energy function E used in the elastic energy algorithm total as follows: In the above formula, represents a contour line connected by a set of coordinate points; δ is the symbol for obtaining the derivative; d is the differential symbol; s represents the independent variable describing the boundary in the form of Fourier transform, s∈[0,1]; for The modulus of the first derivative of represents the elastic energy; for The modulus of the second derivative, the bending energy; α represents the weighting coefficient of elastic energy; β represents the weighting coefficient of bending energy; is a function that calculates the degree of conformity of the deformation curve to the original curve, and the function value represents the external energy; The value of the energy function is minimized through iteration to obtain the corresponding second metacarpal bone image after contour correction.
9. An online detection system for bone age and bone density, characterized in that: The method for online detection of bone age and bone density based on hand anteroposterior X-ray images as claimed in claim 1 is adopted to output the detection results of the bone age and bone density of the current user according to the input anteroposterior X-ray images of the hands and gender information of any user; the online detection system comprises: A data input module, which is used to obtain the gender information of the user to be detected and the X-ray image of the hand; An image preprocessing module, which uses a hand recognition model pre-created based on a streamlined U-net network to perform feature recognition on the hand X-ray image, recognize the hand bones and soft tissue parts therein, and generate a hand image after removing the background; A hand bone classification module, which uses a hand bone recognition model created based on a four-layer full-node UNet++ network to perform bone recognition and classification on the hand image output by the image preprocessing module, thereby obtaining feature images corresponding to 29 types of hand bones; The feature bone extraction module first obtains the minimum circumscribed rectangle of the outer contours of 13 feature bones through the contour search algorithm in OpenCV, and then generates a template image corresponding to the internal area of the circumscribed rectangle; finally, the template image and the original hand X-ray image are used for mask operation to obtain 13 feature bone images; The epiphysis grade recognition module uses an epiphysis classification model created based on a three-layer full-node UNet++ network to classify the input feature bone image and obtain the corresponding epiphysis grade classification result; An SMS score generation module is used to select the corresponding epiphyseal grade score table according to gender, and then query the bone grade score table and assign points according to the epiphyseal grades of the 13 input characteristic bone images, and finally add up the assigned points to obtain the SMS score value of the current user; A bone age output module is used to input the SMS score value of the current user into a pre-fitted 50 percentile bone maturity standard function and output the corresponding bone age value; A contour correction module is used to obtain the second metacarpal bone image from the 13 characteristic bone images, and use an elastic energy algorithm to correct its contour, and then perform rotation correction on the corrected image; a metacarpal index generation module, which processes the corrected second metacarpal image using a sampling threshold method, identifies the cortical bone and cancellous bone therein, and then calculates the ratio of the average thickness of the cortical bone to the cancellous bone, which is the required metacarpal index of the second metacarpal; and The bone density output module is used to calculate the corresponding bone density value according to the metacarpal index based on the mapping relationship between the metacarpal index and the bone density.
10. An online detection device for bone age and bone density, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it creates the online bone age and bone density detection system as described in claim 9, and then automatically generates the bone age and bone density detection results of the current user based on the input detection user's gender information and the frontal X-ray image of the hand.
Citation Information
Patent Citations
Bone age evaluation method for realizing image segmentation and classification based on deep learning
CN111080579A
Weighted bone age evaluation method and system based on deep learning
CN113570618A
Radius density detection method, system and device based on deep learning
CN113920088A
Double-bone-age evaluation method based on joint global and local convolutional neural network features
CN114663735A
Bone health index measuring and calculating method for extracting region of interest based on Unet
CN116072296A
Cited By
Bone age assessment method and system combining deep learning and logic correction segmentation
CN120977590A
