An online method for detecting bone age and bone density based on hand posteroanterior x-ray images

Abstract

The application belongs to the field of medical detection equipment, and particularly relates to a bone age bone density online detection method based on a hand orthotopic X-ray image, and a corresponding bone age bone density online detection system and equipment. The scheme firstly creates and trains a high-precision hand recognition model, a hand bone recognition module and an epiphysis grade recognition module, which are respectively used for realizing image optimization, 29 types of hand epiphysis recognition and 13 types of characteristic bone epiphysis grading. Then, scoring and accumulation are performed in combination with the epiphysis grade, and bone age is predicted. A metacarpal index is calculated according to a second metacarpal image, and a bone density value prediction result is obtained. The scheme provided by the application is a pure digital scheme, does not require manual prediction or intervention by a doctor, has high detection efficiency, can generate two results of bone age and bone density through one examination material, and has low detection cost. The application solves the problem that the existing detection scheme cannot meet the requirements in terms of precision, efficiency and cost.

Description

Technical Field

[0001] This invention belongs to the field of medical testing equipment, specifically relating to an online method for detecting bone age and bone mineral density based on anteroposterior X-ray images of the hand, and the corresponding online detection system and equipment for bone age and bone mineral density. Background Technology

[0002] Bone age and bone mineral density are important indicators reflecting a user's skeletal development, and they play a guiding role in the early detection of developmental delays in children and adolescents, and in providing corresponding solutions. Currently, there are two main methods for detecting bone age and bone mineral density: one is manual analysis by experienced orthopedic surgeons based on the user's X-ray images, and the other is identification using artificial intelligence algorithms based on machine learning.

[0003] Of the two detection schemes mentioned above, the former has higher detection accuracy, but its efficiency is relatively low due to the limited number and energy of experienced medical personnel. The latter, although capable of large-scale rapid detection, still has the following problems: (1) The algorithm training stage uses labeled data of samples from specific regions, which is not suitable for bone age assessment of Chinese children, resulting in large errors in assessment results in some regions. (2) Most existing bone age detection algorithms and bone density detection algorithms are based on direct feature extraction and recognition from medical images of the palm, and output corresponding detection results. The interpretability of the evaluation results is poor and cannot be verified. (3) Existing bone age detection algorithms and bone density detection algorithms have strict requirements for sample data, and the robustness of the algorithms is insufficient. The detection accuracy is easily affected by the image quality. In the presence of interference information, the accuracy of the detection results will decrease significantly. (4) In existing schemes, bone age and bone density are usually measured separately, and cannot be detected at the same time using the same medical image, thus increasing the user's detection cost. Summary of the Invention

[0004] To address the shortcomings of existing bone age and bone mineral density (BMD) detection methods in terms of accuracy, efficiency, and cost, this invention provides an online BMD detection method based on anteroposterior X-ray images of the hand, along with a corresponding online BMD detection system and device.

[0005] This invention is achieved using the following technical solution:

[0006] An online method for detecting bone age and bone mineral density based on anteroposterior X-ray images of the hand includes the following steps:

[0007] S1: Obtain the current user's gender information and the corresponding orthogonal X-ray image of their hand.

[0008] S2: A hand recognition model for identifying and segmenting the palm and forearm is pre-built and trained using a simplified U-net network; and a hand image with background removed is generated based on the orthogonal X-ray image of the hand using the hand recognition model.

[0009] S3: A hand bone recognition model for recognizing 29 types of bones in the hand is pre-built and trained using a four-layer full-node UNet++ network; and the hand bone recognition model is used to recognize and locate various types of bones in hand images.

[0010] S4: Based on the recognition and localization results from the previous step, the minimum bounding rectangle region containing the 13 types of feature bones is generated using the contour-finding algorithm in OpenCV, and image segmentation is performed to obtain the corresponding 13 feature bone images Pc. i , i = 1……13.

[0011] S5: A three-layer, fully node UNet++ network is used to build and train an epiphyseal classification model for classifying the epiphyseal grade of feature bones; and the epiphyseal classification model is used to generate Pc for each feature bone image. i Corresponding epiphyseal rank i .

[0012] S6: Combine the current user's gender and the 13 epiphyseal plate levels to query a pre-built "epithelial plate level score table" and obtain the score value corresponding to each bone level. i Then, the corresponding SMS score is calculated using the following formula:

[0013]

[0014] S7: Input the SMS score into a pre-fitted 50th percentile bone maturity standard function and output the corresponding bone age value.

[0015] S8: Select the second metacarpal image from the 13 feature bone images in step S4; analyze the second metacarpal image and generate bone density values, as follows:

[0016] S81: Use the elastic energy algorithm to correct the contour of the second metacarpal image.

[0017] S82: Perform rotation correction on the image after the previous step.

[0018] S83: Distinguish between the cortical bone and cancellous bone regions in the second metacarpal image based on a preset grayscale threshold.

[0019] S84: Statistical analysis of the average thickness D of cortical and cancellous bone 皮 and D 松 .

[0020] S85: Calculate the metacarpal index (MCI) of the second metacarpal bone of the current user: MCI = D 皮 / (D 松 +D 皮 );

[0021] S86: Based on the mapping relationship between the metacarpal index and bone mineral density, the corresponding bone mineral density value is calculated according to the metacarpal index.

[0022] As a further improvement to this invention, the hand recognition model is a fully convolutional neural network using a UNet network as its backbone. The backbone network comprises four downsampling layers and four upsampling layers. Each downsampling layer uses a fully convolutional ReLU activation sub-layer with a kernel of 3, a batch normalization sub-layer, and a max-pooling sub-layer. Each upsampling layer uses a fusion layer, a fully convolutional ReLU activation sub-layer, and an upsampling sub-layer. Then, a 2-channel feature fully convolutional layer with a kernel of 1 is used, followed by a softmax function to output a 2-classification feature image. The final output of the fully convolutional neural network is compared with pre-labeled supervision data t. n Cross-entropy error calculation is performed, followed by backpropagation of the network to learn and correct the parameters of each layer. The hand recognition model network can learn binary image features of the background and hand contour edges.

[0023] As a further improvement to this invention, the hand bone recognition model is a fully convolutional neural network using a UNet++ network as its backbone; the backbone network includes four downsampling layers and multiple upsampling layers. Each downsampling layer uses a fully convolutional ReLU activation sub-layer with a kernel of 3, a batch normalization sub-layer, and a max pooling sub-layer. Each upsampling layer uses a fusion layer, a fully convolutional ReLU activation layer, and an upsampling sub-layer. Then, a 10-channel feature fully convolutional layer with a kernel of 1 is used, followed by a softmax function to output a 29-classification feature image. The output of the final layer of the fully convolutional neural network is compared with pre-labeled supervision data t. n Cross-entropy error calculation is performed, followed by backpropagation of the network to learn and correct the parameters of each layer. The hand recognition model network can learn 29 classification image features of 29 different bone features.

[0024] As a further improvement to this invention, in step S5, the epiphyseal classification model adopts a simplified 3-layer UNet++ network; its backbone network performs backpropagation in two paths. One path uses the same fully convolutional neural network as the hand bone recognition model to perform cross-entropy error calculation on the feature layer output and labels for supervised learning. Then, Flatten dimensionality reduction is performed on the output image, and feature information is extracted through Dense1024 and Dense256 fully connected layers, and then fused with the feature information from the other path. The other path extracts features from each final layer X30, X21, X12, and X03, processes them through a Dense256 fully connected layer, and then performs feature fusion through a Concatenate layer; then it is fused with the feature information from the previous path through a Dense256 fully connected layer. Finally, the Dense15 output is a 15-level epiphyseal classification; and cross-entropy calculation is performed with the labels for backpropagation to complete the learning and training.

[0025] As a further improvement of the present invention, the 29 types of bones identified by the hand bone recognition model in step S3 include: ulna, radius, hamate, capitate, triquetrum, lunate, trapezium, trapezium scaphoid, pisiform, metacarpals I-V, proximal phalanges I-V, distal phalanges I-V, and middle phalanges II-V;

[0026] In step S4, the 13 types of characteristic bones that are segmented include: radius, ulna, metacarpal I, metacarpal III, metacarpal V, proximal phalanx I, proximal phalanx III, proximal phalanx V, middle phalanx III, middle phalanx V, distal phalanx I, distal phalanx III, and distal phalanx V.

[0027] As a further improvement of the present invention, in step S6, the "epithelial grade score table" is a first data comparison table established with reference to the "RUS-CHN method bone development grade score table" in "Methods for Evaluation of Wrist Bone Maturity in Chinese Children and Adolescents" TY / T 3001-2006.

[0028] The first data comparison table establishes the score values ​​corresponding to the epiphyseal grade of each of the 13 characteristic bones; the first data comparison table is divided into two types according to gender: "First Data Comparison Table (Male)" and "First Data Comparison Table (Female)", which are respectively applicable to the epiphyseal grade scoring of males and females.

[0029] As a further improvement of the present invention, in step S7, the 50th percentile bone maturity standard function is a function fitted to the 50th percentile curve of the "RUS-CHN Bone Maturity Percentile Standard Curve" in "Methods for Evaluation of Wrist Bone Maturity in Chinese Adolescents and Children" TY / T 3001-2006. Its independent variable is the SMS score and the dependent variable is the corresponding bone age test result.

[0030] The 50th percentile bone maturity standard function is also divided into two types based on gender: "50th percentile bone maturity standard function (male)" and "50th percentile bone maturity standard function (female)". These two are respectively applicable to bone age prediction for males and females.

[0031] As a further improvement of the present invention, in step S81, the energy function E used in the elastic energy algorithm is... total as follows:

[0032]

[0033] In the above formula, δ represents the contour line formed by connecting a set of coordinate points end to end with a curve; δ is the sign for taking the derivative; d is the differential sign; s represents the independent variable describing the boundary in the form of a Fourier transform, s∈[0,1]; for The magnitude of the first derivative of represents the elastic energy; for The modulus of the second derivative, the bending energy; α represents the weighting coefficient of the elastic energy; β represents the weighting coefficient of the bending energy; This is a function that calculates the degree of fit between the deformed curve and the original curve, and the function value represents the external energy.

[0034] By iterating to minimize the value of the energy function, the corresponding contour-corrected second metacarpal image is obtained.

[0035] The present invention also includes an online bone age and bone mineral density detection system, which adopts the online bone age and bone mineral density detection method based on the anteroposterior X-ray image of the hand as described above. Based on the input anteroposterior X-ray image of the hand of any user and gender information, the system outputs the detection results of the current user's bone age and bone mineral density.

[0036] The online bone age and bone mineral density detection system includes: a data input module, an image preprocessing module, a hand bone classification module, a feature bone extraction module, an epiphyseal grade recognition module, an SMS score generation module, a bone age output module, a contour correction module, a metacarpal index generation module, and a bone mineral density output module.

[0037] The data input module is used to acquire the gender information of the user to be tested and an orthogonal X-ray image of the hand.

[0038] The image preprocessing module uses a hand recognition model pre-created based on a simplified U-net network to perform feature recognition on the orthogonal X-ray image of the hand, identify the hand bones and soft tissues, and generate a hand image after removing the background.

[0039] The hand bone classification module uses a hand bone recognition model created based on a four-layer full-node UNet++ network to perform bone recognition and classification on the hand images output by the image preprocessing module, thereby obtaining feature images corresponding to 29 types of hand bones.

[0040] The feature bone extraction module first uses the contour-finding algorithm in OpenCV to obtain the minimum bounding rectangle of the outer contours of 13 feature bones. It then generates a template image corresponding to the region inside the bounding rectangle. Finally, it performs masking operations using the template image and the original orthogonal X-ray image of the hand to obtain 13 feature bone images.

[0041] The epiphyseal grade recognition module uses a three-layer, fully node UNet++ network to classify input feature bone images and obtain the corresponding epiphyseal grade classification results. During network training, Gaussian noise, salt-and-pepper noise, and logarithmic compression are used for data augmentation to address the issue of the output structure being easily affected by noise.

[0042] The SMS score generation module selects the corresponding epiphyseal grade score table based on gender, then queries the skeletal grade score table based on the epiphyseal grades of the 13 input feature bone images and assigns scores. Finally, the assigned scores are summed to obtain the current user's SMS score.

[0043] The bone age output module takes the current user's SMS score and inputs it into a pre-fitted 50th percentile bone maturity standard function, then outputs the corresponding bone age value.

[0044] The contour correction module is used to acquire the second metacarpal image from 13 feature bone images, and uses the elastic energy algorithm to correct its contour, and then performs rotation correction on the corrected image.

[0045] The metacarpal index generation module uses a sampling threshold method to process the corrected second metacarpal image, identify the cortical bone and cancellous bone, and then calculates the ratio of the average thickness of cortical bone and cancellous bone, which is the required metacarpal index of the second metacarpal.

[0046] The bone mineral density output module calculates the corresponding bone mineral density value based on the mapping relationship between the metacarpal index and bone mineral density.

[0047] The present invention also includes an online bone age and bone mineral density detection device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it creates the aforementioned online bone age and bone mineral density detection system, and then automatically generates the detection results of the current user's bone age and bone mineral density based on the input gender information and an anteroposterior X-ray image of the hand.

[0048] The technical solution provided by this invention has the following beneficial effects:

[0049] This invention classifies and detects bones in a user's hand anteroposterior image using a created network, extracting images of 13 characteristic bones. These characteristic bone images are then used to detect bone age and bone mineral density. The method provided by this invention is a fully digital solution, automating online processing of medical image recognition, bone classification, epiphyseal grade, epiphyseal grade scoring, bone age prediction, and prediction of metacarpal index extremes and bone mineral density values. Users only need to provide an anteroposterior X-ray image of the hand and gender information to simultaneously obtain bone age and bone mineral density detection results. This significantly improves the detection efficiency of medical institutions and reduces detection costs for users.

[0050] Furthermore, the network model and various comparison tables generated from statistical data used in the detection scheme provided by this invention are all generated from sample data of Chinese children, thus resulting in higher detection accuracy and stronger reliability. Attached Figure Description

[0051] Figure 1 This is a flowchart illustrating the steps of an online bone age and bone mineral density detection method based on an anteroposterior X-ray image of the hand, as provided in Embodiment 1 of the present invention.

[0052] Figure 2 This is a schematic diagram illustrating the principle of an online bone age and bone mineral density detection method based on an orthogonal X-ray image of the hand, as provided in Embodiment 1 of the present invention.

[0053] Figure 3 This is a typical frontal X-ray image of the hand.

[0054] Figure 4 This is the network architecture of the hand recognition model designed in Embodiment 1 of the present invention.

[0055] Figure 5 This is the network architecture of the hand bone recognition model designed in Embodiment 1 of the present invention.

[0056] Figure 6 This is a schematic diagram illustrating the principle of generating segmentation results of various feature bone images based on the output of the hand bone recognition model in Embodiment 1 of the present invention.

[0057] Figure 7 This is the network architecture of the hand recognition model designed in Embodiment 1 of the present invention.

[0058] Figure 8 This is the graph of the 50th percentile bone maturity standard function (male) fitted in Embodiment 1 of the present invention.

[0059] Figure 9This is the function image of the 50th percentile bone maturity standard function (female) fitted in Embodiment 1 of the present invention.

[0060] Figure 10 This is a module architecture diagram of the online bone age and bone mineral density detection system provided in Embodiment 1 of the present invention.

[0061] Figure 11 This is a schematic diagram of the training strategy for the hand bone recognition model in the performance testing experiment.

[0062] Figure 12 This is a loss function image of the hand bone recognition model during the training phase in a performance testing experiment.

[0063] Figures 13-15 The results of skeletal recognition and localization from three orthogonal X-ray images of the hand during the performance test experiment.

[0064] Figure 16 This is a typical feature bone recognition result obtained using an online bone age and bone mineral density detection system during a performance testing experiment.

[0065] Figure 17 This is an image of the second metacarpal bone after correction using the elastic energy algorithm in a performance testing experiment.

[0066] Figure 18 This is the loss function image of the epiphyseal classification model in the performance testing experiment.

[0067] Figure 19 This is a bone age examination report generated by an online bone age and bone mineral density detection system for a female user during a performance testing experiment.

[0068] Figure 20 This is a bone age examination report generated by an online bone age and bone mineral density detection system for a male user during a performance testing experiment. Detailed Implementation

[0069] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0070] Example 1

[0071] This embodiment provides an online method for detecting bone age and bone mineral density based on anteroposterior X-ray images of the hand, such as... Figure 1 and Figure 2 As shown, it includes the following steps:

[0072] S1: Obtain the current user's gender information and the corresponding orthogonal X-ray image of their hand.

[0073] A anteroposterior X-ray examination of the hand is a routine and standardized procedure for examining the bones of the hand; a typical anteroposterior X-ray image of the hand is a medical image taken with the user's palm facing down through a medical X-ray fluoroscopy machine. Figure 3 As shown, the anteroposterior X-ray image of the hand contains 29 types of bones in the human hand: ulna, radius, hamate, capitate, triquetrum, lunate, trapezium, trapezium scaphoid, pisiform, metacarpals I-V, proximal phalanges I-V, distal phalanges I-V, and middle phalanges II-V, as well as the soft tissues surrounding the bones. In addition, the anteroposterior X-ray image of the hand inevitably includes the background image.

[0074] S2: A hand recognition model for identifying and segmenting the palm and forearm is pre-built and trained using a simplified U-net network; and a hand image with background removed is generated based on the orthogonal X-ray image of the hand using the hand recognition model.

[0075] In this embodiment, as Figure 4 As shown, the hand recognition model is a fully convolutional neural network (LCN) with a UNet network as its backbone. The backbone network consists of four downsampling layers and four upsampling layers. Each downsampling layer uses a fully convolutional ReLU activation sub-layer with a kernel of 3, a batch normalization sub-layer, and a max pooling sub-layer. Each upsampling layer uses a fusion layer, a fully convolutional ReLU activation sub-layer, and an upsampling sub-layer. Then, it passes through a 2-channel feature fully convolutional layer with a kernel of 1, and finally through a softmax layer to output a 2-classification feature image. The output of the final layer of the LCN is compared with pre-labeled supervision data t. n Cross-entropy error calculation is performed, followed by backpropagation of the network to learn and correct the parameters of each layer. The hand recognition model network can learn binary image features of the background and hand contour edges.

[0076] During the training phase of the hand recognition model, the input is an 8-bit image dataset in the format (n, w, h, c). Here, n represents the number of samples in the training dataset; w is the sample width, set to 288 in this embodiment; h is the sample height, set to 320 in this embodiment; and c represents the number of sample channels. In this embodiment, only segmented hand images are used as training samples, therefore c is set to 1.

[0077] S3: A hand bone recognition model for recognizing 29 types of bones in the hand is pre-built and trained using a four-layer full-node UNet++ network; and the hand bone recognition model is used to recognize and locate various types of bones in hand images.

[0078] In this embodiment, as Figure 5As shown, the hand bone recognition model is a fully convolutional neural network (LCN) with UNet++ as its backbone. The backbone network consists of four downsampling layers and multiple upsampling layers. Each downsampling layer uses a fully convolutional ReLU activation sub-layer with a kernel of 3, a batch normalization sub-layer, and a max pooling sub-layer. Each upsampling layer uses a fusion layer, a fully convolutional ReLU activation sub-layer, and an upsampling sub-layer. Then, a 10-channel feature fully convolutional layer with a kernel of 1 is used, followed by a softmax function to output a 29-classification feature image. The final output of the LCN is compared with pre-labeled supervision data t. n Cross-entropy error calculation is performed, followed by backpropagation of the network to learn and correct the parameters of each layer. The hand recognition model network can learn 29 classification image features of 29 different bone features.

[0079] During the training phase of the hand bone recognition model, the input is an 8-bit image dataset in the format (n, w, h, c). Here, n represents the number of samples in the training dataset; w is the sample width, set to 288 in this embodiment; h is the sample height, set to 320 in this embodiment; and c represents the number of sample channels. In this embodiment, 29 different types of segmented hand bones and soft tissue sample images are used as the training set to train the hand bone recognition model; therefore, c in the hand bone recognition model is set to 30.

[0080] S4: Based on the recognition and localization results from the previous step, the minimum bounding rectangle region containing the 13 types of feature bones is generated using the contour-finding algorithm in OpenCV, and image segmentation is performed to obtain the corresponding 13 feature bone images Pc. i , i = 1……13. The 13 types of characteristic bones that were segmented include: radius, ulna, metacarpal I, metacarpal III, metacarpal V, proximal phalanx I, proximal phalanx III, proximal phalanx V, middle phalanx III, middle phalanx V, distal phalanx I, distal phalanx III, distal phalanx V.

[0081] In this embodiment, as Figure 6As shown, the detailed steps for generating the feature bone image are as follows: First, obtain the hand image A output in step S2 and the recognition and localization results of the 29 feature bones generated in step S3. Then, select the recognition and localization results corresponding to 13 types of feature bones, and use the contour finding algorithm in OpenCV to generate the minimum bounding rectangle region of the 13 types of feature bones contained therein. Next, generate a template image M corresponding to each feature bone based on the minimum bounding rectangle of the 13 types of feature bones. The template image is a binary image, where the region inside the minimum bounding rectangle is assigned a value of 1, and the region outside the minimum bounding rectangle is assigned a value of 0. Finally, perform a mask operation between the template image and the original hand image A to obtain 13 images that only include local information of the region inside the minimum bounding rectangle in the hand image, i.e., feature bone images Pc. i , i = 1……13.

[0082] S5: A three-layer, fully node UNet++ network is used to build and train an epiphyseal classification model for classifying the epiphyseal grade of feature bones; and the epiphyseal classification model is used to generate Pc for each feature bone image. i Corresponding epiphyseal rank i .

[0083] like Figure 7 As shown, the epiphyseal classification model provided in this embodiment uses a simplified 3-layer UNet++ network; its backbone network performs backpropagation in two paths. One path uses the same fully convolutional neural network as the hand bone recognition model to perform cross-entropy error calculation on the feature layer output and labels for supervised learning. Then, Flatten dimensionality reduction is performed on the output image, and feature information is extracted through Dense1024 and Dense256 fully connected layers, and then fused with the feature information from the other path. The other path extracts features from each final layer X30, X21, X12, and X03, processes them through Dense256 fully connected layers, and then performs feature fusion through a concatenate layer; then it is fused with the feature information from the previous path through a Dense256 fully connected layer. Finally, the Dense15 output is a 15-level epiphyseal classification; and backpropagation is performed with the labels to complete the learning and training.

[0084] The strategy for training the epiphyseal classification model in this embodiment is as follows: First, a large number of medical images of 13 characteristic bones are acquired as sample images, ensuring that the data for each characteristic bone in the sample images contains samples corresponding to 15 bone grades (0-14). Then, professional medical personnel manually identify the bone grades in the sample images and add a bone grade label. Next, to improve the robustness of the network model, Gaussian noise, salt-and-pepper noise, and logarithmic compression are used for data augmentation in the sample images, solving the problem that the output structure is easily affected by noise. Furthermore, the same epiphyseal grade label as the original image is added to the augmented sample images. Finally, the designed epiphyseal classification model is trained using the labeled sample dataset until the preset training objective or iteration termination condition is reached.

[0085] It should be emphasized that the hand recognition model, hand bone recognition model and epiphyseal classification model designed in this embodiment all use the following cross-entropy error E as the optimization function during the training phase.

[0086]

[0087] In the above formula, t nk y represents the label information of the k-th feature element of the n-th input sample image. nk This represents the classification result of the k-th feature element of the n-th sample data image output.

[0088] S6: Combine the current user's gender and the 13 epiphyseal plate levels to query a pre-built "epithelial plate level score table" and obtain the score value corresponding to each bone level. i Then, the corresponding SMS score is calculated using the following formula:

[0089]

[0090] In this embodiment, the "epithelial grade score table" is a first data comparison table established with reference to the "RUS-CHN method bone development grade score table" in "Methods for Evaluation of Wrist Bone Maturity in Chinese Children and Adolescents" TY / T 3001-2006.

[0091] The first data comparison table establishes the score values ​​corresponding to the epiphyseal grade of each of the 13 characteristic bones; the first data comparison table is divided into two types according to gender: the "first data comparison table (male)" shown in Table 1 and the "first data comparison table (female)" shown in Table 2, which are respectively applicable to the epiphyseal grade scoring of males and females.

[0092] Table 1: First Data Comparison Table (Male)

[0093]

[0094] Table 2: First Data Comparison Table (Female)

[0095]

[0096]

[0097] The usage of the first data lookup table (male) and the first data lookup table (female) in this embodiment is as follows: First, determine the lookup table type to be queried based on the user's gender, and then sequentially select the lookup table based on the epiphyseal rank of 13 characteristic bones. i The corresponding score is obtained by querying the first data lookup table. i Finally, the 13 score values ​​Score1 to Score2 are calculated. 13 The scores are accumulated to obtain the corresponding SMS score.

[0098] In practical application, this embodiment encodes 13 characteristic bone categories as 'a', where a = 1 to 13; and encodes 15 epiphyseal grades as 'b', where b = 0 to 14. Then, matrices representing the first data lookup table (male) and the first data lookup table (female) are created, namely Form1 (male) and Form1 (female). The rows of the matrices are encoded according to bone grade, and the columns are encoded according to characteristic bone type. Each element in the matrix is ​​the corresponding score. Next, matrices Form1 (male) and Form1 (female) are converted into vector representations to obtain a vector table. Each vector in the vector table has the format (a, b, Score). a,b ), where Score a,b This represents the score value corresponding to bone grade b for bone type a. Finally, using a and b as search criteria, a table is looked up to obtain the required score value.

[0099] S7: Input the SMS score into a pre-fitted 50th percentile bone maturity standard function f stard The corresponding bone age value is output.

[0100] Bone age value = f stard (SMS)

[0101] In this embodiment, the 50th percentile bone maturity standard function is a function fitted to the 50th percentile curve of the "RUS-CHN Bone Maturity Percentile Standard Curve" in TY / T 3001-2006, "Methods for Evaluating Wrist Bone Maturity in Chinese Adolescents and Children". Its independent variable is the SMS score, and the dependent variable is the corresponding bone age test result. The obtained 50th percentile bone maturity standard function is also divided into two types according to gender: "50th Percentile Bone Maturity Standard Function (Male)" and "50th Percentile Bone Maturity Standard Function (Female)". These two are respectively applicable to bone age prediction for males and females.

[0102] In this embodiment, Matlab was used as a tool for function fitting during the design process. The function graphs of the fitted standard functions for 50th percentile bone maturity in men and women are shown below. Figure 8 and Figure 9 As shown.

[0103] S8: Select the second metacarpal image from the 13 feature bone images in step S4; analyze the second metacarpal image and generate bone density values, as follows:

[0104] S81: The contour of the second metacarpal image is corrected using the elastic energy algorithm. The energy function E used in the elastic energy algorithm of this embodiment is... total as follows:

[0105]

[0106] In the above formula, δ represents the contour line formed by connecting a set of coordinate points end to end with a curve; δ is the sign for taking the derivative; d is the differential sign; s represents the independent variable describing the boundary in the form of a Fourier transform, s∈[0,1]; for The magnitude of the first derivative of represents the elastic energy; for The modulus of the second derivative, the bending energy; α represents the weighting coefficient of the elastic energy; β represents the weighting coefficient of the bending energy; This is a function that calculates the degree of fit between the deformed curve and the original curve, and the function value represents the external energy.

[0107] By iterating to minimize the value of the energy function, the corresponding contour-corrected second metacarpal image is obtained.

[0108] S82: Perform rotation correction on the image after the previous step.

[0109] S83: Distinguish between the cortical bone and cancellous bone regions in the second metacarpal image based on a preset grayscale threshold.

[0110] S84: Statistical analysis of the average thickness D of cortical and cancellous bone皮 and D 松 .

[0111] S85: Calculate the metacarpal index (MCI) of the second metacarpal bone of the current user: MCI = D 皮 / (D 松 +D 皮 );

[0112] S86: Based on the mapping relationship between the metacarpal index and bone mineral density, the corresponding bone mineral density value is calculated according to the metacarpal index. In this embodiment, the functional relationship between the bone mineral density value and the metacarpal index is as follows:

[0113] Bone mineral density value = (MCI + 4) / 46.5.

[0114] Therefore, the online bone age and bone mineral density detection method based on anteroposterior X-ray images of the hand provided in this embodiment only requires acquiring an anteroposterior X-ray image of any user's hand and their gender to simultaneously perform dual detection of bone age and bone mineral density, resulting in higher detection efficiency. Furthermore, this embodiment avoids the need for users to take two types of X-ray images (hand and second metacarpal bones), significantly reducing detection costs.

[0115] Example 2

[0116] This embodiment provides an online bone age and bone mineral density detection system. It adopts the online bone age and bone mineral density detection method based on the anteroposterior X-ray image of the hand as described in Embodiment 1. Based on the input anteroposterior X-ray image of any user's hand and gender information, it outputs the detection results of the current user's bone age and bone mineral density.

[0117] like Figure 10 As shown, the online bone age and bone mineral density detection system includes: a data input module, an image preprocessing module, a hand bone classification module, a feature bone extraction module, an epiphyseal grade recognition module, an SMS score generation module, a bone age output module, a contour correction module, a metacarpal index generation module, and a bone mineral density output module.

[0118] The data input module is used to acquire the gender information of the user to be tested and an orthogonal X-ray image of the hand.

[0119] The image preprocessing module uses a hand recognition model pre-created based on a simplified U-net network to perform feature recognition on the orthogonal X-ray image of the hand, identify the hand bones and soft tissues, and generate a hand image after removing the background.

[0120] The hand bone classification module uses a hand bone recognition model created based on a four-layer full-node UNet++ network to perform bone recognition and classification on the hand images output by the image preprocessing module, thereby obtaining feature images corresponding to 29 types of hand bones.

[0121] The feature bone extraction module first uses the contour-finding algorithm in OpenCV to obtain the minimum bounding rectangle of the outer contours of 13 feature bones. It then generates a template image corresponding to the region inside the bounding rectangle. Finally, it performs masking operations using the template image and the original orthogonal X-ray image of the hand to obtain 13 feature bone images.

[0122] The epiphyseal grade recognition module is used to classify the input feature bone image using an epiphyseal classification model created based on a three-layer full-node UNet++ network, and obtain the corresponding epiphyseal grade classification result.

[0123] The SMS score generation module selects the corresponding epiphyseal grade score table based on gender, then queries the skeletal grade score table based on the epiphyseal grades of the 13 input feature bone images and assigns scores. Finally, the assigned scores are summed to obtain the current user's SMS score.

[0124] The bone age output module takes the current user's SMS score and inputs it into a pre-fitted 50th percentile bone maturity standard function, then outputs the corresponding bone age value.

[0125] The contour correction module is used to acquire the second metacarpal image from 13 feature bone images, and uses the elastic energy algorithm to correct its contour, and then performs rotation correction on the corrected image.

[0126] The metacarpal index generation module uses a sampling threshold method to process the corrected second metacarpal image, identify the cortical bone and cancellous bone, and then calculates the ratio of the average thickness of cortical bone and cancellous bone, which is the required metacarpal index of the second metacarpal.

[0127] The bone mineral density output module calculates the corresponding bone mineral density value based on the mapping relationship between the metacarpal index and bone mineral density.

[0128] Example 3

[0129] This embodiment provides an online bone age and bone mineral density detection device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it creates an online bone age and bone mineral density detection system as described in Embodiment 2, and then automatically generates the detection results of the current user's bone age and bone mineral density based on the input gender information and an anteroposterior X-ray image of the hand.

[0130] The online bone age and bone mineral density detection device in this embodiment is essentially a computer device. This computer device can be connected to a medical X-ray fluoroscopy system as a backend device for analyzing and processing acquired anteroposterior X-ray images of the hand. This computer device can be a smartphone, tablet, laptop, desktop computer, rack server, blade server, tower server, or cabinet server (including standalone servers or server clusters composed of multiple servers), etc., capable of executing programs. The computer device in this embodiment includes, but is not limited to, a memory and a processor that can communicate with each other via a system bus.

[0131] In this embodiment, the memory (i.e., the readable storage medium) includes flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory can be an internal storage unit of a computer device, such as the hard disk or RAM of the computer device. In other embodiments, the memory can also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the computer device. Of course, the memory can also include both internal storage units and external storage devices of the computer device. In this embodiment, the memory is typically used to store the operating system and various application software installed on the computer device. In addition, the memory can also be used to temporarily store various types of data that have been output or will be output.

[0132] In some embodiments, the processor may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor is typically used to control the overall operation of a computer device.

[0133] Performance testing

[0134] Based on the schemes of Examples 1 and 2, the inventors created a corresponding online detection system for bone age and bone mineral density, and trained and tested it. Among these, the hand bone recognition model is a key aspect of this invention. To achieve the goal of multi-classification, this invention uses a UNet++ network as the backbone network and employs methods such as... Figure 11 The training method is used to train the created network model, and the training results are as follows: Figure 12As shown in the figure, the hand bone recognition model designed in this invention achieves convergence in average performance metrics after approximately 100 training rounds, and the recognition accuracy meets the requirements. Furthermore, this experiment selected three samples to verify the performance of the hand bone recognition model, and the verification results are shown below. Figures 13-15 As shown in the figure, the solution of the present invention can accurately identify 29 types of hand bones in the hand X-ray image after removing background information.

[0135] The online bone age and bone mineral density detection system established in this experiment identifies the location distribution of 13 types of characteristic bones in hand images, as follows: Figure 16 As shown. The image of the second metacarpal bone after correction using the elastic energy algorithm is as follows. Figure 17 As shown. The loss curve of the epiphyseal grading classification model during the training phase is as follows. Figure 18 As shown. Figure 19 and Figure 20 This refers to the inspection reports generated by the system designed using this invention for two users, one female and one male.

[0136] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for online detection of bone age and bone mineral density based on anteroposterior X-ray images of the hand, characterized in that, It includes the following steps: S1: Obtain the current user's gender information and the corresponding orthogonal X-ray image of their hand; S2: A hand recognition model for recognizing and segmenting the palm and forearm is pre-built and trained using a simplified U-net network; and a hand image with background removed is generated based on the hand anteroposterior X-ray image using the hand recognition model; S3: A hand bone recognition model for recognizing 29 types of bones in the hand is pre-built and trained using a four-layer full-node UNet++ network; and the hand bone recognition model is used to recognize and locate various types of bones in the hand image; S4: Based on the recognition and localization results from the previous step, the minimum bounding rectangle region containing the 13 types of feature bones is generated using the contour-finding algorithm in OpenCV, and image segmentation is performed to obtain the corresponding 13 feature bone images Pc. i , i = 1……13; S5: A three-layer, fully node UNet++ network is used to build and train an epiphyseal classification model for classifying the epiphyseal grade of the feature bones; and the epiphyseal classification model is used to generate Pc for each feature bone image. i Corresponding epiphyseal rank i ; S6: Combine the current user's gender and the 13 epiphyseal plate levels to query a pre-built "epithelial plate level score table" and obtain the score value corresponding to each bone level. i Then, the corresponding SMS score is calculated using the following formula: S7: Input the SMS score into a pre-fitted 50th percentile bone maturity standard function and output the corresponding bone age value; S8: Select the second metacarpal image from the 13 feature bone images in step S4; analyze the second metacarpal image and generate bone density values, as follows: S81: Use the elastic energy algorithm to correct the contour of the second metacarpal image; S82: Perform rotation correction on the image after the previous step; S83: Distinguish the cortical bone and cancellous bone regions in the second metacarpal image according to a preset grayscale threshold; S84: Statistically calculate the average thickness D of the cortical bone and cancellous bone. 皮 and D 松 ; S85: Calculate the metacarpal index (MCI) of the second metacarpal bone of the current user: MCI = D 皮 / (D 松 +D 皮 ); S86: Based on the following mapping relationship between metacarpal index and bone mineral density, calculate the corresponding bone mineral density value: Bone mineral density value = (MCI + 4) / 46.

5.

2. The online bone age and bone mineral density detection method based on anteroposterior X-ray images of the hand as described in claim 1, characterized in that: The hand recognition model is a fully convolutional neural network (LCN) with a UNet network as its backbone. The backbone network consists of four downsampling layers and four upsampling layers. Each downsampling layer uses a 3-kernel fully convolutional ReLU activation sub-layer, a batch normalization sub-layer, and a max-pooling sub-layer. Each upsampling layer uses a fusion layer, a fully convolutional ReLU activation layer, and an upsampling sub-layer. Then, a 2-channel feature fully convolutional layer with a 1-kernel core is used, followed by a softmax function to output a 2-classification feature image. The final output of the LCN is compared with pre-labeled supervision data t. n Cross-entropy error is calculated, and then backpropagation of the network is performed to learn and correct the parameters of each layer.

3. The online bone age and bone mineral density detection method based on anteroposterior X-ray images of the hand as described in claim 1, characterized in that: The hand bone recognition model is a fully convolutional neural network (LCN) with a UNet++ backbone. The backbone network consists of four downsampling layers and multiple upsampling layers. Each downsampling layer uses a 3-kernel LCN ReLU activation sub-layer, a batch normalization sub-layer, and a max-pooling sub-layer. Each upsampling layer uses a fusion layer, a LCN ReLU activation sub-layer, and an upsampling layer. Then, a 10-channel LCN feature layer with a 1-kernel softmax layer outputs a 29-class feature image. The final output of the LCN is compared with pre-labeled supervision data t. n Cross-entropy error is calculated, and then backpropagation of the network is performed to learn and correct the parameters of each layer.

4. The online bone age and bone mineral density detection method based on anteroposterior X-ray images of the hand as described in claim 1, characterized in that: In step S5, the epiphyseal classification model uses a simplified 3-layer UNet++ network. Its backbone network performs backpropagation in two paths. One path uses the same fully convolutional neural network as the hand bone recognition model to perform cross-entropy error calculation on the feature layer output and labels for learning supervision. Then, the output image is subjected to Flatten dimensionality reduction, and feature information is extracted through Dense1024 fully connected layers and Dense256 fully connected layers, and then fused with the feature information from the other path. The other path extracts features from each final layer X30, X21, X12, and X03, processes them through Dense256 fully connected layers, and then performs feature fusion through a concatenate layer. Then, it is fused with the feature information from the previous path through a Dense256 fully connected layer. Finally, the Dense15 output is a 15-level epiphyseal classification, and cross-entropy calculation is performed with the labels for backpropagation to complete the learning training.

5. The online bone age and bone mineral density detection method based on anteroposterior X-ray images of the hand as described in claim 3, characterized in that: The 29 types of bones identified by the hand bone recognition model in step S3 include: ulna, radius, hamate, capitate, triquetrum, lunate, trapezium, trapezium scaphoid, pisiform, metacarpals I-V, proximal phalanges I-V, distal phalanges I-V, and middle phalanges II-V; In step S4, the 13 types of characteristic bones that are segmented include: radius, ulna, metacarpal I, metacarpal III, metacarpal V, proximal phalanx I, proximal phalanx III, proximal phalanx V, middle phalanx III, middle phalanx V, distal phalanx I, distal phalanx III, and distal phalanx V.

6. The online bone age and bone mineral density detection method based on anteroposterior X-ray images of the hand as described in claim 1, characterized in that: In step S6, the "epithelial grade scoring table" is a first data comparison table established with reference to the "RUS-CHN method bone development grade scoring table" in "Methods for Evaluation of Wrist Bone Maturity in Chinese Children and Adolescents" TY / T 3001-2006. The first data comparison table establishes the score values ​​corresponding to the epiphyseal grade of 13 characteristic bones. The first data comparison table is divided into two types according to gender.

7. The online bone age and bone mineral density detection method based on anteroposterior X-ray images of the hand as described in claim 1, characterized in that: In step S7, the 50th percentile bone maturity standard function is a function fitted to the 50th percentile curve of the "RUS-CHN Bone Maturity Percentile Standard Curve" in "Methods for Evaluation of Wrist Bone Maturity in Chinese Adolescents and Children" TY / T 3001-2006. Its independent variable is the SMS score and the dependent variable is the corresponding bone age test result. The 50th percentile bone maturity standard function is also divided into two types according to gender.

8. The online bone age and bone mineral density detection method based on anteroposterior X-ray images of the hand as described in claim 1, characterized in that: In step S81, the energy function E used in the elastic energy algorithm is... total as follows: In the above formula, δ represents the contour line formed by connecting a set of coordinate points end to end with a curve; δ is the sign for taking the derivative; d is the differential sign; s represents the independent variable describing the boundary in the form of a Fourier transform, s∈[0,1]; for The magnitude of the first derivative represents the elastic energy; for The modulus of the second derivative, the bending energy; α represents the weighting coefficient of the elastic energy; β represents the weighting coefficient of the bending energy; This is a function that calculates the degree of fit between the deformed curve and the original curve, where the function value represents the external energy. By iterating to minimize the value of the energy function, the corresponding contour-corrected second metacarpal image is obtained.

9. An online system for detecting bone age and bone mineral density, characterized in that, It employs the online bone age and bone mineral density detection method based on anteroposterior X-ray images of the hand as described in claim 1. Based on an input anteroposterior X-ray image of any user's hand and gender information, it outputs the detection results of the current user's bone age and bone mineral density. The online detection system includes: The data input module is used to acquire the gender information of the user to be tested and an orthogonal X-ray image of the hand; The image preprocessing module uses a hand recognition model pre-created based on a simplified U-net network to perform feature recognition on the orthogonal X-ray image of the hand, identify the hand bones and soft tissues, and generate a hand image after removing the background. The hand bone classification module uses a hand bone recognition model created based on a four-layer full-node UNet++ network to perform bone recognition and classification on the hand images output by the image preprocessing module, thereby obtaining feature images corresponding to 29 types of hand bones; The feature bone extraction module first obtains the minimum bounding rectangle of the outer contour of 13 feature bones using the contour finding algorithm in OpenCV, and then generates a template image corresponding to the inner region of the bounding rectangle; finally, it uses the template image and the original orthogonal X-ray image of the hand to perform mask operation to obtain 13 feature bone images. The epiphyseal grade recognition module uses an epiphyseal classification model created based on a three-layer full-node UNet++ network to classify the input feature bone image and obtain the corresponding epiphyseal grade classification result. The SMS score generation module is used to select the corresponding epiphyseal grade score table according to gender, then query the bone grade score table according to the epiphyseal grade of the 13 input feature bone images and assign scores, and finally sum the assigned scores to obtain the current user's SMS score value. The bone age output module is used to input the current user's SMS score into a pre-fitted 50th percentile bone maturity standard function and output the corresponding bone age value. The contour correction module is used to acquire the second metacarpal image from 13 feature bone images, correct its contour using the elastic energy algorithm, and then perform rotation correction on the corrected image. The metacarpal index generation module processes the corrected second metacarpal image using a sampling threshold method to identify the cortical and cancellous bone, and then calculates the ratio of the average thickness of the cortical and cancellous bone, which is the desired metacarpal index of the second metacarpal. The bone mineral density output module is used to calculate the corresponding bone mineral density value based on the mapping relationship between the metacarpal index and bone mineral density.

10. An online bone age and bone mineral density detection device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it creates an online bone age and bone density detection system as described in claim 9, and then automatically generates the detection results of the current user's bone age and bone density based on the input gender information of the user and an orthogonal X-ray image of the hand.

Images