A method for calculating bone health indicators based on Unet region of interest extraction

Abstract

This invention discloses a method for calculating bone health indicators based on Unet extraction of regions of interest (ROIs), relating to the field of medical technology. It provides an end-to-end fully automated solution for locating ROIs and extracting metacarpals and cortex using Unet, followed by calculation of the bone health index. The method involves: uniformizing and compressing the image size; extracting the hand region from the regularized image using the Unet contour model, locating the ROI center, extracting patches from the original image using this center, and compressing the size to the network input size; extracting the metacarpal and cortical regions using a trained Unet segmentation network; calculating the midline based on the metacarpal regions and contours using linear regression, locating the metacarpal center, distal endpoint, and proximal endpoint, and drawing the ROI based on the midline and the specified ROI center; sampling the metacarpal and cortical contours within the ROI, and calculating the average width of the metacarpals and the average thickness of the cortex using Euclidean distance; calculating the bone health index (BHI) for each metacarpal, and averaging the results to obtain the individual's BHI index.

Description

Technical Field

[0001] This invention relates to the field of medical technology, and in particular to a method for calculating bone health indicators based on Unet extraction of regions of interest. Background Technology

[0002] Bone age is currently the most widely used method for assessing biological age, with wrist bone age being the most extensively used. Data shows that fragile bones in childhood are a major cause of osteoporosis in the elderly. In recent years, in the field of pediatrics, there has been increasing attention focused on methods for assessing bone mass.

[0003] In this research field, the common practice is to construct evaluation metrics by manually or semi-manually measuring the average length and width of a specified metacarpal bone, as well as its average cortical thickness. Measuring these metrics requires obtaining the contours of the metacarpal bones and cortex beforehand, and then using deep learning methods to extract the contours of these regions. However, this approach proves very impractical under noisy inputs with varying distributions. For example, when the distribution of the input image differs significantly from that of the training image—a common phenomenon occurring between different machines and shooting environments—this method struggles to provide a stable and accurate prediction. Furthermore, since the regions of interest (metacarpals and cortex) typically occupy only a small portion of the input image, this increases the difficulty of segmentation and affects segmentation accuracy. Therefore, in the Bone Density Index (BHI) task, extracting the metacarpal cortical region requires an accurate and stable solution.

[0004] Therefore, those skilled in the art are dedicated to developing a method for calculating bone health indicators based on Unet-based region of interest (ROI) extraction. This provides an end-to-end, fully automated solution for locating ROIs and extracting metacarpals and cortex using Unet, followed by BHI calculation. First, the image is normalized and its size compressed. The hand region is extracted from the normalized image using the Unet contour model, and the ROI center is located. This center is then used to extract patches from the normalized original image, and their size is compressed to the network input size. The trained Unet segmentation network is then used to extract the metacarpal and cortical regions. Based on the metacarpal regions and contours, the midline is calculated using linear regression, locating the metacarpal center, distal endpoint, and proximal endpoint. The ROI is then drawn based on the midline and the specified ROI center. The metacarpal and cortical contours within the ROI are sampled separately, and the average metacarpal width and average cortical thickness are calculated using Euclidean distance. The BHI for each metacarpal is calculated separately, and the average is taken as the individual's BHI index.

[0005] CLAHE Histogram Equalization: Histogram equalization is a method to enhance image contrast. Its main idea is to transform the histogram distribution of an image into an approximately uniform distribution, thereby enhancing the image contrast.

[0006] Adaptive histogram equalization (AHE) is a computer image processing technique used to improve image contrast. Unlike regular histogram equalization, the adaptive method calculates multiple histograms, each corresponding to a different part of the image, and then uses them to redistribute the image's brightness values. Therefore, it is well-suited for improving local contrast and enhancing edge sharpness in each region of an image.

[0007] Ordinary AHE often amplifies contrast in near-constant regions of an image because histograms are highly concentrated in these areas. As a result, AHE can amplify noise in near-constant regions. Contrast-limited AHE (CLAHE) is a variant of adaptive histogram equalization where contrast amplification is limited, thus reducing this noise amplification problem.

[0008] In CLAHE, the contrast magnification near a given pixel value is given by the slope of the transform function. This is proportional to the slope of the neighborhood cumulative distribution function (CDF), and therefore proportional to the value of the histogram at that pixel value. CLAHE limits the magnification by cropping the histogram to a predetermined value before calculating the CDF. This limits the slope of the CDF, and thus the slope of the transform function. The value at which the histogram is cropped, the so-called cropping limit, depends on the normalization of the histogram, and therefore on the size of the neighborhood. A common value limits the resulting magnification to between 3 and 4. Advantageously, instead of discarding the portion of the histogram that exceeds the cropping limit, it is distributed evenly across all histogram patches.

[0009] The implementation of CLAHE mainly includes 6 steps.

[0010] Step 1: Blocking. Divide the input image into non-overlapping sub-blocks of equal size, each containing M pixels.

[0011] Step 2: Calculate the sub-block histogram.

[0012] Step 3: Calculate the shear threshold.

[0013] Step 4: Pixel reallocation. For each sub-block, reallocate the extra pixels from Step 3.

[0014] Step 5: Histogram equalization.

[0015] Step 6: Pixel grayscale value reconstruction. (Bilinear interpolation is used in the source code.)

[0016] Grabcut segmentation algorithm: Grabcut is an image segmentation algorithm based on graph cut. It requires the user to input a bounding box as the location of the segmentation target, so as to separate / segment the target from the background.

[0017] The implementation steps of the GrabCut algorithm are as follows:

[0018] ① Define (one or more) rectangles in the image that contain the object. The area outside the rectangles is automatically considered the background.

[0019] ② For a user-defined rectangular area, the foreground and background areas within it can be distinguished using data from the background.

[0020] ③ Use a Gaussian mixture model (GMM) to model the background and foreground, and mark undefined pixels as possible foreground or background.

[0021] ④ Each pixel in the image is considered to be connected to surrounding pixels through virtual edges, and each edge has a probability of belonging to the foreground or background, based on its color similarity with surrounding pixels.

[0022] ⑤ Each pixel (i.e., a node in the algorithm) is connected to a foreground or background node.

[0023] ⑥ After the nodes are connected (possibly to the background or foreground), if the edges between the nodes belong to different ends (i.e., one node belongs to the foreground and the other to the background), the edges between them will be cut, which will segment the different parts of the image.

[0024] UNet segmentation model: UNet is a variant of fully convolutional neural networks (FCNs). It is arguably the most commonly used and simplest segmentation model. It is simple, efficient, easy to understand, easy to build, and can be trained on small datasets. UNet was proposed in 2015 in the paper "U-Net: Convolutional Networks for Biomedical Image Segmentation". UNet was initially designed to solve the problem of medical image segmentation, specifically cell-level segmentation. Subsequently, UNet has been widely applied in various directions of semantic segmentation due to its outstanding segmentation performance.

[0025] UNet network structure as follows Figure 1As shown, its network structure is symmetrical, resembling the English letter U, hence the name UNet. Overall, UNet is an encoder-decoder structure (similar to FCN), with the first half being feature extraction and the second half being upsampling.

[0026] Encoder It consists of convolution and downsampling operations, using a uniform 3x3 kernel with 0 padding and 1 stride. Without padding, the height (H) and width (W) of the feature map decrease after each convolution. Therefore, the dimensionality of the feature map needs to be considered when using skip connections.

[0027] Decoder To restore the original resolution of the feature maps, besides convolution, the key steps in this process are upsampling and skip connections. Upsampling is commonly implemented using transposed convolution and interpolation. Among interpolation methods, bilinear interpolation has good overall performance and is quite common. Skip connections are essentially indispensable for the network to achieve good results. In UNet, skip connections fuse the positional information from the lower layers with the semantic information from the deeper layers through concatenation. It is important to note that in FCN, feature maps are fused additively.

[0028] The main advantages of UNet are:

[0029] ① Deep and shallow features have their own significance: the deeper the network, the larger the receptive field, and the network focuses on global features (more abstract and essential); shallow networks focus more on local features such as texture.

[0030] ② Edge feature recovery is achieved through feature concatenation. While upsampling (transposed convolution) can yield larger feature maps, the edges of these maps lack information. This is because each downsampling extraction inevitably results in the loss of some edge features, which upsampling cannot recover. Summary of the Invention

[0031] In view of the above-mentioned deficiencies of the prior art, the technical problem to be solved by the present invention is that the measurement of bone health indicators depends on manual and manual methods; the proportion of the region of interest in the image affects the results of deep learning, and the extraction of the metacarpal cortex region requires an accurate and stable solution.

[0032] To achieve the above objectives, this invention provides a method for calculating bone health indicators based on Unet extraction of regions of interest, comprising the following steps:

[0033] Step 1: Perform image homogenization and size compression processing;

[0034] Step 2: Extract the hand region from the image using the Unet contour model trained on Grabcut, then locate the center of the ROI (Region of Interest), and use the ROI center to extract the patch from the image.

[0035] Step 3: Extract the metacarpal and cortical regions using the Unet segmentation model;

[0036] Step 4: Based on the metacarpal region and contour, locate the center, distal end, and proximal end of the metacarpal, calculate the midline using linear regression, and draw the ROI based on the midline and the specified ROI center.

[0037] Step 5: Sample the metacarpal and cortical contours within the ROI, and calculate the average width of the metacarpals and the average thickness of the metacarpal cortex using Euclidean distance.

[0038] Step 6: Calculate the Bone Health Index (BHI) for each metacarpal bone and take the average as the BHI index.

[0039] Furthermore, step 1, the image processing includes pixel value uniformization processing, pixel value inversion operation, and CLAHE processing.

[0040] Furthermore, in step 1, the image format includes JPG, PNG, and DCM formats.

[0041] Furthermore, in step 2, the training set of the Unet contour model is generated by the Grabcut algorithm.

[0042] Furthermore, in step 2, the foreground is separated using the Grabcut algorithm and used as the label for the training data to train the Unet contour model.

[0043] Furthermore, in step 2, the Unet contour model locates the palm position, extracts the patch, and focuses the region of interest.

[0044] Furthermore, in step 3, the training set of the Unet segmentation model consists of manually labeled data of the metacarpals and cortex.

[0045] Furthermore, in step 3, the metacarpal segmentation model outputs predictions for four channels, namely the background and the three metacarpals.

[0046] Furthermore, in step 3, the cortical segmentation model outputs predictions for 7 channels, namely the background of three metacarpal bones and two narrow cortical regions corresponding to each metacarpal bone.

[0047] Further, in step 6, the bone health index BHI = πT(1-T / W) / L, where L represents the length of the metacarpal, W represents the average thickness of the metacarpal, T represents the average thickness of the metacarpal cortex, and π is pi.

[0048] In a preferred embodiment of the present invention, a fully automated end-to-end solution is provided for locating regions of interest (ROIs) and extracting metacarpals and cortex based on Unet, followed by calculation of the bone health index (BHI). First, the image is normalized and its size compressed. The hand region is extracted from the normalized image using the Unet contour model, and the ROI center is located. This center is then used to extract patches from the normalized original image, and their size is compressed to the network input size. The trained Unet segmentation network is used to extract the metacarpal and cortical regions. Based on the metacarpal regions and contours, the midline is calculated using linear regression, and the center, distal end, and proximal end of the metacarpals are located. The ROI is then drawn based on the midline and the specified ROI center. The metacarpal and cortical contours within the ROI are sampled separately, and the average width of the metacarpals and the average thickness of the cortex are calculated using Euclidean distance. The BHI for each metacarpal is calculated separately, and the average is taken as the individual's BHI index.

[0049] To address the issue that indicator calculation relies on manual labor, this paper utilizes deep learning and computer programming, employs Unet to extract regions of interest, designs algorithms to calculate indicators, and achieves fully automated, end-to-end, and rapid BHI calculation.

[0050] To address the impact of the proportion of the region of interest in the image on the results of deep learning, deep learning is used for preprocessing. Patches are extracted from the input image and used as input for the subsequent segmentation network. The metacarpal region is pre-located in the input image to make the segmentation network more stable.

[0051] Compared with the prior art, the present invention has the following obvious substantive features and significant advantages:

[0052] 1. Implement a fully automated end-to-end bone health indicator calculation process;

[0053] 2. Using Unet to preprocess the image increases the proportion of the region of interest in the image, thereby improving the stability and accuracy of segmentation;

[0054] The following will further explain the concept, specific structure, and technical effects of the present invention in conjunction with the accompanying drawings, so as to fully understand the purpose, features, and effects of the present invention. Attached Figure Description

[0055] Figure 1 This is a UNet network structure diagram of a preferred embodiment of the present invention;

[0056] Figure 2 This is a schematic diagram of the workflow of a preferred embodiment of the present invention;

[0057] Figure 3 This is a schematic diagram illustrating the application of a preferred embodiment of the present invention;

[0058] Figure 4 This is a schematic diagram of the input image of a preferred embodiment of the present invention;

[0059] Figure 5 This is a schematic diagram of patch positioning according to a preferred embodiment of the present invention;

[0060] Figure 6 This is a schematic diagram of metacarpal segmentation according to a preferred embodiment of the present invention;

[0061] Figure 7 This is a schematic diagram of cortical segmentation according to a preferred embodiment of the present invention;

[0062] Figure 8 This is a schematic diagram illustrating the determination of the central axis, the region of interest, and the calculation width of the metacarpal sampling points according to a preferred embodiment of the present invention.

[0063] Figure 9 This is a schematic diagram illustrating the calculation of cortical sampling point thickness according to a preferred embodiment of the present invention. Detailed Implementation

[0064] The following description, with reference to the accompanying drawings, illustrates several preferred embodiments of the present invention to make its technical content clearer and easier to understand. The present invention can be embodied in many different forms, and the scope of protection of the present invention is not limited to the embodiments mentioned herein.

[0065] In the accompanying drawings, components with the same structure are indicated by the same numerical designation, and components with similar structures or functions are indicated by similar numerical designations. The dimensions and thicknesses of each component shown in the drawings are arbitrary, and the present invention does not limit the dimensions and thicknesses of each component. To make the illustrations clearer, the thickness of some components has been appropriately exaggerated in the drawings.

[0066] This invention discloses a method for calculating bone health indicators based on Unet extraction of regions of interest. It provides a method for segmenting metacarpals and the cortex using the Unet model; an approach to improve the proportion of regions of interest by training a Unet hand contour model using Grabcut; and the BHI algorithm and workflow.

[0067] Since the input images may have multiple formats and resolutions, regularization is required first. To increase the proportion of the region of interest (metacarpals) in the image, a Unet contour model pre-trained with Grabcut is used to extract the hand contour, thereby locating the center of the palm and constructing the region of interest. Because calculating the BHI index requires measuring several dimensional parameters, including metacarpal length, width, and cortical thickness, the metacarpals and cortex need to be separated before calculation. Based on Unet's significant advantages in small object segmentation and medical imaging, Unet is chosen as the base network for segmenting the metacarpals and cortex. After obtaining the segmentation results of the metacarpals and cortex, the algorithm of this invention can be used to calculate the required dimensional parameters, thus obtaining the Bone Health Index (BHI). A schematic diagram of the workflow is shown below. Figure 2 As shown.

[0068] like Figure 4 As shown, input images of any size (JPG / PNG or DCM format) first undergo pixel value normalization. For DCM format images, pixel values ​​are also inverted. Next, CLAHE processing is applied to improve image contrast. Afterward, a pre-trained Unet contour model is used to predict the approximate contour of the hand in the image. The centroid of the contour is calculated, taking into account the vertical deviation (+400) caused by the upper arm, to locate the center of the hand. The contour is then uniformly extended outward from the center to extract pre-processed patches, set to a size of 1024*1024, as shown. Figure 5 As shown. Regularization enables the algorithm to adapt to images of various formats and resolutions.

[0069] The training set for the pre-trained Unet contour model is generated using the Grabcut algorithm. Specifically, 100 representative data points are selected, and the foreground (palm) is separated using the Grabcut algorithm. These foreground elements are then used as labels for the training data to train the Unet contour model. The trained model can assist in locating the palm, thereby extracting patches and focusing on the region of interest (metacarpal region), increasing the accuracy and stability of the Unet segmentation model. Due to the algorithmic principle of the Grabcut model, it sometimes fails to separate the foreground and background when processing large numbers of images, leading to algorithm failure. Using the Unet contour model, however, improves the algorithm's stability, thanks to the ability of deep learning to learn the deep, essential features underlying the training set sample distribution.

[0070] The training set for the Unet segmentation model consisted of 94 manually labeled metacarpal and cortical data points. The metacarpal and cortical segmentation models were trained for 100 and 300 epochs respectively. Figure 6As shown, the metacarpal segmentation model outputs predictions for four channels: the background and the three metacarpals; as... Figure 7 As shown, the cortical segmentation model outputs predictions in 7 channels, namely the background and two narrow cortical regions corresponding to each metacarpal bone.

[0071] The BHI calculation process applies the same operation to all three metacarpals; the following description focuses on only one metacarpal. The bone health index model based on right-hand CT images is BHI = πT(1-T / W) / L. Here, L represents the length of the metacarpal, W represents the average thickness of the metacarpal, T represents the average thickness of the metacarpal cortex, and π is pi, a dimensionless exponent. To calculate the BHI of the metacarpal, the metacarpal dimension LWT needs to be measured. This depends on the establishment of the metacarpal coordinate system and the selection of sampling points, as outlined in the following steps. The metacarpal coordinate system consists of the central axis l and the region of interest I. The central axis is used to calculate the metacarpal length L, and sampling points are selected within the region of interest to calculate the average thickness T and width W.

[0072] ① Determine the midline l and calculate the metacarpal length L.

[0073] First, the metacarpal region is predicted using the Unet segmentation model. The contour of the metacarpal is extracted using the Canny function and sampled and denoted as Set. The midline l of the metacarpal is fitted by minimizing the sum of the distances from all points in Set to the straight line using the Fitline function. At the proximal and distal ends, the points in Set closest to the midline are denoted as the proximal endpoint P and the distal endpoint DP, respectively. The distance between points P and D is denoted as L and is considered as the length of the metacarpal.

[0074] ② Determine the region of interest, I.

[0075] Extend a distance αL along the central axis l from the near endpoint P (α takes values ​​of 0.5 [left], 0.44 [middle], and 0.44 [right]), and locate it as the center U of the region of interest. Establish a region of interest I with a width of 0.3L centered on U.

[0076] ③ Calculate the width W of the metacarpal bone.

[0077] like Figure 8 As shown, three points are sampled at equal intervals upwards and downwards from the center U of the region of interest to the boundary of the region of interest I, and named u in sequence. -3 u -2 u -1 Given u0, u1, u2, u3, according to the previous description, the distance between any two adjacent points is 0.05L.

[0078] At each sampling point u i Calculate the width w of the metacarpal bone at this location sequentially. iThe calculation method is as follows: calculate the perpendicular line l′ of the central axis at the sampling point, and search for the two points on the left and right sides of the metacarpal contour point set Set that are closest to l′. The distance between the two points is denoted as w. i The average width measured at the seven sampling points is recorded as the average width W of the metacarpal bone within the region of interest I.

[0079] ④ Calculate the cortical thickness T.

[0080] like Figure 9 As shown, the cortical region is predicted by the Unet segmentation model, and the contour of the cortex is extracted using the Canny function and sampled and denoted as Set2.

[0081] At each sampling point u i Calculate the cortical thickness t here in sequence. i The calculation method is as follows: calculate the perpendicular line l′ of the central axis at the sampling point, and search for the two points on the left and right sides of the cortical contour point set Set2 that are closest to l′. The distance between the two points is denoted as t. i The average thickness of the metacarpal bone measured at the seven sampling points is recorded as the average thickness T of the metacarpal bone within the region of interest I.

[0082] The calculation process for metacarpal width and cortical thickness is similar, but since a metacarpal has a pair of left and right cortices, the thickness of these two cortices is calculated separately, while sharing a set of sampling points u. i .

[0083] ⑤ Calculate the Biotic Health Index (BHI) for metacarpals.

[0084] like Figure 3As shown, this technology provides an end-to-end fully automated solution for locating regions of interest (ROIs) and extracting metacarpals and cortex based on Unet, followed by calculation of the bone health index (BHI). It can handle images of different formats and resolutions generated by existing imaging systems. The pediatric bone health index system first performs image normalization and size compression. The Unet contour model is used to extract the hand region from the normalized image, thereby locating the ROI center. This center is then used to extract patches from the normalized original image, and the size is compressed to the network input size. The algorithm described in this paper can segment the metacarpals and cortex, and calculate their physical dimensions to obtain the BHI. The algorithm first uses a trained Unet segmentation network to extract the metacarpal and cortical regions. Based on the metacarpal regions and contours, a linear regression method is used to calculate the midline, locate the metacarpal center, distal end, and proximal end, and draw the ROI based on the midline and the specified ROI center. The metacarpal and cortical contours within the ROI are sampled separately, and the average width of the metacarpals and the average thickness of the cortex are calculated using Euclidean distance. The BHI of each metacarpal is calculated separately, and the average is taken as the individual's BHI index. Compared with existing methods, this invention can achieve fully automatic and accurate segmentation and calculation, and introduces a Grabcut-based Unet contour model preprocessing stage, making the algorithm more accurate and stable.

[0085] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A method for calculating bone health indicators based on Unet extraction of regions of interest, characterized in that, Includes the following steps: Step 1: Perform image homogenization and size compression processing; Step 2: Extract the hand region from the image using the Unet contour model trained on Grabcut, then locate the ROI center, and use the ROI center to extract the patch from the image. Step 3: Extract the metacarpal and cortical regions using the Unet segmentation model; Step 4: Based on the metacarpal region and contour, locate the center, distal end, and proximal end of the metacarpal, calculate the midline using linear regression, and draw the ROI based on the midline and the specified ROI center. Step 5: Sample the metacarpal and cortical contours within the ROI, and calculate the average width of the metacarpals and the average thickness of the metacarpal cortex using Euclidean distance. Step 6: Calculate the Bone Health Index (BHI) for each metacarpal bone and take the average as the BHI index. In step 2, the training set of the Unet contour model is generated by the Grabcut algorithm; In step 2, the foreground is separated using the Grabcut algorithm and used as the label for training data to train the Unet contour model. In step 2, the Unet contour model is used to locate the palm position, extract the patch, and focus the region of interest. In step 3, the training set of the Unet segmentation model consists of manually labeled data of the metacarpals and cortex; In step 3, the metacarpal segmentation model outputs predictions for four channels, namely the background and the three metacarpals. In step 3, the cortical segmentation model outputs predictions for 7 channels, namely the background of the three metacarpal bones and the two narrow cortical regions corresponding to each metacarpal bone. Step 6, Bone Health Index (BHI) , Represents the length of the metacarpal bone. Represents the average thickness of the metacarpals. This represents the average thickness of the metacarpal cortex. Pi is the mathematical constant of a circle.

2. The method for calculating bone health indicators based on Unet region of interest extraction as described in claim 1, characterized in that, Step 1, image processing includes pixel value uniformization, pixel value inversion, and CLAHE processing.

3. The method for calculating bone health indicators based on Unet region of interest extraction as described in claim 1, characterized in that, In step 1, the image format includes JPG, PNG, and DCM formats.

Images