A bone age intelligent auxiliary assessment method and system based on feature point detection

Abstract

The present invention proposes a method and system for intelligent bone age assessment based on feature point detection. The method includes: preprocessing acquired hand X-ray images; inputting the preprocessed images into an epiphyseal detection model to detect bone key points in the images; the epiphyseal detection model includes: a local network and a global network; centered on the bone key points, the preprocessed images are cut with a fixed length and width to obtain epiphyseal ROI regions; the epiphyseal ROI regions are input into a classification model to obtain corresponding bone categories and developmental levels; and according to a curve chart, the final bone age assessment result is obtained based on the category and developmental level corresponding to each bone. The present invention utilizes a positioning network plus a classification network to replace the target detection network, greatly reducing the amount of data required for training. Furthermore, the data for training the positioning network does not require detailed annotation by professionals, saving annotation costs. The prediction accuracy and time required are basically the same as those of a target detection network trained with large amounts of data.

Description

Technical Field

[0001] The present invention belongs to the field of data processing technology, and in particular relates to a bone age intelligent auxiliary assessment method and system based on feature point detection. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] The intelligent assisted bone age assessment system for children based on the Zhonghua-05 standard is medically based on the "Chinese Wrist Bone Development Standard - Zhonghua-05," edited by Professor Zhang Shaoyan. It incorporates deep learning technology to accurately predict bone age. With improved living standards, parents are increasingly prioritizing their children's height development. A healthy height offers advantages in education, finances, and athletics, making accurate bone age prediction crucial. Over the past 20 years, with increasing affluence, the height of urban children in China has changed significantly. Previously cited foreign bone age standards and older standard atlas methods are no longer applicable. Therefore, around 2005, Professor Zhang Shaoyan's team conducted another survey on bone development in Chinese children and revised the bone age assessment standards, the "Chinese Wrist Bone Development Standard - Zhonghua-05."

[0004] The defects and shortcomings of existing technologies are as follows: ① The traditional atlas method for detecting bone age is closely related to the subjective factors of doctors. There are obvious differences in the DR image evaluation results of different doctors for the same child. In most cases, judgments can only be made based on the doctor's experience; ② Based on target detection algorithms such as SSD and YOLO, the candidate box judgment level is directly generated. Such algorithms are usually used in scenarios with less Ibel, such as pedestrian detection; the Ibel of the Zhonghua 05TW3-C and RUS-CHN methods are both above 100, and it is approximately necessary to obtain higher accuracy. More than 20,000 data points are required. Summary of the Invention

[0005] In order to overcome the above-mentioned deficiencies of the prior art, the present invention provides a method and system for intelligently assisting bone age assessment based on feature point detection.

[0006] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0007] A first aspect of the present invention provides a method for intelligently assisting bone age assessment based on feature point detection, comprising:

[0008] Preprocessing the acquired hand X-ray images;

[0009] Inputting the preprocessed image into an epiphysis detection model to detect bone key points in the preprocessed image; the epiphysis detection model includes: a local network and a global network;

[0010] The pre-processed image is cut with a fixed length and width centered on the key points of the skeleton to obtain the epiphyseal ROI area;

[0011] The epiphyseal ROI region is input into the classification model to obtain the corresponding bone category and development level;

[0012] According to the curve chart, the final bone age assessment result is obtained from the category and development level corresponding to each bone.

[0013] A second aspect of the present invention provides a bone age intelligent auxiliary assessment system based on feature point detection, comprising:

[0014] The preprocessing module is configured to: preprocess the acquired hand X-ray image;

[0015] The skeleton key point detection module is configured to: input the preprocessed image into the epiphysis detection model and detect the skeleton key points in the preprocessed image;

[0016] The epiphysis ROI region acquisition module is configured to: cut the pre-processed image with a fixed length and width centered on the bone key point to obtain the epiphysis ROI region;

[0017] The bone classification module is configured to: input the epiphyseal ROI region into the classification model to obtain the corresponding bone category and development level;

[0018] The bone age assessment result acquisition module is configured to obtain the final bone age assessment result based on the category and development level corresponding to each bone according to the curve chart.

[0019] The third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps of the intelligent assisted bone age assessment method based on feature point detection as described in the first aspect of the present invention.

[0020] The fourth aspect of the present invention provides an electronic device, including a memory, a processor, and a program stored in the memory and runnable on the processor. When the processor executes the program, it implements the steps in the intelligent assisted bone age assessment method based on feature point detection as described in the first aspect of the present invention.

[0021] One or more of the above technical solutions have the following beneficial effects:

[0022] (1) The present invention uses a positioning network plus a classification network to replace the target detection network, which greatly reduces the amount of data required for training. In addition, the data for training the positioning network does not require high-precision labeling by professionals. Doctors only need a simple labeling level, which saves labeling costs. The method proposed in the present invention can achieve good results with 1,100 data, and the prediction accuracy and time are basically the same as the results of directly performing the SSD target detection network with 20,000 data.

[0023] (2) When constructing a feature library for classification, the present invention uses SVM classification and Euclidean distance to calculate similarity simultaneously, which can alleviate the error caused by data imbalance.

[0024] Advantages of additional aspects of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0026] Figure 1 This is a flow chart of the method of the first embodiment of the present invention.

[0027] Figure 2 This is a preprocessing diagram of the first embodiment of the present invention.

[0028] Figure 3 This is the overall structural diagram of the skeleton detection model of the first embodiment of the present invention.

[0029] Figure 4 This is a specific structural diagram of the skeleton detection model of the first embodiment of the present invention.

[0030] Figure 5 This is a skeleton key point generation graph for the first embodiment of the present invention.

[0031] Figure 6 This is a maturity score curve of the RUS-CHN method according to the first embodiment of the present invention. DETAILED DESCRIPTION

[0032] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0033] Example 1

[0034] like Figure 1 As shown, this embodiment discloses a bone age intelligent auxiliary assessment method based on feature point detection, including:

[0035] Step 1: preprocessing the acquired hand X-ray image;

[0036] Step 2: inputting the preprocessed image into an epiphysis detection model to detect bone key points in the preprocessed image; the epiphysis detection model includes: a local network and a global network;

[0037] Step 3: Cut the pre-processed image with a fixed length and width centered on the key point of the bone to obtain the epiphyseal ROI area;

[0038] Step 4: Input the epiphyseal ROI region into the classification model to obtain the corresponding bone category and development level;

[0039] Step 5: Based on the curve chart, the final bone age assessment result is obtained from the category and developmental level corresponding to each bone.

[0040] In step 1, the acquired hand X-ray image is preprocessed, specifically including:

[0041] like Figure 2 As shown in the figure, the original X-ray image format is obtained in DICOM format. After converting it into PNG format, it is first converted into 8-bit grayscale image and then enhanced. The X-ray photo is enhanced by the following methods: rotation, translation, brightness / contrast enhancement and other data augmentation operations. During training, the image is resized to 368×512 size.

[0042] In step 2, if Figure 3 As shown in the figure, the preset epiphysis detection model GU2net consists of two parts, namely the local network and the global network.

[0043] The local network is a Unet network structure, where each convolution is replaced by a separable convolution, which is composed of channel convolution and point convolution (e.g. Figure 4 As shown), this replacement can reduce the number of parameters and increase the model loading speed;

[0044] The global network consists of five 3x3 dilated convolutions with dilation coefficients of [1, 2, 5, 2, 1]. With the enhancement of dilated convolutions, a larger receptive field is achieved, which is conducive to capturing important global information.

[0045] The local input is a 368×512 original image, and the output is a 368×512 local heat map;

[0046] F i L =Φ LN (X i ,θ L ),

[0047] in, Output local heat map; Φ LN : local network; X i : input image; θ L : convolution parameters;

[0048] The global network input is the original image of 368×512 and the local heat map of the local network. After three ordinary convolution activations of the same size, it passes through four dilation coefficients of [1, 2, 5, 2, 1] respectively. The output is a feature layer of (368×512×128) for extracting global information. Finally, the output of the global network and the feature map output by the local network (local net) are multiplied pixel by pixel to output the final global key point heat map F. i , faster and more accurate than traditional positioning networks.

[0049] The output of the global network is:

[0050] F i G =Φ GN (X i , F i L ,θ G ),

[0051] in, Output global heat map; Heat map of local network output; Φ GN : Global network; X i : input image; θ G : convolution parameters;

[0052] At the output global keypoint heat map F i After finding the maximum position of the kth channel, we get the kth key point.

[0053] Keypoints k =argmax(F i，k );

[0054] like Figure 4 As shown in the figure, the specific structure of the global network and local network of this embodiment is shown. In the figure: ① depthwise separable convolution kernel_size = 3, stride = 1, padding = 1; batch normalization; LeakyReLU activation function;

[0055] ②Maximum pooling kernel_size=2, stride=2;

[0056] ③ Depthwise separable convolution kernel_size=3, stride=1, padding=1; batch normalization; LeakyReLU activation function;

[0057] ④ Upsampling, scale_factor = 2.0, mode = bilinear (scaling factor, bilinear interpolation upsampling), depthwise separable convolution kernel_size = 3, stride = 1, padding = 1; batch normalization; LeakyReLU activation function;

[0058] ⑤After concat, do a separate convolution with half the channel;

[0059] ⑥ Downsampling: the image size is reduced to a quarter of the original size, and the output channel is 128;

[0060] ⑦Dilated convolution kernel_size = (3, 3), stride = (1, 1), padding = (1, 1);

[0061] ⑧ Dilated convolution kernel_size = (3, 3), stride = (1, 1), padding = (2, 2);

[0062] ⑨Dilated convolution kernel_size = (3, 3), stride = (1, 1), padding = (5, 5);

[0063] ⑩Dilated convolution kernel_size = (3, 3), stride = (1, 1), padding = (2, 2);

[0064] Dilated convolution kernel_size = (3, 3), stride = (1, 1), padding = (1,, 1);

[0065] The epiphysis detection model in this embodiment is determined after training using multiple hand X-ray images with artificial pre-marked bone key points as training samples. The loss function used in training the epiphysis detection model is the cross entropy loss function.

[0066]

[0067] Among them, y is the actual key point, f is the final heat map prediction,

[0068] like Figure 5As shown in the figure, the trained skeleton detection model generates 13 to 20 key points, of which the 13 points represent the 13 epiphyseal regions of the palm, which are bones present at birth, including the radius, ulna, first metacarpal, third metacarpal, fifth metacarpal, first proximal phalanx, third proximal phalanx, fifth proximal phalanx, third middle phalanx, fifth middle phalanx, first distal phalanx, third distal phalanx, and fifth distal phalanx.

[0069] The remaining 7 points appear gradually during growth and development, and in order of appearance they are: capitate, hamate, triquetrum, lunate, scaphoid, trapezium, and trapezium;

[0070] In step 3, a ROI region is cropped on the original image with a fixed length and width, centered on the key point generated in step 2. The original image here refers to the image before resizing to (368*512*1). The cropped image is resized to (224×224) and then fed into the classification network EfficientNet for feature compression.

[0071] The 224×224×3 image input to the classification network is converted into a 1024-dimensional vector after a series of convolutions, and the final predicted category probability is obtained after softmax. In this embodiment, feature compression is to directly save the 1024-dimensional vector without softmax and maximum value operations.

[0072] In step 4, after the EfficientNet classification is completed, the classification results of the last layer are discarded, and the 2048-dimensional feature vector output by the penultimate layer is saved to establish a feature library (epiphyseal region name + developmental level). The feature library includes the ulna library, radius library, first metacarpal library, etc.

[0073] For example: the radius library includes: radius 1, radius 2, radius 3, radius 4, radius 5, radius 6, radius 7, radius 8, radius 9, radius 10, radius 11, radius 12, radius 13, radius 14; the ulna library includes: ulna 1, ulna 2, ulna 3, ulna 4, ulna 5, ulna 6, ulna 7, ulna 8, ulna 9, ulna 10, ulna 11, ulna 12; the first metacarpal library includes: metacarpal I_1, metacarpal I_2, metacarpal I_3, metacarpal I_4, metacarpal I_5, metacarpal I_6, metacarpal I_7, metacarpal I_8, metacarpal I_9, metacarpal I_10, metacarpal I_11; the third metacarpal library includes: metacarpal III, V_1, metacarpal Ⅲ, V_2, metacarpal Ⅲ, V_3, metacarpal Ⅲ, V_4, metacarpal Ⅲ, V_5, metacarpal Ⅲ, V_6, metacarpal Ⅲ, V_7, metacarpal Ⅲ, V_8, metacarpal Ⅲ, V_9, metacarpal Ⅲ, V_10; the first proximal phalanx library includes: proximal phalanx Ⅰ_1, proximal phalanx Ⅰ_2, proximal phalanx Ⅰ_3, proximal phalanx Ⅰ_4, proximal phalanx Ⅰ_5, proximal phalanx Ⅰ_6, proximal phalanx Ⅰ_7, proximal phalanx Ⅰ_8, proximal phalanx Ⅰ_9, proximal phalanx Ⅰ_10, proximal phalanx Ⅰ_11, proximal phalanx Ⅰ_12; the third proximal phalanx library includes: proximal phalanx Ⅲ, V_1, proximal phalanx Ⅲ, V_2, proximal phalanx Ⅲ, V_3, proximal phalanx Phalanges III, V_4, proximal phalanx III, V_5, proximal phalanx III, V_6, proximal phalanx III, V_7, proximal phalanx III, V_8, proximal phalanx III, V_9, proximal phalanx III, V_10, proximal phalanx III, V_11, proximal phalanx III, V_12; the middle phalanx library includes: middle phalanx III, V_1, middle phalanx III, V_2, middle phalanx III, V_3, middle phalanx III, V_4, middle phalanx III, V_5, middle phalanx III, V_6, middle phalanx III, V_7, middle phalanx III, V_8, middle phalanx III, V_9, middle phalanx III, V_10, middle phalanx III, V_11, middle phalanx III, V_12; The first distal phalanx library includes: distal phalanx I_1, distal phalanx I_2, distal phalanx I_3, distal phalanx I_4, distal phalanx I_5, distal phalanx I_6, distal phalanx I_7, distal phalanx I_8, distal phalanx I_9, distal phalanx I_10, distal phalanx I_11; the third distal phalanx library includes: distal phalanx III, V_1, distal phalanx III, V_2, distal phalanx III, V_3, distal phalanx III, V_4, distal phalanx III, V_5, distal phalanx III, V_6, distal phalanx III, V_7, distal phalanx III, V_8, distal phalanx III, V_9, distal phalanx III, V_10, distal phalanx III, V_11;

[0074] After the new ROI image is loaded, the final category is determined using the SVM classifier and Euclidean distance comparison. That is, the SVM is used within the feature library (for example, the radius library has 14 levels, so SVM training is divided into 14 levels. For a total of 20 feature libraries, 20 SVMs are trained) plus the Euclidean distance-based similarity calculation to determine the final category;

[0075] Euclidean distance comparison does not require training and can be used directly during prediction. For example, during inference prediction, after compressing it into a 1024-dimensional vector, it can be directly compared with the feature information in the feature library one by one, and the closest one is used to determine the category of the image.

[0076] During training, SVM and Euclidean distance-based similarity comparison are used simultaneously. Different classification methods are used for epiphyseal joints that fail based on the results on the validation set. They are used simultaneously because it was found on the validation set that the two methods have complementary functions in predicting some bone and joint areas.

[0077] There are two main reasons why the present invention does not directly use the results output by EfficientNet as the final prediction value: ① After experimental comparison, adding SVM is much better than using EfficientNet directly; ② SVM is faster than neural networks, and it is less likely to have ROI area prediction errors through feature library comparison. For example, ulna level 2 is inferred to radius level. The present invention uses feature library comparison to control the inference result only within the ulna.

[0078] In step 5, after the grade of each epiphyseal region is determined, the scores are added together and the final age can be inferred by looking at the score curve. Figure 6 shown.

[0079] Example 2

[0080] This embodiment discloses an intelligent auxiliary bone age assessment system based on feature point detection, which is characterized by comprising:

[0081] The preprocessing module is configured to: preprocess the acquired hand X-ray image;

[0082] The skeleton key point detection module is configured to: input the preprocessed image into the epiphysis detection model and detect the skeleton key points in the preprocessed image;

[0083] The epiphysis ROI region acquisition module is configured to: cut the pre-processed image with a fixed length and width centered on the bone key point to obtain the epiphysis ROI region;

[0084] The bone classification module is configured to: input the epiphyseal ROI region into the classification model to obtain the corresponding bone category and development level;

[0085] The bone age assessment result acquisition module is configured to obtain the final bone age assessment result based on the category and development level corresponding to each bone according to the curve chart.

[0086] Example 3

[0087] The purpose of this embodiment is to provide a computer-readable storage medium.

[0088] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of an intelligent assisted bone age assessment method based on feature point detection as described in Example 1 of the present disclosure.

[0089] Example 4

[0090] The purpose of this embodiment is to provide an electronic device.

[0091] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, the steps of the intelligent assisted bone age assessment method based on feature point detection as described in Example 1 of the present disclosure are implemented.

[0092] The steps involved in the apparatuses of Examples 2, 3, and 4 above correspond to those of Method Example 1. For detailed implementations, please refer to the relevant description of Example 1. The term "computer-readable storage medium" should be understood to mean a single medium or multiple media containing one or more instruction sets; it should also be understood to include any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and causing the processor to perform any method of the present invention.

[0093] Those skilled in the art will appreciate that the modules or steps of the present invention described above can be implemented using a general-purpose computer device. Alternatively, they can be implemented using program code executable by a computing device, which can then be stored in a storage device and executed by the computing device. Alternatively, they can be fabricated into separate integrated circuit modules, or multiple modules or steps can be fabricated into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0094] Although the above describes the specific embodiments of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without any creative work are still within the scope of protection of the present invention.

Claims

1. A bone age intelligent assisted assessment method based on feature point detection, characterized in that: include: Preprocessing the acquired hand X-ray images; Input the preprocessed image into the epiphysis detection model to detect the bone key points in the preprocessed image; The step of inputting the preprocessed image into the epiphysis detection model and detecting the key bone points in the preprocessed image specifically includes: The preprocessed image is first input into the local network to obtain the local heat map; Then, the preprocessed image and local heat map are input into the global network to obtain the final global keypoint heat map; The epiphysis detection model includes: a local network and a global network, wherein the local network is an improved Unet network, and each convolution in the Unet network is replaced by a separable convolution, and the separable convolution is composed of channel convolution and point convolution; The pre-processed image is cut with a fixed length and width centered on the key points of the skeleton to obtain the epiphyseal ROI area; The epiphyseal ROI region is input into the classification model to obtain the corresponding bone category and development level, including: The classification model is an EfficientNet network based on the attention mechanism. The image to be detected is input into the EfficientNet network based on the attention mechanism to obtain compressed features; The support vector machine and / or Euclidean distance are used to calculate the similarity between the compressed features and the features in the preset feature library to determine the final category and developmental level; According to the curve chart, the final bone age assessment result is obtained from the category and development level corresponding to each bone.

2. The bone age intelligent assisted assessment method based on feature point detection according to claim 1, characterized in that: The epiphysis detection model and the classification model are determined after training using a plurality of hand X-ray images with bone key points manually pre-marked as training samples.

3. The bone age intelligent assisted assessment method based on feature point detection according to claim 1, characterized in that: The loss function used in training the epiphyseal detection model is the cross entropy loss function. Among them, y is the actual key point, f is the final heat map prediction, is the final global keypoint heat map.

4. The bone age intelligent assisted assessment method based on feature point detection according to claim 1, characterized in that: The process of establishing the preset feature library includes: inputting the epiphyseal ROI area of ​​the training sample into the EfficientNet network based on the attention mechanism for feature compression, taking the output features of the EfficientNet network based on the attention mechanism as the 1024-dimensional vector of the second-to-last layer, and establishing the feature library; Alternatively, the support vector machine training process includes: determining the number of training items and the training level of the support vector machine according to the number of skeleton feature libraries and the corresponding levels.

5. An intelligent bone age assisted assessment system based on feature point detection, characterized by: include: The preprocessing module is configured to: preprocess the acquired hand X-ray image; The skeleton key point detection module is configured to: input the preprocessed image into the epiphyseal detection model and detect the skeleton key points in the preprocessed image. The inputting the preprocessed image into the epiphyseal detection model and detecting the skeleton key points in the preprocessed image specifically includes: The preprocessed image is first input into the local network to obtain the local heat map; Then, the preprocessed image and local heat map are input into the global network to obtain the final global keypoint heat map; The epiphysis detection model includes: a local network and a global network, wherein the local network is an improved Unet network, and each convolution in the Unet network is replaced by a separable convolution, and the separable convolution is composed of channel convolution and point convolution; The epiphysis ROI region acquisition module is configured to: cut the pre-processed image with a fixed length and width centered on the bone key point to obtain the epiphysis ROI region; The bone classification module is configured to input the epiphyseal ROI region into the classification model to obtain the corresponding bone category and development level, specifically including: The classification model is an EfficientNet network based on the attention mechanism. The image to be detected is input into the EfficientNet network based on the attention mechanism to obtain compressed features; The support vector machine and / or Euclidean distance are used to calculate the similarity between the compressed features and the features in the preset feature library to determine the final category and developmental level; The bone age assessment result acquisition module is configured to obtain the final bone age assessment result based on the category and development level corresponding to each bone according to the curve chart.

6. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the steps of the intelligent assisted bone age assessment method based on feature point detection as described in any one of claims 1 to 4 are implemented.

7. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the intelligent assisted bone age assessment method based on feature point detection as described in any one of claims 1 to 4 are implemented.

Images