A bone age assessment method based on adaptive relationship learning between phalanges and carpal bones

Abstract

The present invention discloses a bone age assessment method based on adaptive relationship learning of phalanges and carpal bones, belonging to the technical field of computer vision and image processing, including: S1, constructing a bone age regression model, including a target detection module, a feature extraction module and a regression module; S2, extracting the middle finger region and carpal region of the target bone through the trained target detection module; S3, respectively extracting the bone age features of the target bone through the feature extraction module, and splicing them into a bone feature map according to the channel; S4, adaptively learning the relationship between the bone features in the bone feature map through the regression module, and assessing the bone age. The model architecture proposed by the present invention uses a variety of attention mechanisms to adaptively learn the developmental characteristics of bones in parts such as phalanges and carpal bones in different age groups, greatly improving the interpretability of the model, obtaining more accurate bone age assessment results, and being closer to clinical medical experience, and the model performance is better than the model using the global image for learning and regression.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision and image processing, and in particular relates to a bone age assessment method based on adaptive relationship learning between phalanges and carpal bones. Background Art

[0002] The development process of human bones is continuous and cyclical. Therefore, bone age represents the biological age of children, reflects their growth potential and sexual maturity, and is often used to predict the height of children in adulthood, as well as to assist in the diagnosis of some endocrine diseases. In clinical practice, doctors usually focus on the development of ossification centers in the phalanges, carpal bones and other areas in X-rays of the hands to infer bone age. The characteristics of bone development are different in different age groups. Doctors have different observation focuses for children of different ages, and make comprehensive judgments based on multiple parts. For example, for younger children, when the skull and hook bones appear in the carpal area, the bone age is about 6 months; when the triangular bone appears, the bone age is about 3 years old. In addition, the degree of development of the phalanges is also crucial to the judgment of bone age. For example, observing the degree of development of the ossification centers of the proximal and distal phalanges can determine their grades and then judge the bone age.

[0003] With the development of artificial intelligence, deep learning methods are increasingly used for automatic analysis of medical images, such as bone analysis in X-ray or computed tomography (CT) images. Existing public datasets, such as those released by RSNA (The Radiological Society of North America) and DHA (Digital Hand Atlas), provide good conditions for the development of artificial intelligence bone age detection models.

[0004] Existing deep learning-based bone age assessment methods have the following problems: First, some methods do not extract regions of interest, or use the entire palm as the region of interest for model training and prediction. However, at different ages, the development of ossification centers in different parts of the phalanges, wrist bones, etc. is different, so the key areas that need to be paid attention to are different; second, the difference in bone development between men and women is significant, and different standards are often used for bone age detection in clinical practice. However, most existing methods do not use gender labels, or train data of different genders independently. Summary of the invention

[0005] In response to the above-mentioned deficiencies in the prior art, the bone age assessment method based on adaptive relationship learning between phalanges and carpal bones provided by the present invention solves the problem that the existing bone age assessment method based on deep learning does not pay attention to the relationship between different parts and bone age, and does not consider the influence of gender on bone age assessment, thereby affecting the final accuracy of bone age assessment.

[0006] In order to achieve the above-mentioned invention object, the technical solution adopted by the present invention is: a bone age assessment method based on adaptive relationship learning between phalanges and carpal bones, comprising the following steps:

[0007] S1. Construct a bone age regression model, including a target detection module, a feature extraction module and a regression module;

[0008] S2, extracting the middle finger area and wrist bone area of ​​the target bone by completing the trained target detection module;

[0009] S3, extracting the bone age features of the target bones respectively through the feature extraction module, and splicing them into a bone feature map according to the channels;

[0010] The bone age characteristics include age group characteristics, gender characteristics, phalangeal characteristics and carpal bone characteristics;

[0011] S4. Adaptively learn the relationship between the bone features in the bone feature graph through the regression module and evaluate the bone age.

[0012] Furthermore, the target detection module extracts a middle finger region and a wrist bone region of fixed size in the target skeleton through a target detection network.

[0013] Further, the S3 includes:

[0014] The original image of the target skeleton is downsampled using the CNN network and then input into the Transformer network to extract global information and obtain its age characteristics;

[0015] The CNN network is used to extract features from the extracted middle finger area and carpal bone area, respectively, to extract the corresponding phalangeal features and carpal bone features;

[0016] Use a matrix of all 0s or all 1s as the gender feature of the target bone;

[0017] The age group characteristics, phalange characteristics, wrist bone characteristics and gender characteristics are spliced ​​according to channels to form a bone feature map.

[0018] Furthermore, the S4 comprises the following sub-steps:

[0019] S41, using the skeleton feature map as the input feature map, using the channel attention mechanism to learn the relationship and weights between different channels of the input feature map, obtaining a channel attention map, and multiplying it with the input feature map to obtain a first feature map;

[0020] S42, using the spatial attention mechanism to learn the features of the first feature map in the spatial dimension, obtain a spatial attention map, and multiply it with the input feature map to obtain a second feature map;

[0021] S43. Use the MLP layer to regress the second feature map to obtain the bone age assessment result.

[0022] Furthermore, in S41, in the channel attention mechanism, global average pooling is first performed on each channel in the skeleton feature map, and then an excitation operation is performed through two sequentially connected fully connected layers to obtain a channel attention map.

[0023] Furthermore, the excitation operation F ex ( z,W ) is expressed as:

[0024] F ex ( z,W )=σ( W 2δ( W 1 z ))

[0025] In the formula, W is the weight matrix, including the weights of the two fully connected layers W 1 and W 2, is the ReLU function, is the Sigmoid function, z For compression channel.

[0026] Furthermore, in S42, in the spatial attention mechanism, the number of channels of the input feature map is compressed using maximum pooling and average pooling, and then the number of channels is changed to 1 using a convolutional layer to obtain the multiplication of the spatial attention map and the input feature map.

[0027] Furthermore, the loss function of the bone age regression model is the MSE loss function.

[0028] Furthermore, the feature extraction module for extracting the age group features also includes an MLP layer and is pre-trained using an MSE loss function.

[0029] The beneficial effects of the present invention are:

[0030] (1) The present invention makes full use of medical prior knowledge to guide the design of the bone age regression model. The proposed model architecture uses multiple attention mechanisms to adaptively learn the developmental characteristics of bones such as phalanges and wrist bones at different age groups, thereby obtaining more accurate bone age assessment results.

[0031] (2) The present invention adopts the method of using global features to guide the learning of local key features, which greatly improves the interpretability of the model and is closer to clinical medical experience.

[0032] (3) The performance of the bone age regression model proposed in this invention is better than the model that uses global images for learning and regression. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A flow chart of the bone age assessment method based on adaptive relationship learning between phalanges and carpal bones provided by the present invention.

[0034] Figure 2 This is a structural diagram of the bone age regression model provided by the present invention.

[0035] Figure 3 Structural diagram of the channel attention mechanism provided by the present invention.

[0036] Figure 4 Structural diagram of the spatial attention mechanism provided for the present invention. DETAILED DESCRIPTION

[0037] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0038] The embodiment of the present invention provides a bone age assessment method based on adaptive relationship learning between phalanges and carpal bones, such as Figure 1 As shown, the following steps are included:

[0039] S1. Construct a bone age regression model, including a target detection module, a feature extraction module and a regression module;

[0040] S2, extracting the middle finger area and wrist bone area of ​​the target bone by completing the trained target detection module;

[0041] S3, extracting the bone age features of the target bones respectively through the feature extraction module, and splicing them into a bone feature map according to the channels;

[0042] Bone age characteristics include age group characteristics, gender characteristics, phalangeal characteristics, and carpal bone characteristics;

[0043] S4. Adaptively learn the relationship between the bone features in the bone feature graph through the regression module and evaluate the bone age.

[0044] like Figure 2As shown, this embodiment provides a bone age regression model based on adaptive relationship learning of multiple attention mechanisms, which extracts the global information of the input image and obtains its age group characteristics through the attention mechanism, and then uses channel attention and spatial attention to adaptively guide the model to learn the relationship between age group, gender, phalangeal characteristics and wrist bone characteristics and output the final bone age assessment result.

[0045] In this embodiment, the target detection module extracts a middle finger region and a wrist bone region of a fixed size from the target skeleton through a target detection network; the target detection network may be any advanced target detection model.

[0046] In an optional embodiment of the present invention, the target detection network is a YoloV5 model, through which the middle finger and wrist bone are extracted, and the two detection areas are limited to a fixed size. , are the height and width of the detection area respectively.

[0047] It should be noted that, in this embodiment, since the target detection module needs to be trained separately, in order to reduce the complexity of training, in this embodiment, only the two areas of the wrist bone and the middle finger are analyzed.

[0048] S3 of the embodiment of the present invention includes:

[0049] The original image of the target skeleton is downsampled using the CNN network and then input into the Transformer network to extract global information and obtain its age characteristics;

[0050] The CNN network is used to extract features from the extracted middle finger area and carpal bone area, respectively, to extract the corresponding phalangeal features and carpal bone features;

[0051] Use a matrix of all 0s or all 1s as the gender feature of the target bone;

[0052] Age group features, phalange features, wrist bone features and gender features are spliced ​​by channel to form a bone feature map.

[0053] Specifically, in this embodiment, when extracting age group features, a multi-head attention mechanism is used to extract global information from the original image using a CNN network combined with a Transformer network to obtain its age group features. The age group features here are different from the precise bone age ultimately output by the entire model. Transformer is used here to extract the global features of the image, estimate the approximate age group of the sample, and output it in the form of a feature map.

[0054] In this embodiment, the input original image , using a convolutional layer with a stride greater than 1 to downsample it to And input into the Transformer layer, the output is The feature map of is the number of channels of the original image, The number of channels of the feature map after being processed by the Transformer layer.

[0055] In this embodiment, when extracting the features of the middle finger and wrist bone area, the input size is , the output size is .

[0056] In this embodiment, the extracted gender features are represented by a matrix of all 1s or all 1s. express.

[0057] In this embodiment, the age group features, finger bone features, wrist bone features and gender features are spliced ​​according to the channel to form a bone feature map: .

[0058] In the embodiment of the present invention, channel attention and spatial attention are used in the regression module to adaptively learn the relationship between the above four features, and finally the MLP layer is used to accurately regress the bone age. Based on this, S4 of the embodiment of the present invention includes the following sub-steps:

[0059] S41, taking the skeleton feature map as the input feature map, using the channel attention mechanism SENet to learn the relationship and weights between different channels of the input feature map, obtaining the channel attention map, and multiplying it with the input feature map to obtain the first feature map;

[0060] S42, using the spatial attention mechanism SAM to learn the features of the first feature map in the spatial dimension, obtain the spatial attention map, and multiply it with the input feature map to obtain the second feature map;

[0061] S43. Use the MLP layer to regress the second feature map to obtain the bone age assessment result.

[0062] In S41 of this embodiment, in the channel attention mechanism, as Figure 3 As shown, global average pooling is first performed on each channel in the bone feature map ( Figure 3 F sq (.) operation), and then the excitation operation is performed through two sequentially connected fully connected layers to obtain the channel attention map; among them, the excitation operation F ex ( z,W ) is expressed as:

[0063] F ex ( z,W )=σ( W 2δ( W 1z ))

[0064] In the formula, W is the weight matrix, including the weights of the two fully connected layers W 1 and W 2, is the ReLU function, is the Sigmoid function, z For compression channel.

[0065] In this embodiment, it is assumed that the skeleton feature map in the input channel attention mechanism is When , the feature map is first squeezed, that is, each channel is globally averaged pooled to obtain the compressed channel , and then through the incentive operation F ex ( .,W ) can be obtained Channel attention map of .

[0066] In S42 of this embodiment, in the spatial attention mechanism, as Figure 4 As shown in the figure, the maximum pooling and average pooling are used to compress the number of channels of the input feature map, and then the convolution layer is used to change its number of channels to 1, and the spatial attention map is multiplied by the input feature map.

[0067] In this embodiment, it is assumed that the input feature map is When , the maximum pooling and average pooling are used to compress the number of channels to 2 ( ), and then use the convolutional layer to change the number of channels to 1 to obtain the spatial attention map 1. Finally, multiply it with the input feature map to get the feature map .

[0068] In the embodiment of the present invention, except that the target detection network needs to be trained separately, the training of the entire bone age regression model is end-to-end, and its loss function is the MSE loss function.

[0069] In an embodiment of the present invention, in order to better sequence model and improve performance, the feature extraction module for extracting age group features includes an MLP layer in addition to the CNN network and the transformer network, and uses the MSE loss function for pre-training.

[0070] The present invention uses specific embodiments to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

[0071] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.

Claims

1. A bone age assessment method based on adaptive relationship learning between phalanges and carpal bones, characterized in that: The following steps are involved: S1. Construct a bone age regression model, including a target detection module, a feature extraction module and a regression module; S2, extracting the middle finger area and wrist bone area of ​​the target bone by completing the trained target detection module; S3, extracting the bone age features of the target bones respectively through the feature extraction module, and splicing them into a bone feature map according to the channels; The bone age characteristics include age group characteristics, gender characteristics, phalangeal characteristics and carpal bone characteristics; S4, adaptively learning the relationship between the bone features in the bone feature map through the regression module, and evaluating the bone age; The S3 includes: The original image of the target skeleton is downsampled using the CNN network and then input into the Transformer network to extract global information and obtain its age characteristics; The CNN network is used to extract features from the extracted middle finger area and carpal bone area, respectively, to extract the corresponding phalangeal features and carpal bone features; Use a matrix of all 0s or all 1s as the gender feature of the target bone; The age group features, phalange features, wrist bone features and gender features are spliced ​​according to the channels to form a bone feature map; The S4 comprises the following sub-steps: S41, using the skeleton feature map as the input feature map, using the channel attention mechanism to learn the relationship and weights between different channels of the input feature map, obtaining a channel attention map, and multiplying it with the input feature map to obtain a first feature map; S42, using the spatial attention mechanism to learn the features of the first feature map in the spatial dimension, obtain a spatial attention map, and multiply it with the input feature map to obtain a second feature map; S43. Use the MLP layer to regress the second feature map to obtain the bone age assessment result.

2. The bone age assessment method based on adaptive relationship learning between phalanges and carpal bones according to claim 1 is characterized in that: The target detection module extracts a middle finger region and a wrist bone region of fixed size from the target skeleton through a target detection network.

3. The bone age assessment method based on adaptive relationship learning between phalanges and carpal bones according to claim 1 is characterized in that: In S41, in the channel attention mechanism, global average pooling is first performed on each channel in the skeleton feature map, and then an excitation operation is performed through two sequentially connected fully connected layers to obtain a channel attention map.

4. The bone age assessment method based on adaptive relationship learning between phalanges and carpal bones according to claim 3 is characterized in that: The stimulus operation F ex ( z,W ) is expressed as: F ex ( z,W )=σ( W 2d( W 1 z )) In the formula, W is the weight matrix, including the weights of the two fully connected layers W 1 and W 2, is the ReLU function, is the Sigmoid function, z For compression channel.

5. The bone age assessment method based on adaptive relationship learning between phalanges and carpal bones according to claim 1, characterized in that: In S42, in the spatial attention mechanism, the number of channels of the input feature map is compressed using maximum pooling and average pooling, and then the number of channels is changed to 1 using a convolutional layer to obtain the multiplication of the spatial attention map with the input feature map.

6. The bone age assessment method based on adaptive relationship learning between phalanges and carpal bones according to claim 1, characterized in that: The loss function of the bone age regression model is the MSE loss function.

7. The bone age assessment method based on adaptive relationship learning between phalanges and carpal bones according to claim 1, characterized in that: The feature extraction module for extracting the age group features also includes an MLP layer and is pre-trained using an MSE loss function.

Images