An intelligent bone age assessment method based on an attention mechanism
By performing image preprocessing and region detection on hand bones, combined with a deep learning network using an attention mechanism, the problem of low accuracy in bone age assessment in existing technologies has been solved, achieving high-precision bone age assessment for the Chinese population.
Patent Information
- Application Number
- CN202310553380.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-17
- Publication Date
- 2026-01-02
- Estimated Expiration
- 2043-05-17
AI Technical Summary
Existing intelligent bone age assessment methods are not very accurate in the Chinese population, and assessment methods based on the whole wrist bone have errors, making it difficult to accurately assess bone age.
After image filtering, sharpening, and Gaussian noise reduction, a modified YOLOv5 network was used for hand bone region detection, and a deep learning network combining channel and spatial attention mechanisms was used for bone age assessment. The final assessment was performed using the CHN bone development staging score table.
It improves the accuracy and efficiency of bone age assessment, and enables precise division and segmentation of 14 reference bone regions, thereby enhancing the accuracy of bone age assessment.
Smart Images
Figure CN116596877B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to a bone age intelligent assessment method. BACKGROUND
[0002] Bone age intelligent assessment is to analyze X-ray films to determine a person's bone age and thus estimate his growth and development status, future height, and other information. The CHN method of bone age intelligent assessment is a bone age intelligent assessment method based on Chinese children standard data set and deep learning technology.
[0003] Specifically, the CHN method of bone age intelligent assessment uses deep learning algorithms to train a large number of normal children X-ray films end-to-end, thereby establishing a database of children's bone age sequences and corresponding image features. When a new child's wrist or finger X-ray film needs to be assessed for bone age, the system will automatically extract the image features of the X-ray film and compare them with the existing children's bone age sequences in the database to obtain the assessment results.
[0004] The CHN method of bone age intelligent assessment can effectively reduce the subjective judgment error of doctors and improve the accuracy and repeatability of bone age assessment, and is widely used in hospitals and community clinics. SUMMARY
[0005] The present application overcomes the above-mentioned shortcomings of the prior art and provides a bone age intelligent assessment method based on an attention mechanism.
[0006] The technical problem solved by the present application is solved by the following technical solution:
[0007] A bone age intelligent assessment method based on an attention mechanism, comprising the following steps:
[0008] Step 1: Detect and segment the entire hand bone to generate pictures of 14 bone regions required by the CHN method for bone age assessment;
[0009] Step 2: This part is to complete the entire training process by identifying the grades of the 14 bones respectively;
[0010] Step 3: This part is used for bone age assessment of the testee, compares the bones identified in steps 1 and 2 to obtain the corresponding scores, and then obtains the final assessed bone age through the corresponding scores.
[0011] Step 1 specifically includes:
[0012] 1) Preprocess the entire hand bone, use image filtering and sharpening technology to make the bone area clearer and more explicit, this part uses mean filtering, the formula is as follows:
[0013]
[0014] 2) Gaussian denoising of the picture, which uses a Gaussian filter to smooth the image by calculating the weighted average of the neighborhood pixels around each pixel. Specifically, it creates a small normal distribution kernel around the pixel, assigning the highest weight to the pixels that have the most impact on the pixel when using convolution operations for smoothing. The formula of Gaussian filter is as follows:
[0015] G(x,y) = 1 / (2πσ 2 ) × e∧(-x ∧ 2+y∧2) / 2σ 2 (2-2)
[0016] Where G(x,y) represents the filtered pixel value; (x,y) represents the coordinates of the current pixel; σ represents the standard deviation of the filter kernel, in actual operation, we control the smoothing degree by adjusting the standard deviation and window size.
[0017] 3) Binaryzation of the processed image, the formula is as follows:
[0018]
[0019] Where image(x,y) represents the pixel value of the original image at coordinates (x,y), result(x,y) represents the pixel value of the processed binary image at coordinates (x,y), and threshold represents the threshold value.
[0020] 4) Using the improved YOLOv5 network to detect the region of the processed whole hand bone image and extract the corresponding 14 bone regions, the whole network structure is divided into three parts: Backbone, Neck and Head. Backbone uses CSPNet(CenterNet-style Feature Pyramid Network) structure, mainly realizes feature extraction through residual block, Neck module uses SPP(Spatial Pyramid Pooling) and PAN(Path Aggregation Network) two mechanisms, respectively used for extracting multi-scale features and realizing information fusion, Head module is responsible for predicting the position and category information of the detection frame, and the segmented 14 bone regions are used for the next work.
[0021] Step 2 specifically includes:
[0022] 1) Select ResNet34 as the backbone network, which includes: 1 convolution layer, 1 pooling layer, about 32 convolution layers and 64 standardization layers composed of basic blocks, and 1 global average pooling layer and 1 fully connected layer.
[0023] 2) Add the channel attention mechanism module and the spatial attention mechanism module to the network after the first convolution, the channel attention mechanism aims to adjust the weight of the feature map according to the relationship between each channel in the input tensor, and the channel attention mechanism formula is as follows:
[0024] f exc = ReLU (BN (E1 averagepooling (x))) (2-4)
[0025] f gate = sigmoid (BN (W2 averagepooling (x))) (2-5)
[0026] y = f gate ⊙x + (1-f gate )⊙f exc (2-6)
[0027] Where x represents the input feature map, W1 and W2 represent the weight matrix of 1x1 convolution respectively, averagepooling is the global average pooling operation, f exc is the activation function ReLU applied to the output after pooling, f gate is the gating function sigmoid applied to the output after pooling, and represents element-wise multiplication; the spatial attention mechanism focuses on learning the dependency relationship in space to ensure that the region with important information receives more attention. The spatial attention mechanism used in the present application is a common self-attention mechanism, also known as a multi-head attention mechanism, and its formula is:
[0028]
[0029] Where Q, K, and V represent query, key and value respectively, and the divisor in the formula is used to adjust the scale of the result, and through the self-attention mechanism, the relationship between any two pixels in the feature map can be learned, and multiple attention heads are used at different layers to improve performance.
[0030] 3) Add the two attention mechanism modules described above before the last pooling layer and the fully connected layer, then perform the last pooling and fully connected operation to obtain the recognized corresponding level, and train in this process to obtain the final network structure.
[0031] Step 3 specifically includes:
[0032] 1) The whole hand bone of the human body to be tested is subjected to the region detection of step 1, and is segmented into 14 corresponding bone regions, including the radius, metacarpal bone I, metacarpal bone III, metacarpal bone V, proximal phalanx I, proximal phalanx III, proximal phalanx V, middle phalanx III, middle phalanx V, distal phalanx I, distal phalanx III, distal phalanx V, capitate bone and hamate bone.
[0033] 2) The grade identification of step 2 is performed on each reference bone, and the trained network is used for grade identification.
[0034] 3) According to the grades of the 14 reference bones, the score of each bone is obtained by referring to the CHN method bone development stage score table.
[0035] 4) According to the above scores, sum operation is performed on them, and the final bone age value is obtained by comparing the total score with the CHN method bone development maturity score and bone age comparison table.
[0036] The general bone age intelligent evaluation system is based on the X-ray film of the whole wrist bone, although the process is simple, but the precision is not high, and for the bone age evaluation of Chinese people, the CHN method more suitable for the bone development process of Chinese people is more suitable. The application first uses filtering technology, Gaussian noise reduction and binaryzation processing, and then uses YoloV5 network to detect and segment the region of interest, realizes accurate division and segmentation of 14 reference bone regions, then recognizes the grade through the recognition network after adding attention mechanism, and finally obtains the final bone age value by comparing the bone development stage score table and the CHN method bone development maturity score and bone age comparison table.
[0037] The application has the following beneficial effects:
[0038] (1) The original image is subjected to data enhancement and preprocessing, and more accurate 14 reference bone regions are extracted.
[0039] (2) Two attention mechanism methods are integrated into the recognition network, which greatly improves the recognition efficiency and recognition accuracy, thereby improving the precision of the final bone age evaluation. BRIEF DESCRIPTION OF DRAWINGS
[0040] Figure 1 is the image preprocessing flowchart of the application.
[0041] Figure 2 is the hand bone segmentation diagram of the application.
[0042] Figure 3 is the bone age evaluation flowchart of the application. DETAILED DESCRIPTION
[0043] The technical solutions of the application will be further described below with reference to the drawings.
[0044] An intelligent bone age assessment method based on an attention mechanism, comprising the following steps:
[0045] Step 1: Detect and segment the whole hand bone to generate a picture of 14 bone regions required by CHN method for bone age assessment;
[0046] Step 2: This part is to complete the whole training process by grade recognition of 14 bones respectively;
[0047] Step 3: This part is used for bone age assessment of the testee, and the bones identified by grade in steps 1 and 2 are compared to obtain the corresponding score, and then the final assessed bone age is obtained through the corresponding score.
[0048] Step 1 specifically includes:
[0049] 1) Preprocess the whole hand bone, adopt image filtering and sharpening technology to make the bone region more clear and explicit, this part adopts mean filter, the formula is as follows:
[0050]
[0051] 2) Gaussian noise reduction is performed on the picture, this process adopts Gaussian filter, which smoothes the image by calculating the weighted average value of the neighborhood pixels around each pixel point. Specifically, it assigns the highest weight to the pixels that have the greatest impact on the pixel by creating a small normal distribution kernel around the pixel when using convolution operation for smoothing, the formula of Gaussian filter is as follows:
[0052] G(x,y)=1 / (2πσ 2 )×e∧(-x∧2+y∧2) / 2σ 2 (2-2)
[0053] Wherein, G(x,y) represents the pixel value after filtering; (x,y) represents the coordinates of the current pixel; σ represents the standard deviation of the filter kernel, in actual operation, we control the smoothing degree by adjusting the standard deviation and window size.
[0054] 3) Perform binaryzation processing on the above processed image, the binaryzation formula is as follows:
[0055]
[0056] Wherein, image(x,y) represents the pixel value of the original image at coordinates (x,y), result(x,y) represents the pixel value of the processed binary image at coordinates (x,y), and threshold represents the threshold value.
[0057] 4) Using the improved YOLOv5 network to detect the region of the processed whole hand bone picture and extract the corresponding 14 bone regions, the whole network structure is divided into three parts: Backbone, Neck and Head, Backbone uses CSPNet(CenterNet-style Feature Pyramid Network) structure, mainly through residual block to realize feature extraction, Neck module uses SPP(Spatial Pyramid Pooling) and PAN(Path Aggregation Network) two mechanisms, respectively used for extracting multi-scale features and realizing information fusion, Head module is responsible for predicting the position and category information of the detection box, and the segmented 14 bone blocks are used for the next work.
[0058] Step 2 specifically includes:
[0059] 1) Select ResNet34 as the backbone network, which includes: 1 convolution layer, 1 pooling layer, about 32 convolution layers and 64 standardization layers composed of basic blocks, and 1 global average pooling layer and 1 fully connected layer.
[0060] 2) Add channel attention mechanism module and spatial attention mechanism module to the network after the first convolution, the channel attention mechanism aims to adjust the weight of the feature map according to the relationship between each channel in the input tensor, and the channel attention mechanism formula is as follows:
[0061] f exc =ReLU(BN(E1 averagepooling(x))) (2-4)
[0062] f gate =sigmoid(BN(E2 averagepooling(x))) (2-5)
[0063] y=f gate ⊙x+(1-f gate )⊙f exc (2-6)
[0064] Where x represents the input feature map, W1 and W2 represent the weight matrix of 1x1 convolution respectively, averagepooling is the global average pooling operation, f exc is the output after applying the activation function ReLU to the pooling, f gateis a gating function sigmoid applied to the output after pooling, and represents element-wise multiplication; the spatial attention mechanism focuses on learning the dependence relationship in space to ensure that the region with important information is paid more attention. The spatial attention mechanism adopted in the present application is a common self-attention mechanism, also known as a multi-head attention mechanism, and its formula is:
[0065]
[0066] wherein Q, K and V represent query, key and value respectively, and the divisor is used to adjust the scale of the result. The self-attention mechanism can learn the relationship between any two pixels in the feature map, and multiple attention heads are used at different layers to improve performance.
[0067] 3) The two attention mechanism modules described above are added before the last pooling layer and the fully connected layer, and then the last pooling and fully connected operation is performed to obtain the recognized corresponding grade. In this process, the final network structure is obtained by training.
[0068] Step 3 specifically includes:
[0069] 1) The region detection of step 1 is performed on the entire human hand bone to be tested, and is segmented into 14 corresponding bone regions, including radius, metacarpal bone I, metacarpal bone III, metacarpal bone V, proximal phalanx I, proximal phalanx III, proximal phalanx V, middle phalanx III, middle phalanx V, distal phalanx I, distal phalanx III, distal phalanx V, capitate bone and hamate bone.
[0070] 2) The grade identification of step 2 is performed on each reference bone, and the trained network is used for grade identification.
[0071] 3) According to the grades of the 14 reference bones obtained, the scores of each bone are obtained by referring to the CHN method bone development stage score table.
[0072] 4) According to the above scores, sum operation is performed on them, and the final bone age value is obtained by comparing the total score with the CHN method bone development maturity and bone age comparison table.
[0073] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not limited thereto; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions described in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. An attention mechanism-based bone age intelligent evaluation method, comprising the following steps: Step 1: Detect and segment the whole hand bone to generate the picture of 14 bone regions required by CHN method for bone age evaluation; specifically including: 11) Pretreat the whole hand bone, adopt image filtering and sharpening technology to make the bone region more clear and explicit, this part adopts mean filtering, the formula is as follows: 12) Gaussian noise reduction is performed on the picture, which adopts Gaussian filter, which smoothes the image by calculating the weighted average value of the neighborhood pixels around each pixel point; specifically, by creating a small normal distribution kernel around the pixel, the pixel with the greatest impact on the smoothing process is assigned the highest weight when using convolution operation, the formula of Gaussian filter is as follows: G(x, y) = 1 / (2πσ 2 ) x e ∧ (-x ∧ 2+y ∧ 2) / 2σ 2 (2-2) Where G(x,y) represents the pixel value after filtering; (x,y) represents the coordinates of the current pixel; σ represents the standard deviation of the filter kernel, in actual operation, we control the smoothing degree by adjusting the standard deviation and window size; 13) The above processed image is binarized, the formula of binarization is as follows: Where image(x,y) represents the pixel value of the original image at coordinates (x,y), result(x,y) represents the pixel value of the processed binary image at coordinates (x,y), and threshold represents the threshold value; 14) The improved YOLOv5 network is used to detect the region of the processed whole hand bone picture and extract the corresponding 14 bone regions, the whole network structure is divided into three parts: Backbone, Neck and Head, Backbone adopts CSPNet(CenterNet-style Feature Pyramid Network) structure, mainly realizes feature extraction through residual block, Neck module adopts SPP(Spatial Pyramid Pooling) and PAN(Path Aggregation Network) two mechanisms, respectively used for extracting multi-scale features and realizing information fusion, Head module is responsible for predicting the position and category information of the detection frame, and the segmented 14 bones are used for the following work; Step 2: This part is to identify the grades of the 14 bones respectively to complete the whole training process; specifically including: 21) Select ResNet34 as the backbone network, which includes: 1 convolution layer, 1 pooling layer, about 32 convolution layers and 64 standardization layers composed of basic blocks, and 1 global average pooling layer and 1 fully connected layer; 22) Add channel attention mechanism module and spatial attention mechanism module to the network after the first convolution, channel attention mechanism aims to adjust the weight of its feature map according to the relationship between each channel in the input tensor, the formula of channel attention mechanism is as follows: f exc = ReLU(BN(W1averagepooling(x))) (2-4) f gate = sigmoid(BN(W2averagepooling(x))) (2-5) y = f gate ⊙x + (1 - f gate )⊙f exc (2-6) where x represents the input feature map, W1and W2represent the weight matrix of 1x1 convolution, averagepooling is the global average pooling operation, f exc is the activation function ReLU applied to the output after pooling, f gate is the gating function sigmoid applied to the output after pooling, and represents element-wise multiplication; the spatial attention mechanism focuses on learning the spatial dependency to ensure that the region with important information receives more attention. A common spatial attention mechanism used in this method is the self-attention mechanism, also known as the multi-head attention mechanism, whose formula is: where Q, K, V represent query, key and value respectively, and the divisor in the formula To adjust the scale of the results, the self-attention mechanism can learn the relationship between any two pixels in the feature map, and multiple attention heads are used in different layers to improve performance. 23) The two attention mechanism modules described above are added before the last pooling layer and fully connected layer, and then the final pooling and fully connected operation is performed to obtain the recognized corresponding grade. In this process, the final network structure is obtained by training; Step 3: This part is used for the evaluation of the bone age of the subject. The bones identified in steps 1 and 2 are compared to obtain the corresponding scores, and the final evaluated bone age is obtained through the corresponding scores. Specifically, it includes: 31) The whole hand bones of the subject are subjected to the region detection of step 1, and are segmented into 14 corresponding bone regions, including radius, metacarpal bone I, metacarpal bone III, metacarpal bone V, proximal phalanx I, proximal phalanx III, proximal phalanx V, middle phalanx III, middle phalanx V, distal phalanx I, distal phalanx III, distal phalanx V, capitate bone, and hamate bone; 32) The grade identification of step 2 is performed on each reference bone, and the trained network is used for grade identification; 33) According to the grades of the 14 reference bones, the scores of each bone are obtained by referring to the CHN method bone development stage score table; 34) According to the above scores, sum operation is performed on them, and the final evaluated bone age value is obtained by comparing the total score with the CHN method bone development maturity and bone age comparison table.