From coarse to fine bone age regression method based on multi-scale high-order self-attention

Abstract

The application discloses a coarse-to-fine bone age regression method based on multi-scale high-order self-attention, first extracts multi-scale features and multi-scale global self-attention response maps of the original hand bone image, fuses the latter and the down-sampling features of the original hand bone image to obtain fusion features, inputs the fusion features into a regression module, calculates an adaptive triad loss in the regression module to obtain a bone age regression value and an MSE loss for calculating the bone age, adjusts model parameters, trains the model on an original data set, then inputs the original data set into a backbone network, extracts high-dimensional representations of the image and inputs the high-dimensional representations into a K-means clustering algorithm, clusters the data set into multiple age stages, finally inputs the data in each age stage into multiple original backbone networks, trains multiple regression models, and obtains bone age prediction values in each age stage. The method can effectively perform left-hand bone regression detection of teenagers, and provides a general and efficient regression model and regression framework for a clinical computer bone age regression application scenario.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer vision and medical image processing, and particularly relates to a coarse-to-fine bone age regression method based on multi-scale high-order self-attention. BACKGROUND

[0002] Bone age (BA) is used in clinical practice to reveal the biological age of a child's skeleton and diagnose the degree of its development and maturity. Bone age assessment (BAA) is a widely used radiological examination method that requires determining specific locations of the skeleton in an X-ray image. In clinical practice, X-ray images of the skeleton of a child's left hand are usually taken. The development of the human skeleton has continuity and different stages, and the skeletons of different ages have different morphological characteristics. The results of bone age assessment clearly indicate the level of growth, development and maturity of an individual.

[0003] In recent years, due to the rapid development of artificial intelligence, especially deep learning, high-dimensional features can be extracted from pictures, and in the case of sufficient data, only a single model can be used to adapt to any complex function, making deep learning technology widely used in the field of computer science and medicine. Deep learning methods have made remarkable achievements in the field of automated bone age assessment. More and more automated bone age assessment systems have been introduced, but most of them have adopted one of the two mainstream architectures.

[0004] The documents "C. Spampinato, S. Palazzo, D. Giordano, M. Aldinucci, and R. Leonardi. Deep learning for automated skeletal bone age assessment in x-ray images. Medical Image Analysis, 2017" and "S. Koitka, M. S. Kim, M. Qu, A. Fischer, C. M. Friedrich, and F. Nensa. Mimicking the radiologists' workflow: Estimating pediatric hand bone age with stacked deep neural networks. Medical image analysis, 64:101743, 2020" respectively propose a mainstream architecture, the first mainstream architecture of an early single end-to-end neural network, for example, an architecture represented by BoNet, directly processes convolutional features on the input picture, and finally estimates the bone age value through a fully connected layer regression. The second mainstream architecture represented by Seven et al. strictly follows the clinical bone age diagnosis method, uses an existing target detection network to extract about a dozen joint regions of the original left hand bone picture, and finally trains an equal number of regression models to judge the bone age. Compared with the former, the second mainstream architecture is more likely to obtain higher accuracy, but it needs to label a large number of hand bone region labels and train a large number of regression networks, which consumes a lot of manpower and material resources. But in fact, both of the two mainstream architectures of bone age regression method face the defects of effect bottleneck and huge workload, which is difficult to apply in actual scene. SUMMARY

[0005] To solve the above technical problems, the present application provides a coarse-to-fine bone age regression method based on multi-scale high-order self-attention, which obtains a BoGFF-Net regression model by fusing multi-scale global information, and solves the problem of heterogeneity of hand bones in each age group and the difficulty of a single regression model to fit all data, to obtain a general and simple and efficient coarse-to-fine bone age regression model and framework.

[0006] The technical scheme adopted by the present application is as follows:

[0007] S1, input the original hand bone image with real bone age label into the backbone network, extract the multi-scale features and multi-scale global self-attention response map of the original hand bone image;

[0008] S2, fuse the multi-scale global self-attention response graph and the down-sampled final features of the original hand bone image to obtain multi-scale fusion features, and input the multi-scale fusion features into a loss function calculation and regression module;

[0009] S3, calculate an adaptive triplet loss in the regression module, and obtain a final bone age regression value and an MSE loss of the bone age through two fully connected layers;

[0010] S4, adjust model parameters, and perform model training on an original data set; after the training is completed, input the original data set into a backbone network, and extract high-dimensional representations of images from the final two fully connected layers;

[0011] S5, input the high-dimensional representations of the original data set into a K-means clustering algorithm, and use an unsupervised manner to cluster the data set into multiple age stages;

[0012] S6, input data of each age stage into multiple original backbone networks, repeat steps S1 to S3, train the multiple original backbone networks, and finally obtain bone age prediction values of each age stage.

[0013] Further, the step S1 is specifically as follows:

[0014] The multi-scale features X1, X2, X3 and X4 of the extracted original hand bone image and the multi-scale global self-attention response graph C1, C2, C3 and C4 are specifically as follows:

[0015] X i+1 =Stages(X i ) (1)

[0016] Wherein, Stages represents stacked convolutional neural network layers and batch normalization layers, X i represents a scale feature map of the i-th level, i=1, 2, 3, 4.

[0017] The hierarchical features X i are input into a block embedding layer PE to obtain a query matrix Q i , a key matrix K i and a value matrix V i , and the expressions are as follows:

[0018] Q i =PE Q (X i ) (2)

[0019] K i =PE K (X i ) (3)

[0020] V i =PE V(X i ) (4)

[0021] wherein PE represents a patch embedding layer PatchEmbedding, which is composed of a fully connected layer; the patch embedding size follows:

[0022] P N = P1 / 2 N-1 (5)

[0023] wherein P represents an image patch, the value of P1 is 16x16 pixels, and N represents a level in the global feature modeling module, N = 1, 2, 3, 4.

[0024] Then, the obtained query matrix Q i , key matrix K i and value matrix V i are used to calculate a single high-order self-attention head:

[0025]

[0026] wherein, represent three linear change matrices in the self-attention process respectively, and have the same dimension, H u represents the calculation result of a single high-order self-attention head, and in this work, u = 1, 2,..., 8; A is defined by the following formula:

[0027]

[0028] wherein, T represents a matrix transposition operation, and represents a normalized change scale, which is a constant; Softmax represents a commonly used maximum value transmission function in a multi-class classification problem.

[0029] The calculated single high-order self-attention head is then spliced in any spatial dimension to obtain the global feature fusion feature C i of the level:

[0030] C i = Concatate(H1, H2,..., H u ) (8)

[0031] wherein Concatate represents a spatial splicing operation on a feature map.

[0032] Further, the step S2 is specifically as follows:

[0033] The multi-scale fusion feature C is expressed as follows:

[0034]

[0035] where Concat denotes the spatial concatenation operation on feature maps, FPN denotes the feed-forward neural network composed of ordinary fully connected layers and normalization layers, G denotes the gender label of the original input picture, and denotes the addition of the element values in the corresponding positions of the matrices.

[0036] The obtained multi-scale global fusion features are subjected to a global average pooling layer GAP to obtain the final feature F corresponding to one picture:

[0037] F=GAP(C) (10)

[0038] Further, the step S3 is specifically as follows:

[0039] For the data in one batch, any one picture is selected as an anchor point, and another two pictures are selected as a far point and a near point, respectively, to calculate the adaptive triplet loss, and the expression is as follows:

[0040] L triplet =||F a -F n || 2 -||F a -F f || 2 +αm (11)

[0041] where L triplet denotes the adaptive triplet loss, F a denotes the representation of the anchor point picture, F n denotes the representation of the near point picture, F f denotes the representation of the far point picture, m denotes an edge constant, and α denotes an adaptive coefficient determined by the following formula:

[0042] α=||y a -y n ||-||y a -y f || (12)

[0043] where y a , y n , and y f denote the anchor point label age, the near point label age, and the far point label age, respectively; then the final feature F corresponding to a single picture is subjected to an MLP network to obtain the final predicted bone age Y, and then the MSE loss function L MSE is calculated between the bone age label and the predicted bone age:

[0044] Y=MLP(F) (13)

[0045] L MSE =||Y-Y l || 2 (14)

[0046] wherein, the MLP network is composed of two separate fully connected layers; Y l represents the bone age label corresponding to the hand bone picture.

[0047] Further, the step S4 is specifically as follows:

[0048] The model is trained by the loss function, and the total loss function L expression is as follows:

[0049] L=λL triplet +L MSE (15)

[0050] wherein, λ represents the proportion of adaptive triplet loss in the total loss function.

[0051] After the model training is completed, the 1024-dimensional high-dimensional features D 1024 corresponding to the original picture are extracted from the last MLP layer.

[0052] D 1024 =Extract(MLP(F)) (16)

[0053] wherein, Extract represents removing a part of the model and directly outputting.

[0054] Further, the step S5 is specifically as follows:

[0055] The expression of the clustering class obtained by the K-means clustering algorithm is as follows:

[0056] G1,G2,...,G g =KMEANS({D 1024}) (17)

[0057] wherein, {D 1024} represents the set of 1024-dimensional features of the data set, KMEANS represents the K-means clustering algorithm, G1,G2,...,G g represents the clustering class obtained by the K-means clustering algorithm, and g is fixed as 5.

[0058] Further, the step S6 is specifically as follows:

[0059] The expression of the bone age prediction value of each age stage is as follows:

[0060] BoneAge=BoGFF g (G g ) (18)

[0061] wherein BoneAge represents the final predicted age, BoGFF g represents the gth regression model.

[0062] The method of the present application firstly extracts the multi-scale features and multi-scale global self-attention response map of the original hand bone image, fuses the latter and the down-sampling features of the original hand bone image to obtain the fusion features, inputs the fusion features into the regression module, calculates the adaptive triad loss in the regression module to obtain the bone age regression value and the MSE loss of the bone age, adjusts the model parameters, trains the model on the original data set, then inputs the original data set into the backbone network, extracts the high-dimensional representation of the image and sends it into the K-means clustering algorithm, clusters the data set into multiple age stages, and finally inputs the data of each age stage into multiple original backbone networks to train multiple regression models to obtain the bone age prediction value of each age stage. The method of the present application breaks through the original technical framework, discards the high labor consumption brought by the target detection method, obtains the BoGFF-Net regression model by fusing multi-scale global information, solves the problem of heterogeneity of hand bones in each age stage and the difficulty of a single regression model to fit all data, and obtains a general and simple and efficient bone age regression model and framework from coarse to fine, which can effectively perform left hand bone regression detection of teenagers and is suitable for clinical computer bone age regression application scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0063] Figure 1 The flowchart of the method of the present application is shown in the figure.

[0064] Figure 2 The model training flowchart in the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0065] The method of the present application will be further described below in combination with the drawings and embodiments.

[0066] As shown in the figure, the flowchart of the method of the present application is shown in the figure. Figure 1 S1, input the original hand bone image with real bone age label into the backbone network, extract the multi-scale features and multi-scale global self-attention response map of the original hand bone image;

[0067] S2, fuse the multi-scale global self-attention response map and the down-sampling final features of the original hand bone image to obtain the multi-scale fusion features, and input them into the loss function calculation and regression module;

[0068]

[0069] ​S3, calculate the adaptive triplet loss in the regression module, and obtain the final bone age regression value and calculate the MSE loss of bone age through two full connection layers;

[0070] S4, adjust the model parameters, and train the model on the original data set; after the training is completed, input the original data set into the backbone network, and extract the high-dimensional representation of the image from the final two full connection layers;

[0071] S5, send the high-dimensional representation of the original data set into the K-means clustering algorithm, and use the unsupervised method to cluster the data set into multiple age stages;

[0072] S6, send the data of each age stage into multiple original backbone networks, repeat the steps S1 to S3, train multiple original backbone networks, and finally obtain the bone age prediction value of each age stage.

[0073] Figure 2 Model training flowchart in the embodiment of the application

[0074] In the embodiment, the step S1 is specifically as follows:

[0075] The extracted multi-scale features X1, X2, X3 and X4 of the original hand bone image and the multi-scale global self-attention response graph C1, C2, C3 and C4 are specifically as follows:

[0076] X i+1 =Stages(X i ) (1)

[0077] Wherein, Stages represents a stacked convolutional neural network layer and a batch normalization layer, as shown in the FE module in figure (2); in particular, as long as the down-sampling scale is met, there is no specific requirement for the number of stacked neural network layers; X i represents the scale feature map of the i-th level, i=1, 2, 3, 4.

[0078] Specifically, Stages can be represented by formula (2):

[0079] Stages=Stacked(Conv,BN,...) (2)

[0080] Wherein, Stages represents a stacked convolutional neural network layer and a batch normalization layer, as shown in the FE module in figure (2); in particular, as long as the down-sampling scale is met, there is no specific requirement for the number of stacked neural network layers; X i represents the scale feature map of the i-th level, i=1, 2, 3, 4.

[0081] The hierarchical feature X i is input into the patch embedding layer PE to obtain the query matrix Q i , the key matrix K i and the value matrix V i , and the expression is as follows:

[0082] Q i = PE Q (X i ) (3)

[0083] K i = PE K (X i ) (4)

[0084] V i = PE V (X i ) (5)

[0085] While the feature map size is reduced, the hierarchical feature X i of each layer is input into the global feature fusion module GFF, so that it is input into the patch embedding layer PE to obtain the query matrix Q i , the key matrix K i and the value matrix V i , and the three matrices are important parameters for subsequent calculation of self-attention; PE in equations (3), (4) and (5) represents the patch embedding layer PatchEmbedding, which is composed of a fully connected layer; the patch embedding size follows:

[0086] P N = P1 / 2 N-1 (6)

[0087] wherein P represents an image block, the value of P1 is 16x16 pixels, and the image block size of each subsequent layer is 8x8, 4x4 and 2x2; N represents the level in the global feature modeling module, and N = 1, 2, 3, 4.

[0088] Under the condition that the input image size and the selected image block size are reduced by the same ratio, each layer in the GFF module obtains the same number of image blocks, and the number of image blocks is constant at 64, which also makes each layer in the GFF module output a global information fusion feature map of the same scale, with a size of 256x64, i.e., each image block finally has a 256-dimensional vector to represent it;

[0089] The obtained query matrix Q i , the key matrix K i and the value matrix V i are used to calculate a single high-order self-attention head:

[0090]

[0091] wherein, denote three linear transformation matrices in the self-attention process respectively, have the same dimension, H u denote the calculation results of a single high-order self-attention head, in this work, u = 1, 2,..., 8, that is, in the i-th level, 8 single high-order self-attention heads are calculated, and each self-attention head corresponds to different A can be defined by the following formula:

[0092]

[0093]

[0094] wherein, T denote the matrix transposition operation, and represent the normalized change scale, which is a constant, and the purpose is to make the result of QK satisfy the distribution with expectation of 0 and variance of 1, which can be understood as normalization; formula (8) is the original self-attention calculation formula, and formula (9) is the high-order self-attention calculation formula after improvement; define the first matrix multiplication as the first-order spatial information interaction, the original calculation formula obtains the second-order spatial information interaction, and the calculation formula after improvement can obtain an arbitrary order spatial information interaction; Softmax represents a commonly used maximum value transmission function in a multi-class classification problem, which is used to convert the numerical dimension of a specific vector into a probability dimension, which can be expressed by formula (10):

[0095]

[0096] The calculated single high-order self-attention head is spliced in an arbitrary spatial dimension to obtain the global feature fusion feature C i :

[0097] C i = Concatate (H1, H2,..., H u ) (11)

[0098] wherein, Concatate represents a spatial splicing operation on the feature map.

[0099] In this embodiment, the step S2 is specifically as follows:

[0100] For the data in a batch, any one picture is selected as an anchor point, and any other two pictures are selected in addition to the picture, one as a far point and the other as a near point to calculate the adaptive triplet loss, and the expression is as follows:

[0101]

[0102] FPN(x) = Relu(FC(x)) (13)

[0103] where Concatate denotes the spatial concatenation operation of the feature maps C1, C2, C3, C4 and X4 are all obtained by step S1, and according to the foregoing description, each layer in the GFF module outputs a global information fusion feature map of the same size, with a size of 256x64, the vector is spliced according to the second spatial dimension, and finally the GFF module outputs a 256x256 tensor representation; denotes the addition of the element values in the corresponding positions of the matrices, and the high-dimensional representation obtained through this step contains both multi-scale global information and local information extracted by the convolutional layer; FPN represents a feedforward neural network composed of a general fully connected layer FC and a linear rectifier unit Relu activation function, and the linear rectifier unit can be simply expressed as G represents the gender label of the original input picture, and the male and female gender labels are simply coded as numbers 0 and 1, expressed as

[0104] The obtained multi-scale global fusion feature is input into a global average pooling layer GAP to obtain the final feature F corresponding to a picture:

[0105] F = GAP(C) (14)

[0106] The formula is where the sizes of H and W are both 256, obtained from equation (12); after equation (14), the dimension of the feature vector corresponding to a single picture changes to 100x1.

[0107] In the embodiment, the step S3 is specifically as follows:

[0108] For the data in a batch, any one picture is selected as an anchor point, and in addition to the picture, another two pictures are selected, one as a far point and the other as a near point to calculate the adaptive triplet loss, expressed as follows:

[0109] L triplet =||F a -F n || 2 -||F a -F f || 2 +αm (15)

[0110]

[0111] Since the present application involves a regression problem, and the label of the regression problem is continuous, it is expected that the representation of the picture can also be continuous in the high-dimensional space, so the adaptive triplet loss is designed to punish the discontinuous picture representation in the high-dimensional space.

[0112] wherein L triplet represents the adaptive triplet loss, F a represents the representation of the anchor picture, F n represents the representation of the near picture, and F f represents the representation of the far picture, m represents an edge constant that can force a certain distance to be maintained between the two teams of samples, and when the triplet loss is less than 0, the calculation result is ignored, and is represented by formula (16); and a represents an adaptive coefficient that can be determined by the following formula:

[0113] a = || y a -y n || - || y a -y f || (17)

[0114] wherein y a , y n , and y f respectively represent the anchor label age, the near label age, and the far label age, and at this time, the value of the triplet loss can be adaptively represented by the sample labels; then the final feature F corresponding to a single picture is subjected to an MLP network to obtain a final predicted bone age Y, and then the MSE loss function L MSE is calculated between the bone age label and the bone age Y:

[0115] Y = MLP (F) (18)

[0116] L MSE = || Y-Y l || 2 (19)

[0117] wherein the MLP network is composed of two separate fully connected layers; Y l represents the bone age label corresponding to the hand bone picture.

[0118] In the embodiment, the step S4 is specifically as follows:

[0119] The model is trained by the loss function, and the total loss function L is expressed as follows:

[0120] L = lambda L triplet + L MSE (20)

[0121] wherein lambda represents the proportion of the adaptive triplet loss in the total loss function, and in the embodiment, the value of lambda is set to 0.2.

[0122] After the model training is completed, the 1024-dimensional high-dimensional features D corresponding to the original picture are extracted from the last MLP layer 1024 :

[0123] D 1024 = Extract (MLP (F)) (21)

[0124] wherein Extract represents removing a certain part of the model, directly outputting, saving the result, and sending it to the subsequent algorithm.

[0125] In the embodiment, the step S5 is specifically as follows:

[0126] The cluster expression obtained by the K-means clustering algorithm is as follows:

[0127] G1, G2,..., G g = KMEANS ({D 1024}) (22)

[0128] wherein {D 1024} represents the set of 1024-dimensional features of the data set, KMEANS represents the K-means clustering algorithm, and G1, G2,..., G g represent the clusters obtained by the K-means clustering algorithm, which means that the original data set is divided according to each age group; in the embodiment, g = 5 is taken; actually, since the data dimension is 1024, it will bring a large overhead to the K-means clustering algorithm, because the data dimension determines the calculation and clustering time of the algorithm, so in order to reasonably reduce the time consumption of this step, T-SNE dimension reduction is used in the operation process, and then the implementation of the clustering algorithm is performed, the process of T-SNE dimension reduction is as follows:

[0129]

[0130]

[0131]

[0132] wherein S(x i , x j ) is the similarity of the data i and the data j, and it is considered that the closer the distance between two data representations, the higher the similarity; P(i|j) is defined as the probability distribution of the original data according to formula (23), and the probability distribution of the data after dimension reduction is defined according to formula (24); then the loss function is defined according to the KL divergence as formula (25), the purpose is to find a set of low-dimensional data z1, z2,..., z n so that the loss function reaches the minimum; under this method, the data is processed by dimension reduction.

[0133] In the present embodiment, the step S6 is specifically as follows:

[0134] The bone age prediction value expression of each age group is as follows:

[0135] BoneAge = BoGFF g (G g ) (26)

[0136] Wherein, BoneAge represents the final prediction age, which is obtained by each age group data passing through the BoGFF regression network adapted to the age group, BoGFF g represents the gth regression model BoGFF-Net, each BoGFF-Net model structure is the same, and the maximum value of g is 5.

[0137] In step S5, the data is clustered into five categories, so there are five groups of data here, which are G1, G2, G3, G4 and G5, respectively, corresponding to five neural networks BoGFF1, BoGFF2, BoGFF3, BoGFF4 and BoGFF5 with the same structure but different parameters; In particular, in the training of the network in this stage, the network weight trained in step S4 is imported as the initial weight, and network migration and optimization are performed on this basis; The final prediction age BoneAge is obtained by each age group data passing through the BoGFF regression network adapted to the age group.

[0138] In the present embodiment, the hand bone image size is 500x500, and the images are respectively from the RSNA and DHA data sets; In the present embodiment, the model parameter settings are as follows: the first stage e ochs = 400, lr = 1e-5, b tchsize = 16, FE = ResNet-34; The second stage e ochs = 50, lr = 1e-5, b tchsize = 8, FE = ResNet-18. After steps S1, S2, S3, S4 and S5 (coarse regression stage), step S6 is entered, and steps S1, S2, S3 and S4 are repeated in S6 (fine regression stage) to obtain the final result.

[0139] All the above formulas and examples are described and displayed with the whole hand bone image as an example, and the actual calculation is performed on each pixel on the image to realize multiplication, weighting and other operations.

[0140] To sum up, the method of the present application relates to regression problems and a Transformer network architecture, and also relates to a specific solution for classifying and segmenting actual problems from coarse to fine. A general two-stage algorithm framework from coarse to fine is adopted, the first stage is divided according to age, and the second stage is bone age regression in the specific age stage. The framework can adapt to most regression problems, break through the original technical framework, abandon the high labor consumption brought by the target detection method, obtain the BoGFF-Net regression model by fusing multi-scale global information, and solve the problems of heterogeneity of each age stage of hand bones and difficulty of a single regression model to fit all data, so as to obtain a general and simple and efficient bone age regression model and framework from coarse to fine, provide a general solution for subsequent bone age regression and other regression problems, and effectively perform left hand bone regression detection of adolescents, which is suitable for clinical computer bone age regression application scenarios.

[0141] Those skilled in the art will appreciate that the embodiments described herein are presented for purposes of illustration and understanding of the principles of the present application and are not intended to be exhaustive or to limit the present application to the precise steps and / or forms described. Various modifications and variations can be made to the disclosed embodiments without departing from the spirit or scope of the present application.

Claims

1. A coarse-to-fine bone age regression method based on multi-scale high-order self-attention, the specific steps of which are as follows: S1. Input the original hand bone image with real bone age label into the backbone network to extract multi-scale features and multi-scale global self-attention response map of the original hand bone image. S2. The multi-scale global self-attention response map and the downsampled final features of the original hand bone image are fused to obtain the multi-scale fused features, and then input into the loss function calculation and regression module. S3. Calculate the adaptive triplet loss in the regression module, and obtain the final bone age regression value and calculate the MSE loss of bone age through two fully connected layers. S4. Adjust the model parameters and train the model on the original dataset; After training is complete, the original dataset is input into the backbone network, and high-dimensional representations of the images are extracted from the final two fully connected layers. S5. Feed the high-dimensional representation of the original dataset into the K-means clustering algorithm and use an unsupervised approach to cluster the dataset into multiple age groups. S6. Input the data of each age group into multiple raw backbone networks, repeat steps S1 to S3, train multiple raw backbone networks, and finally obtain the bone age prediction value of each age group. The specific steps of S1 are as follows: The extracted multi-scale features X1, X2, X3, X4 and multi-scale global self-attention response maps C1, C2, C3, C4 from the original hand bone image are as follows: X i+1 =Stages(X i ) (1) Where Stages represents stacked convolutional neural network layers and batch normalization layers, X i Represents the scale feature map of the i-th level, i = 1, 2, 3, 4; Hierarchical feature X i The query matrix Q is obtained after the block embedding layer PE. i Key matrix K i Sum matrix V i The expression is as follows: Q i =PE Q (X i ) (2) K i =PE K (X i ) (3) V i =PE V (X i ) (4) Where PE represents PatchEmbedding, which is composed of fully connected layers; the block embedding size follows: P N =P1 / 2 N-1 (5) Where P represents an image patch, P1 has a value of 16×16 pixels, and N represents the level in the global feature modeling module, N = 1, 2, 3, 4; Then use the obtained query matrix Q i Key matrix K i Sum matrix V i Calculate a single higher-order self-attention head: in, This indicates that three linear transformation matrices with the same dimension are calculated separately during the self-attention process, H. u The result of a single higher-order self-attention head is represented in this work, where u = 1, 2, ..., 8; A is defined by the following formula: in,() T This represents the matrix transpose operation, and The normalization scale is a constant; Softmax represents the commonly used maximum transfer function in multi-class classification problems. The calculated individual high-order self-attention heads are then concatenated in arbitrary spatial dimensions to obtain the global feature fusion feature C at this level. i : C i =Concatate(H1,H2,…,H u ) (8) Concatate represents a spatial concatenation operation on the feature maps.

2. The coarse-to-fine bone age regression method based on multi-scale high-order self-attention according to claim 1, characterized in that, Step S2 is as follows: The expression for the multi-scale fusion feature C is as follows: Here, Concatate represents the spatial concatenation operation on the feature maps; FPN represents a feedforward neural network, consisting of ordinary fully connected layers and normalization layers; G represents the gender label of the original input image, and This represents the addition of the numerical values ​​of corresponding elements in the matrix; The obtained multi-scale global fusion features are passed through a global average pooling layer (GAP) to obtain the final feature F corresponding to an image: F = GAP(C) (10).

3. The coarse-to-fine bone age regression method based on multi-scale high-order self-attention according to claim 1, characterized in that, Step S3 is as follows: For a batch of data, arbitrarily select one image as the anchor point, and then arbitrarily select two other images, one as the far point and the other as the near point, to calculate the adaptive triplet loss, as shown in the following expression: L triplet =∥F a -F n ∥ 2 - ∥F a -F f ∥ 2 +αm (11) Among them, L triplet F represents the adaptive triplet loss. a F represents the representation of the anchor point image. n F represents the representation of a near-point image. f The image represents the far-point image, where m represents the edge constant and α represents the adaptive coefficient, determined by the following formula: α=∥y a -and n ∥- ∥y a -and f ∥ (12) Among them, y a y n y f Let F represent the anchor label age, near-point label age, and far-point label age, respectively. Then, the final feature F corresponding to a single image is processed through an MLP network to obtain the final predicted bone age Y, which is then compared with the bone age label to calculate the MSE loss function L. MSE : Y = MLP(F) (13) L MSE <∥YY l ∥ 2 (14) The MLP network consists of two separate fully connected layers; Y l This indicates the bone age label corresponding to the hand bone image.

4. The coarse-to-fine bone age regression method based on multi-scale high-order self-attention according to claim 1, characterized in that, Step S4 is as follows: The model is trained using a loss function, and the overall loss function L is expressed as follows: L=λL triplet +L MSE (15) Where λ represents the proportion of the adaptive triplet loss in the total loss function; After the model training is complete, the 1024-dimensional high-dimensional features D corresponding to the original image are extracted from the final MLP layer. 1024 : D 1024 =Extract(MLP(F)) (16) Extract means removing a part of the model and outputting it directly.

5. The coarse-to-fine bone age regression method based on multi-scale high-order self-attention according to claim 1, characterized in that, Step S5 is as follows: The clustering expression obtained by the K-means clustering algorithm is as follows: G1,G2,…,G g =KMEANS({D 1024 }) (17) Among them, {D 1024 } represents the set of 1024-dimensional features in the dataset, KMEANS represents the K-means clustering algorithm, and G1, G2, ..., G g This represents the clustering categories obtained using the K-means clustering algorithm, with g fixed at 5.

6. The coarse-to-fine bone age regression method based on multi-scale high-order self-attention according to claim 1, characterized in that, Step S6 is as follows: The expressions for bone age prediction values ​​for each age group are as follows: BoneAge=BoGFF g (G g ) (18) Where BoneAge represents the final predicted age, BoGFF g Let g represent the g-th regression model.

Images