Bone age detection method, system and equipment based on improved Mask R-CNN model, medium and product

Abstract

The invention discloses a bone age detection method, system and device based on an improved Mask R-CNN model, a medium and a product, and relates to the field of medical image processing, and the method comprises the steps: obtaining a human palm normal position X-ray medical image; an improved Mask R-CNN model is constructed and trained; based on the human palm normal position X-ray medical image, a trained improved Mask R-CNN model is adopted to output the grade of each characteristic epiphysis; and calculating the bone age according to the grade of each characteristic epiphysis. The method can improve the bone age evaluation precision, and is more suitable for the evaluation of the bone age of Chinese children.

Description

Technical Field

[0001] The present application relates to the field of medical image processing, and in particular to a bone age detection method, system, equipment, medium and product based on an improved Mask R-CNN model. Background Art

[0002] The existing neural network-based bone age assessment algorithm has the following problems in processing:

[0003] (1) The training data was annotated with the TW3 standard, which is not suitable for bone age assessment of Chinese children;

[0004] (2) The feature extraction of the palm image as a whole only provides the bone age value, but not the epiphyseal development level value. The result cannot be explained or verified.

[0005] (3) The attention mechanism added by the neural network is a black box mechanism. It is impossible to determine whether the attention mechanism is correctly superimposed on the part of interest. It is also unknown whether it is interfered by the image outside the palm. It is easy to extract interference features, thus affecting the evaluation accuracy and reliability.

[0006] (4) Multiple neural networks are required to work together. First, the image contour must be segmented to determine the overall area of ​​the palm, and then the overall feature extraction can be performed. Alternatively, the overall palm area must be determined first, and then each diaphysis and epiphyseal sub-area must be segmented, and then feature extraction must be performed on each sub-area before the bone age can be determined. The overall efficiency is low. Summary of the invention

[0007] In order to solve the above problems, the present application provides a bone age detection method, system, equipment, medium and product based on an improved Mask R-CNN model.

[0008] To achieve the above objectives, this application provides the following solutions:

[0009] In a first aspect, the present application provides a bone age detection method based on an improved Mask R-CNN model, comprising:

[0010] Acquire X-ray medical images of human palm in anteroposterior position;

[0011] Constructing and training an improved Mask R-CNN model; the improved Mask R-CNN model includes a feature extraction backbone network, a feature pyramid network, a region proposal network, a classification branch, a positioning branch, a segmentation branch, and a level branch; the improved Mask R-CNN model is trained by sample images annotated using the Zhonghua 05 standard;

[0012] Based on the human palm X-ray medical image, the trained improved Mask R-CNN model is used to output the grade of each characteristic epiphysis;

[0013] Bone age was calculated based on the grade of each characteristic epiphysis.

[0014] In the second aspect, the present application provides a bone age detection system based on an improved Mask R-CNN model, comprising:

[0015] A medical image acquisition module, used for acquiring an X-ray medical image of a human palm in an upright position;

[0016] A model building and training module, used to build and train an improved Mask R-CNN model; the improved Mask R-CNN model includes a feature extraction backbone network, a feature pyramid network, a region proposal network, a classification branch, a positioning branch, a segmentation branch, and a level branch; the improved Mask R-CNN model is trained by sample images annotated with the Zhonghua 05 standard;

[0017] A characteristic epiphysis grade determination module is used to output the grade of each characteristic epiphysis based on the human palm frontal X-ray medical image using the trained improved Mask R-CNN model;

[0018] The bone age calculation module is used to calculate the bone age according to the grade of each characteristic epiphysis.

[0019] In a third aspect, the present application provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-mentioned bone age detection method based on the improved Mask R-CNN model.

[0020] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned bone age detection method based on the improved Mask R-CNN model.

[0021] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above-mentioned bone age detection method based on the improved Mask R-CNN model.

[0022] According to the specific embodiments provided in this application, this application has the following technical effects:

[0023] The present application provides a bone age detection method, system, device, medium and product based on an improved Mask R-CNN model. By selecting the Chinese 05 standard (not limited to this standard) to annotate sample images and then training the improved Mask R-CNN model, the trained model can eliminate the defects pointed out in the background technology, so that the epiphyseal interest region output by the model corresponds one-to-one with each epiphyseal feature, becoming an accurate visualized area of ​​interest, so that the grade of each characteristic epiphysis can be interpreted according to the Chinese 05 standard grade, and then the bone age is calculated according to the grade of each characteristic epiphysis, which helps to reduce evaluation errors, improve the accuracy of bone age assessment, and make it conform to the bone age assessment of Chinese children. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 A flowchart of a bone age detection method based on an improved Mask R-CNN model provided in one embodiment of the present application;

[0026] Figure 2 A detailed flowchart of a bone age detection method based on an improved Mask R-CNN model provided in one embodiment of the present application;

[0027] Figure 3 This is a schematic diagram of the structure of the improved Mask R-CNN model;

[0028] Figure 4 Schematic diagram of hierarchical branch structure

[0029] Figure 5 Provide a schematic diagram of the level annotation interface;

[0030] Figure 6 This is a schematic diagram of the training data in JSON format obtained by annotation;

[0031] Figure 7 Schematic diagram of multi-classification labeled training data;

[0032] Figure 8 Schematic diagram of the loss function and verification output of the first training of the improved Mask R-CNN model;

[0033] Fig. 9 Schematic diagram of the loss function and verification output of the second training of the improved Mask R-CNN model

[0034] Fig.10 This is a comparison chart of the accuracy of the improved Mask R-CNN model test and the public neural network test, where (a) is the accuracy of the improved Mask R-CNN model test, and (b) is the accuracy of the public neural network test;

[0035] Fig.11 This is a schematic diagram of the characteristic epiphysis identification results;

[0036] Fig.12 This is a schematic diagram of the bone age assessment report;

[0037] Fig.13 Schematic diagram of the RUS-CHN bone maturity percentile standard curve (male);

[0038] Fig.14 Schematic diagram of the RUS-CHN bone maturity percentile standard curve (female);

[0039] Fig.15 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0041] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0042] In an exemplary embodiment, Figure 1-Figure 2 As shown, a bone age detection method based on an improved Mask R-CNN model is provided, which is executed by a computer device, specifically, it can be executed by a computer device such as a terminal or a server alone, or it can be executed by a terminal and a server together. In the embodiment of the present application, the method is applied to a server as an example for explanation, including the following steps S1 to S4. Among them:

[0043] S1: Acquire an X-ray medical image of a human palm in the frontal position.

[0044] Specifically, an X-ray medical image of the human palm in the frontal position is obtained by using appropriate medical equipment (such as an X-ray machine, in dcm format), a film taken by a camera (in jpg, png, etc. format), and the like.

[0045] Based on the original image, the image is randomly rotated, scaled and filled, and random noise is superimposed.

[0046] S2: Build and train an improved Mask R-CNN model. Figure 3 As shown, the improved Mask R-CNN model includes a feature extraction backbone network (ResNet convolutional network), a feature pyramid network, a region proposal network, a classification branch, a positioning branch, a segmentation branch and a level branch; the improved Mask R-CNN model is trained by sample images annotated using the Chinese 05 standard.

[0047] Specifically, sample images were obtained from a public bone age image dataset (png format), and image data were annotated using self-made software that complies with the Chinese 05 standard, such as Figure 5 shown.

[0048] In a specific embodiment, the sample images are normalized in size and pixel value using homemade software to ensure that all images have a uniform size and pixel range and the same dcm format. 28 types of epiphyses (radius, ulna, metacarpal I, metacarpal II, metacarpal III, metacarpal IV, metacarpal V, proximal phalanx I, proximal phalanx II, proximal phalanx III, proximal phalanx IV, proximal phalanx V, middle phalanx II, middle phalanx III, middle phalanx IV, middle phalanx V, distal phalanx I, distal phalanx II, distal phalanx III, distal phalanx IV, distal phalanx V, capitate, hamate, triangular bone, lunate, scaphoid, trapezium and trapezium) are labeled, including classification number (subsequently extracting 13 types of epiphyses required by the Chinese 05 bone age standard according to the number), epiphyseal and nearby diaphysis outline coordinate point set, the minimum circumscribed rectangle coordinate of the outline, and epiphyseal grade. And store the annotation data in json format (such as Figure 6 ) and multi-classification labeled data (such as Figure 7 The labeled sample images are divided into training set, validation set and test set according to the ratio of 70%, 15% and 15% to ensure reasonable data distribution and avoid overfitting.

[0049] Because it involves multiple loss functions, the loss weight of the improved Mask R-CNN model focuses on the localization branch, classification branch, and segmentation branch in the first training. The loss weight of the level branch is 1 / 10 of the loss weight of other branches. After the second training, the loss weight of the level branch is increased to make the loss weight ratio between them equal. The loss function and verification output of the first and second training of the improved Mask R-CNN model are shown in the figure. Figure 8-Figure 9 The accuracy comparison test results of the improved Mask R-CNN model and a currently public neural network on the same test set are shown in Figure 2. Fig.10As shown in (a) and (b) in .

[0050] S3: Based on the human palm frontal X-ray medical image, the trained improved Mask R-CNN model is used to output the grade of each characteristic epiphysis.

[0051] After training, the improved Mask R-CNN model can perform neural recognition on the input human palm X-ray medical images, and finally output the grade information of 28 types of epiphyses through the grade branch. Then, according to the information output by the segmentation branch, the classification branch, and the positioning branch, a rectangular frame of the epiphyseal area of ​​interest, the characteristic epiphyseal classification name, the characteristic epiphyseal grade, and the background mask information before and after the characteristic epiphyseal area are drawn on the human palm X-ray medical images. Then, 13 types of characteristic epiphyses (radius, ulna, metacarpal I, metacarpal III, metacarpal V, proximal phalanx I, proximal phalanx III, proximal phalanx V, middle phalanx III, middle phalanx V, distal phalanx I, distal phalanx III, and distal phalanx V) are selected according to the classification number, as shown in the figure. Fig.11 shown.

[0052] Step S3 specifically includes the following steps S31 to S37:

[0053] S31: Inputting the human palm frontal X-ray medical image into the feature extraction backbone network, and outputting feature maps of different scales.

[0054] The input image (i.e., X-ray medical image of human palm in front view) first passes through a 7×7 convolution layer with a step size of 2, and the output feature map size is halved. Then it passes through a 3×3 maximum pooling layer with a step size of 2 to further reduce the size of the feature map. Features are gradually extracted through multiple residual modules (conv_block and identity_block) to form feature maps of different scales.

[0055] The feature extraction backbone network is composed of multiple residual modules (the first residual module, the second residual module, the third residual module, the fourth residual module and the fifth residual module). Modal-section encoding information is added to the feature map before each residual module to enhance the model's adaptability to different sections.

[0056] The first residual module C1: includes 1 convolution layer and a maximum pooling layer, and the output feature map size is 1 / 4 of the input image.

[0057] The second residual module C2: includes 1 conv_block and 2 identity_blocks, and the output feature map size is 1 / 4 of the input image.

[0058] The third residual module C3: includes 1 conv_block and 3 identity_blocks, and the output feature map size is 1 / 8 of the input image.

[0059] The fourth residual module C4: includes 1 conv_block and 22 identity_blocks, and the output feature map size is 1 / 16 of the input image.

[0060] The fifth residual module C5: includes 1 conv_block and 2 identity_blocks, and the output feature map size is 1 / 32 of the input image.

[0061] S32: Inputting feature maps of different scales into the feature pyramid network, and outputting multi-scale feature maps.

[0062] The feature maps of different scales are downsampled and dot products with the modality-section encoding information by channel to construct a multi-scale feature map. The multi-scale feature maps are concatenated after scale matching, and convolution is used to reduce the channel dimension, so as to fuse and form multi-scale features.

[0063] S33: Input the multi-scale feature map into the region proposal network, and output multiple candidate regions.

[0064] Input: Multi-scale feature maps generated by feature pyramid network.

[0065] Sliding window and anchor generation: At each position of the multi-scale feature map, RPN (Region Proposal Network) uses a sliding window to scan the multi-scale feature map. This sliding window moves with a fixed step size across the entire feature map, similar to a traditional convolution operation. For each sliding window position, a set of anchors are pre-set. Anchors are predefined bounding boxes of different sizes (8, 16, 32, 64, 128) and scales (0.5, 1, 2), which cover different parts of the image and are used as initial guesses for potential target objects. Typically, these anchors are generated at each pixel position on the feature map and have different configurations for different scales and aspect ratios.

[0066] Convolutional layer: RPN first uses a 3x3 convolutional layer to integrate the multi-scale feature maps, with the activation function being ReLU. This step aims to extract more abstract feature representations from the area covered by the sliding window.

[0067] Initial classification branch: Next, a 1x1 convolutional layer is used to perform initial classification on the integrated feature map, and the output shape is [batch, height, width, num_anchors_per_location, 2], where height and width are the spatial dimensions of the feature map, num_anchors_per_location refers to the number of anchor points generated at each spatial location, and 2 represents binary classification (i.e., whether the epiphysis is included). This layer is used to predict the probability of each anchor point belonging to the foreground (containing the target) or the background. After the Reshape layer, the Softmax activation function is used to calculate the classification probability, and finally the probability distribution of whether each anchor point contains the target object is obtained.

[0068] Initial regression branch: At the same time, another 1x1 convolutional layer is used to regress the integrated feature map, and the output shape is [batch, height, width, num_anchors_per_location, 4]. These four values ​​correspond to the bounding box adjustment parameters (dx, dy, dw, dh) relative to the original anchor point, which are used to accurately locate the position and size of the candidate area. The Reshape layer also merges the spatial dimensions to provide a set of adjustment parameters for each anchor point.

[0069] Output: RPN generates a large number of candidate regions (RegionProposals), each of which contains an initial classification score and the coordinates of a bounding box.

[0070] S34: Screening the candidate regions to generate an epiphyseal region of interest, and aligning feature maps in the epiphyseal region of interest.

[0071] Input: Proposal regions generated by RPN.

[0072] Non-Maximum Suppression (NMS): First, non-maximum suppression is performed on the candidate regions generated by RPN to remove candidate regions with high overlap (IoU, which is expressed as the ratio of the area of ​​the intersection between the predicted box Predicted Box and the ground truth box to the area of ​​their union, set here to 0.7), and retain candidate regions with higher scores.

[0073] Threshold screening: Set a classification score threshold and only retain candidate regions with scores higher than the threshold.

[0074] Positive and negative sample control during training: In order to ensure the quality of training data, the model will try to maintain a specific ratio to select positive samples (ROIs with high overlap with the real target) and negative samples (ROIs with background or low overlap). Here, the positive sample ratio is set to 0.33.

[0075] Quantity limit: In order to reduce the amount of calculation, a fixed number of candidate regions is usually set. During training, the 200 candidate regions with the highest scores are retained; during inference, the 100 candidate regions with the highest scores are retained.

[0076] Size screening: exclude candidate regions that are too small or too large to ensure that the size of the candidate region is within the reasonable feature image range.

[0077] ROIAlign: Input includes multi-scale feature maps (usually from feature pyramid network FPN) and a set of candidate boxes (RegionProposals). Each candidate box defines the location and size of a region of interest.

[0078] Grid division: For each candidate box, divide it into a fixed k×k grid (7x7). This means that no matter what the actual size of the candidate box is, it will be mapped into a grid of the same size. This standardization helps the consistency and efficiency of subsequent processing.

[0079] Determine sampling points: In each grid cell, the exact positions of the four corner points on the original feature map are calculated based on the real coordinates of the candidate box. These corner points do not necessarily fall exactly on integer coordinates, so interpolation is required to obtain their corresponding eigenvalues. To improve accuracy, some additional sampling points (such as 4 or more) are evenly distributed inside each grid cell, and the exact position of each sampling point in the original feature map is found.

[0080] Bilinear interpolation: For each sampling point, bilinear interpolation is used to extract feature values ​​from the original feature map. Specifically, the four pixels closest to the point are found and the final feature value is obtained by weighted average based on their distance from the sampling point.

[0081] The bilinear interpolation formula is as follows:

[0082] V=w1·p1+w2·p2+w3·p3+w4·p4

[0083] Among them, V is the eigenvalue of the sampling point, pi is the eigenvalue of the four surrounding pixels, and wi is the weight, which is determined by the distance from the sampling point to these pixels.

[0084] Output: Aggregate the feature values ​​of all sampling points to form an output feature map of fixed size. For each grid unit, you can simply take the average of the feature values ​​of all sampling points as the feature representation of the unit. In this way, even if the original position and size of the candidate box are different, after ROIAlign, you can get a feature representation in a unified format, which is convenient for subsequent classification, regression and other operations.

[0085] S35: Input the aligned feature map into the positioning branch and the classification branch to identify and locate 28 types of epiphyses.

[0086] Input: Aligned feature map

[0087] Classification branch: Use a fully connected layer to classify the aligned feature map, and the output shape is [batch, num_rois, num_classes], which represents the epiphysis category in each ROI. After the Softmax activation function, the output is [batch, num_rois, num_classes]. The last dimension data represents the confidence, that is, the probability, of belonging to the category. Here batch represents the number of samples, num_rois represents the number of regions of interest, and num_classes represents the number of categories, 13 epiphysis categories and 1 fixed background category, for a total of 14 categories (categories and regions of interest can be one-to-many).

[0088] Positioning branch: Use another fully connected layer to position the aligned feature map, and the output shape is [batch, num_rois, num_classes*4], which represents the bounding box adjustment parameters of each ROI. After the Reshape layer, the output is [batch, num_rois, num_classes, 4]. Here, one dimension is increased to generate 4 data representing the ratio of the upper left corner x, y and lower right corner coordinates x, y to the width and height of the original image after linear regression, so as to facilitate the subsequent loss calculation and restore the position in the original image.

[0089] Output: Generates classification results and bounding box adjustment parameters for each ROI.

[0090] S36: Input the aligned feature map into the segmentation branch and the grade branch to segment and grade the 28 types of epiphyses.

[0091] (1) Split branch:

[0092] Input: Aligned feature maps.

[0093] Feature integration: Use multiple convolutional layers to integrate the aligned feature maps. Each convolutional layer is followed by a batch normalization layer and an activation function (ReLU).

[0094] Upsampling: Use the transposed convolution layer (Deconvolution) for upsampling to restore the resolution of the feature map.

[0095] Segmentation head: Use a 1x1 convolutional layer for segmentation, and the output shape is [batch, num_rois, MASK_POOL_SIZE*2, MASK_POOL_SIZE*2, num_classes], which indicates the category of each pixel. After the Sigmoid activation function, the output is [batch, num_rois, MASK_POOL_SIZE*2, MASK_POOL_SIZE*2, num_classes]. Here MASK_POOL_SIZE is defined as 14. The last dimension represents the probability that the pixel belongs to non-background.

[0096] Output: Segmentation mask for each ROI.

[0097] (2) Hierarchical branches Figure 3 Middle red part):

[0098] like Figure 4 As shown in the figure, the hierarchical branch implements the hierarchical classification and regression of the feature epiphysis by performing FCN full convolution layer, Flatten feature dimensionality reduction layer, Linear full connection layer and Reshape dimensionality increase layer on the feature layer.

[0099] Input: Aligned feature maps.

[0100] Feature integration: After the segmentation head, an FCN fully convolutional layer is used to further integrate the feature maps.

[0101] Feature dimensionality reduction: Use the Flatten feature dimensionality reduction layer to flatten the feature map and reduce the high-dimensional features to a one-dimensional vector.

[0102] Fully connected layer: Use one or more Linear fully connected layers to perform linear transformation on the flattened feature vector to extract higher-level features.

[0103] Grade classification: A fully connected layer is used to output the probability distribution of the characteristic epiphyseal grade. After a linear activation function, the grade classification result of each ROI is output.

[0104] Dimensionality: Use the Reshape layer to reorganize the output of the fully connected layer into the shape of [batch, num_rois, num_stages, 1]. The purpose of this is to ensure that the predicted level value of each ROI can be compared with the target level value element by element.

[0105] The loss of the level branch during model training (the loss between the predicted level value and the target level value) is calculated as follows:

[0106] enter:

[0107] target_stage: The shape is [batch,num_rois,num_classes,1], which indicates the actual grade value of each epiphyseal classification in each ROI.

[0108] pred_stage: The shape is [batch, num_rois, num_classes], which indicates the predicted classification level of each epiphysis for each ROI.

[0109] target_class_ids: The shape is [batch, num_rois], which indicates the target category ID of each ROI.

[0110] Convert type and Reshape:

[0111] Convert target_stage to floating point type.

[0112] Flatten target_class_ids into a one-dimensional vector [-1,].

[0113] Flatten the target_stage into a one-dimensional vector [-1,1].

[0114] Flatten pred_stage into a two-dimensional vector [-1, num_stages, 1]. num_stages represents the maximum number of stages, which is 15 for the radial epiphysis.

[0115] Screening positive samples:

[0116] Use the tf.where function to get the index of all positive samples, that is, non-background ROIs whose target_class_ids is greater than 0.

[0117] Use the tf.gather function to extract the target_class_ids and target_stage of the positive samples.

[0118] Use the tf.stack function to build the index and extract the pred_stage of the positive sample.

[0119] Calculate the loss:

[0120] Using Smooth-L1Loss as the loss function, calculate the difference between this output level target_stage and the annotation level pred_stage.

[0121] If there are no positive samples, the loss is 0.

[0122] The losses of all positive samples are averaged to get the final loss value.

[0123] Use tf.expand_dims to expand the loss value into a one-dimensional vector [1,] so that it can be merged with other loss items. Take the loss weight of each branch according to the configured loss weight for comprehensive loss feedback.

[0124] Finally, the network uses non-maximum suppression (NMS) again during inference, that is, ROIs with an IoU less than 0.3 are not retained. The confidence DECTION_MIN_CONFIDENCE=0.7 is set so that only those with a confidence greater than 0.7 are used, and DETECTION_MAX_INSTANCES=28 is set to limit the total number of ROIs to 28.

[0125] S37: According to the recognition and positioning results, segmentation results and grade confirmation results of the 28 types of epiphyses, the grades of 13 types of characteristic epiphyses are extracted.

[0126] S4: Calculate bone age based on the grade of each characteristic epiphysis.

[0127] According to the grade table of each characteristic epiphysis (Table 1 or Table 2), the grade score is obtained and summed up, and then according to Figure 13-14 The bone age is determined by using the RUS-CHN bone maturity percentile standard curve (50th) shown in the figure, and finally a bone age assessment report is generated and printed, as shown in the figure. Fig.12 shown.

[0128] Table 1 RUS-CHN bone development grade score table (male)

[0129]

[0130] Table 2 RUS-CHN bone development grade score table (female)

[0131]

[0132] Based on the same inventive concept, the embodiment of the present application also provides a system for implementing the above-mentioned bone age detection method based on the improved Mask R-CNN model. The implementation scheme for solving the problem provided by the system is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more embodiments of the bone age detection system based on the improved Mask R-CNN model provided below can be found in the above limitations on the bone age detection method based on the improved Mask R-CNN model, which will not be repeated here.

[0133] In an exemplary embodiment, a bone age detection system based on an improved Mask R-CNN model is provided, comprising:

[0134] The medical image acquisition module is used to acquire X-ray medical images of the human palm in the frontal position.

[0135] The model building and training module is used to build and train an improved Mask R-CNN model; the improved Mask R-CNN model includes a feature extraction backbone network, a feature pyramid network, a region proposal network, a classification branch, a positioning branch, a segmentation branch and a level branch; the improved Mask R-CNN model is trained by sample images annotated with the Zhonghua 05 standard.

[0136] The characteristic epiphysis grade determination module is used to output the grade of each characteristic epiphysis based on the human palm frontal X-ray medical image using the trained improved Mask R-CNN model.

[0137] The bone age calculation module is used to calculate the bone age according to the grade of each characteristic epiphysis.

[0138] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program. The computer device may be a server or a terminal, and its internal structure diagram may be as shown in FIG. Fig.15As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data to be processed. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a bone age detection method based on an improved Mask R-CNN model is implemented.

[0139] Those skilled in the art will understand that Fig.15 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components. In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0140] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0141] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0142] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0143] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0144] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a reduced instruction processor, a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, etc., but is not limited thereto.

[0145] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0146] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A bone age detection method based on an improved Mask R-CNN model, characterized in that: include: Acquire X-ray medical images of human palm in anteroposterior position; Constructing and training an improved Mask R-CNN model; the improved Mask R-CNN model includes a feature extraction backbone network, a feature pyramid network, a region proposal network, a classification branch, a positioning branch, a segmentation branch, and a level branch; the improved Mask R-CNN model is trained by sample images annotated using the Zhonghua 05 standard; Based on the human palm X-ray medical image, the trained improved Mask R-CNN model is used to output the grade of each characteristic epiphysis; Bone age was calculated based on the grade of each characteristic epiphysis.

2. The bone age detection method based on the improved Mask R-CNN model according to claim 1, characterized in that: Based on the human palm X-ray medical image, the trained improved Mask R-CNN model is used to output the characteristic epiphyseal grades, including: Inputting the human palm X-ray medical image into the feature extraction backbone network, and outputting feature maps of different scales; Inputting feature maps of different scales into the feature pyramid network and outputting multi-scale feature maps; Inputting the multi-scale feature map into the region proposal network, and outputting a plurality of candidate regions; Screening the candidate regions to generate an epiphyseal region of interest, and aligning feature maps in the epiphyseal region of interest; The aligned feature maps are input into the localization branch and the classification branch to identify and locate 28 types of epiphyses; The aligned feature maps are input into the segmentation branch and the grade branch to segment and grade the 28 types of epiphyses; According to the recognition, positioning, segmentation and grade confirmation results of 28 types of epiphyses, the grades of 13 types of characteristic epiphyses were extracted.

3. The bone age detection method based on the improved Mask R-CNN model according to claim 1, characterized in that: The feature extraction backbone network is composed of multiple residual modules.

4. The bone age detection method based on the improved Mask R-CNN model according to claim 3, characterized in that: The residual module includes a first residual module, a second residual module, a third residual module, a fourth residual module and a fifth residual module which are connected in sequence; The first residual module includes a convolution layer and a maximum pooling layer, and the size of the feature map output by the first residual module is 1 / 4 of the X-ray medical image of the human palm in the frontal position; The second residual module includes 1 conv_block and 2 identity_blocks, and the size of the feature map output by the second residual module is 1 / 4 of the X-ray medical image of the human palm in the frontal position; The third residual module includes 1 conv_block and 3 identity_blocks, and the size of the feature map output by the third residual module is 1 / 8 of the X-ray medical image of the human palm in the frontal position; The fourth residual module includes 1 conv_block and 22 identity_blocks, and the size of the feature map output by the fourth residual module is 1 / 16 of the X-ray medical image of the human palm in the front position; The fifth residual module includes 1 conv_block and 2 identity_blocks, and the size of the feature map output by the fifth residual module is 1 / 32 of the X-ray medical image of the human palm in the frontal position.

5. The bone age detection method based on the improved Mask R-CNN model according to claim 2, characterized in that: The 28 types of epiphyses include: radius, ulna, metacarpal I, metacarpal II, metacarpal III, metacarpal IV, metacarpal V, proximal phalanx I, proximal phalanx II, proximal phalanx III, proximal phalanx IV, proximal phalanx V, middle phalanx II, middle phalanx III, middle phalanx IV, middle phalanx V, distal phalanx I, distal phalanx II, distal phalanx III, distal phalanx IV, distal phalanx V, capitate, hamate, triquetrum, lunate, scaphoid, trapezium and trapezium; The 13 types of characteristic epiphyses include: radius, ulna, metacarpal I, metacarpal III, metacarpal V, proximal phalanx I, proximal phalanx III, proximal phalanx V, middle phalanx III, middle phalanx V, distal phalanx I, distal phalanx III and distal phalanx V.

6. The bone age detection method based on the improved Mask R-CNN model according to claim 1, characterized in that: When the improved Mask R-CNN model is trained for the first time, the loss weight of the hierarchical branch is 1 / 10 of the loss weight of other branches, and the other branches include a positioning branch, a classification branch, and a segmentation branch; After the second training, the loss weight of the level branch is increased so that the loss weight ratios of the level branch, the positioning branch, the classification branch and the segmentation branch are equal.

7. A bone age detection system based on an improved Mask R-CNN model, characterized in that: include: A medical image acquisition module, used to acquire an X-ray medical image of a human palm in an upright position; A model building and training module, used to build and train an improved Mask R-CNN model; the improved Mask R-CNN model includes a feature extraction backbone network, a feature pyramid network, a region proposal network, a classification branch, a positioning branch, a segmentation branch, and a level branch; the improved Mask R-CNN model is trained by sample images annotated using the Zhonghua 05 standard; A characteristic epiphysis grade determination module is used to output the grade of each characteristic epiphysis based on the human palm frontal X-ray medical image using the trained improved Mask R-CNN model; The bone age calculation module is used to calculate the bone age according to the grade of each characteristic epiphysis.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the bone age detection method based on the improved Mask R-CNN model as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the bone age detection method based on the improved Mask R-CNN model described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, it implements the bone age detection method based on the improved Mask R-CNN model described in any one of claims 1 to 6.

Images