Method and device for bone age assessment based on global and local feature collaboration, equipment and medium

Abstract

The application provides a bone age evaluation method and device based on global and local feature cooperation, equipment and medium, relates to the field of image processing, and includes: establishing an initial evaluation model and training to obtain a target evaluation model; obtaining a bone image to be evaluated, and inputting the preprocessed bone image to the target evaluation model; performing feature extraction by using a first convolutional network to obtain global features; performing identification and cutting by using a pre-trained target detection model to obtain a plurality of sub-images containing ROI regions of preset categories; performing feature extraction on each sub-image by using a second convolutional network to obtain a plurality of local features; performing convolution and normalization on the global features and the local features to obtain global context local features; after fusing each local feature with the global context local features, connecting the global features and the local features, and processing through a fully connected layer, a bone age evaluation result is obtained, and the problem that there is no full-automatic bone age evaluation method for sufficiently mining data features is solved.

Description

Technical Field

[0001] This invention relates to the field of image processing, and more particularly to bone age assessment methods, apparatus, devices, and media based on the collaboration of global and local features. Background Technology

[0002] Human growth and development can be represented by two "ages": chronological age (calendar age) and biological age (bone age). The changes in human skeletal development are generally similar, with each bone exhibiting a continuous and phased developmental process. Bones at different stages have different morphological characteristics; therefore, bone age assessment can accurately reflect an individual's growth and development level and maturity. Bone age was first applied in pediatric medicine. It not only determines a child's biological age but also provides early insight into a child's growth potential and sexual maturation trends. Bone age can also predict a child's adult height. A difference between bone age and chronological age within ±1 is considered normal development; a difference >1 is considered precocious puberty; and a difference <-1 year is considered delayed puberty. Abnormal bone age is often a manifestation of certain pediatric endocrine disorders. Therefore, bone age assessment is also very helpful in diagnosing some pediatric endocrine diseases. In addition, because bone age provides a scientific and objective assessment of biological age, it is also frequently used in athlete evaluation and judicial proceedings.

[0003] Bone age assessment typically involves X-raying the subject's hands and wrists, followed by interpretation by a physician. Interpretation methods include the General Patient Age (GP) atlas method, which compares the X-ray images to a standard atlas, and the Torscher Werner (TW) scoring method, which scores bone age according to developmental stages. Clinical bone age interpretation relies heavily on expert experience and suffers from drawbacks such as high workload, long measurement cycles, poor repeatability, strong subjectivity, instability, and unreliability. The GP atlas method is based on standardized radiographic atlases used in child growth studies. It assesses bone age by directly comparing the subject's X-ray images with standard atlases. The GP method is simple, clear, and easy to use, and is widely used internationally. However, its limitation lies in its high subjectivity, and its accuracy cannot be guaranteed. The Torscher Werner (TW) method scores the maturity of specific epiphyses by examining a bone age scale (currently revised to TW3). Because the TW3 method independently scores the maturity of each epiphyseal, it is more objective and robust than the GP method. However, it is worth noting that the TW assessment process is very complex and requires an experienced pediatric radiologist to spend considerable time completing the bone age assessment. Therefore, many researchers are dedicated to developing rapid, accurate, and more objective methods for bone age assessment.

[0004] Bone age assessment is highly subjective due to human interpretation, with significant differences in interpretation among physicians of varying experience. Furthermore, the high cost of acquiring medical images and the need for specialized radiologists for labeling, which is time-consuming and labor-intensive, result in a very limited pool of high-quality labeled datasets specifically designed for bone age prediction. Traditional automated assessment methods mostly require manually designed features as input, failing to meet automation requirements and offering performance unsatisfactory for practical applications. Therefore, it is essential to propose a fully automated bone age assessment method that not only meets clinical needs but also fully leverages data features. Summary of the Invention

[0005] In order to overcome the above-mentioned technical defects, the purpose of this invention is to provide a bone age assessment method, device, equipment and medium based on the collaboration of global and local features, in order to solve the problem of the lack of a fully automated bone age assessment method that meets clinical needs and can fully mine data features.

[0006] This invention discloses a bone age assessment method based on the collaboration of global and local features, comprising:

[0007] An initial evaluation model is established and trained using training samples to obtain a target evaluation model; wherein, the target evaluation model includes a first convolutional network, an object detection model, a second convolutional network, and a Transformer network;

[0008] Obtain bone images to be evaluated, preprocess them to obtain images to be processed, and input them into the target evaluation model;

[0009] A first convolutional network is used to extract features from the image to be processed to obtain global features;

[0010] The image to be processed is identified and cropped using a pre-trained object detection model to obtain several sub-images containing ROI regions of preset categories;

[0011] A second convolutional network is used to extract features from each sub-image to obtain several local features;

[0012] The global and local features are convolutionally and normally processed using a Transformer network to obtain global context local features.

[0013] After fusing each local feature with the global context local features, the global features and local features are connected, and then processed through a fully connected layer to obtain the bone age assessment result.

[0014] Preferably, the step of performing convolution and normalization on the global features and the local features to obtain global context local features includes:

[0015] Each of the aforementioned local features is processed using a 1*1 convolutional layer to obtain multiple first feature data;

[0016] The global features are processed using a 1*1 convolutional layer to obtain the second feature data;

[0017] The second feature data is processed again using a 1*1 convolutional layer to obtain the third feature data;

[0018] The first feature data corresponding to each of the local features is multiplied by the second feature data and then normalized to obtain the global context feature corresponding to each local feature. The global context features corresponding to each local feature are summed and then multiplied by the third feature data to obtain the global context local features.

[0019] Preferably, the global context local features can be represented as:

[0020] in, The matrix representation of the first feature data; The matrix representation of the second feature data; is the matrix representation of the third feature data; T represents the transpose of the matrix; i is the category index of the ROI region of the preset category; j is the position index of the image to be processed; d is the number of channels.

[0021] Preferably, the step of training with training samples to obtain the target evaluation model includes:

[0022] Bone images with labeled bone age information are collected from a database and classified according to the bone age information in natural year cycle to form a dataset containing at least one set of bone age information in the interval [a, b), where a and b are positive integers.

[0023] Histogram equalization and size adjustment are performed on each image in the dataset;

[0024] Images from the dataset are randomly selected to generate training, validation, and test sets to generate training samples, and the initial evaluation model is trained to obtain the target evaluation model.

[0025] Preferably, before generating training samples, the method further includes:

[0026] Data augmentation is performed on the training set and the validation set;

[0027] The data augmentation includes image transposition, horizontal mirroring, rotation, translation, scaling, or scale transformation.

[0028] Preferably, the step of using a pre-trained target detection model to identify the image to be processed, and training the target detection model, includes:

[0029] Several bone images are obtained from a database, and ROI regions of a preset category are pre-labeled to generate training data;

[0030] A target detection model is built based on the YOLO network, and the model parameters are set according to the ROI regions of the preset categories.

[0031] The target detection model is trained using training data, and the loss function and weight parameters are updated to obtain a pre-trained target detection model.

[0032] Preferably, the step of using a pre-trained object detection model to identify and crop the image to be processed to obtain several sub-images containing ROI regions of preset categories includes:

[0033] The image to be processed is identified using a pre-trained object detection model to obtain an image to be processed with multiple prediction boxes, wherein each prediction box corresponds to a ROI region of a preset category.

[0034] If the number of Regions of Interest (ROIs) in the image to be processed is lower than a preset value, the image will be discarded.

[0035] If the number of ROI regions in the image to be processed is not less than a preset value, then the ROI regions of the preset categories are cropped to obtain several sub-images.

[0036] The present invention also provides a bone age assessment device based on the collaboration of global and local features, comprising:

[0037] The training module is used to establish an initial evaluation model and train it using training samples to obtain a target evaluation model; wherein, the target evaluation model includes a first convolutional network, an object detection model, a second convolutional network, and a Transformer network;

[0038] The preprocessing module is used to acquire bone images to be evaluated, and to obtain images to be processed after preprocessing, so as to input them into the target evaluation model;

[0039] A global feature extraction module is used to extract features from the image to be processed using a first convolutional network to obtain global features;

[0040] The local feature extraction module is used to identify and crop the image to be processed using a pre-trained target detection model to obtain several sub-images containing ROI regions of preset categories; and to extract features from each sub-image using a second convolutional network to obtain several local features.

[0041] The processing module is used to perform convolution and normalization on the global features and the local features using a Transformer network to obtain global context local features; after fusing each local feature with the global context local features, the global features and local features are connected and processed through a fully connected layer to obtain bone age assessment results.

[0042] The present invention also provides a computer device, the computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the evaluation method.

[0043] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the evaluation method.

[0044] Compared with existing technologies, the above technical solution has the following advantages:

[0045] This invention provides a bone age assessment method based on a global-local collaborative Transformer, achieving efficient and fully automated bone age assessment. First, it detects and extracts the Region of Interest (ROI) of interest (i.e., local information) during the doctor's image reading process, while the entire image represents global information. Convolution is used to extract global and local feature information separately. Then, a Transformer is employed to extract the feature information between the global and local information to predict bone age, improving the efficiency and accuracy of bone age assessment. Attached Figure Description

[0046] Figure 1 This is a flowchart of an embodiment of the bone age assessment method based on global and local feature collaboration described in this invention;

[0047] Figure 2 This is a schematic diagram of the network structure of the target assessment model in Embodiment 1 of the bone age assessment method based on global and local feature collaboration described in this invention;

[0048] Figure 3 This is a schematic diagram of the structure of the first convolutional network and / or the second convolutional network in Embodiment 1 of the bone age assessment method based on global and local feature collaboration described in this invention;

[0049] Figure 4 This is a schematic diagram illustrating the processing of global context local features by the Transformer network in Embodiment 1 of the bone age assessment method based on global and local feature collaboration described in this invention.

[0050] Figure 5 This is a schematic diagram of the structure of Embodiment 2 of the bone age assessment device based on global and local feature collaboration described in this invention;

[0051] Figure 6 This is a schematic diagram of the computer device described in this invention.

[0052] Figure label:

[0053] 8-Bone age assessment device based on global and local feature collaboration; 81-Training module; 82-Preprocessing module; 83-Global feature extraction module; 84-Local feature extraction module; 85-Processing module; 9-Computer equipment; 91-Memory; 92-Processor. Detailed Implementation

[0054] The advantages of the present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments.

[0055] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0056] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The singular forms “a,” “the,” and “the” as used in this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0057] It should be understood that although the terms first, second, third, etc., may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0058] In the description of this invention, it should be understood that the terms "longitudinal", "lateral", "up", "down", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0059] In the description of this invention, unless otherwise specified and limited, it should be noted that the terms "installation", "connection" and "linking" should be interpreted broadly. For example, they can refer to mechanical or electrical connections, or internal connections between two components. They can be direct connections or indirect connections through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms according to the specific circumstances.

[0060] In the following description, suffixes such as "module," "part," or "unit" used to denote elements are used only for the convenience of the description of the invention and have no specific meaning in themselves. Therefore, "module" and "part" can be used interchangeably.

[0061] Example 1: This example provides a bone age assessment method based on the collaboration of global and local features. (See attached document.) Figure 1 and Figure 2 ,,include:

[0062] S100: Establish an initial evaluation model and train it using training samples to obtain a target evaluation model; wherein, the target evaluation model includes a first convolutional network, an object detection model, a second convolutional network, and a Transformer network (i.e., hereinafter referred to as...). Figure 2 Global-local transformer blocks);

[0063] In this embodiment, the initial evaluation model and the target evaluation model have the same structure. Their model parameters are adjusted during training, and the target evaluation model is obtained only after training is complete. Specifically, the step of training with training samples to obtain the target evaluation model includes:

[0064] S110: Collect bone images with calibrated bone age information from a database, and classify them according to the bone age information in natural year cycle to form a dataset containing at least one set of bone age information located in the interval [a, b), where a and b are positive integers.

[0065] In the above steps, the database can be formed by collecting X-ray images from the radiology department of a hospital. The ratio of males to females can be set to be half and half. Based on the calibrated bone age, the data can be classified in 12-month units, such as [7,8) years old as one group and [8,9) years old as another group, thereby generating a dataset containing several groups.

[0066] S120: Perform histogram equalization and size adjustment on each image in the dataset;

[0067] In the above steps, histogram equalization is performed on each image (X-ray image) in the above dataset to enhance contrast, and then digital sampling is performed to make the image size 224×224. Specifically, it should be noted that the same steps as histogram equalization and size adjustment are performed in the preprocessing process of step S200 below to facilitate subsequent model processing.

[0068] S130: Randomly select images from the dataset to generate a training set, a validation set, and a test set to generate training samples, and train the initial evaluation model to obtain the target evaluation model.

[0069] In the above steps, the training set is used as input to the model (the initial evaluation model mentioned above), and the validation set is used to compare with the model's output and adjust the model parameters. After adjustment, the test set is used for testing. If the test is successful, it means that the training is complete. Then the model parameters can be fixed and the target evaluation model can be obtained. Specifically, 80% of the collected wrist bone data can be randomly selected as the training set, 10% as the validation set, and 10% as the test set.

[0070] To further increase the amount of data in the training samples and make the target evaluation model obtained after training more accurate, before generating the training samples, the method further includes: data augmentation of the training set and the validation set; wherein, the data augmentation includes, but is not limited to, any one or more of image transposition, horizontal mirroring, rotation, translation, scaling, or scale transformation. This increases the data width of the training samples, thereby improving the accuracy of the model after training.

[0071] S200: Obtain the bone image to be evaluated, preprocess it to obtain the image to be processed, and input it into the target evaluation model;

[0072] In the above steps, it should be noted that the bone influence to be evaluated is the subject's bone image. A subject can be associated with one or more bone images. Each bone image is processed individually, thus generating multiple images to be processed. This also allows for the acquisition of multiple global features, and multiple local features corresponding to any ROI region, for bone age estimation to improve accuracy. It should also be noted that this preprocessing is similar to the sample data preprocessing described above, performing histogram equalization to enhance contrast, followed by digital sampling to resize the image to 224 × 224 pixels.

[0073] S300: The first convolutional network is used to extract features from the image to be processed to obtain global features;

[0074] In the above steps, the first convolutional network performs feature extraction, using S-VGGNet (see...). Figure 3 Specifically, it can include four sampling operations. Each sampling operation may include two 3*3 convolutional layers + BN + ACT layers and a max pooling layer to extract global features. Further, the second convolutional network described below can have the same structure as the first convolutional network or a different structure. The global features are denoted as... j represents the position index of the global image.

[0075] S400: The image to be processed is identified and cropped using a pre-trained object detection model to obtain several sub-images containing ROI regions of preset categories;

[0076] In the above steps, the object detection network is mainly used to detect Regions of Interest (ROIs), enabling the subsequent acquisition of corresponding sub-images based on the ROIs to collect local features. Specifically, the step of using a pre-trained object detection model to identify the image to be processed, and training the object detection model, includes:

[0077] Several bone images are obtained from a database, and ROI regions of a preset category are pre-labeled to generate training data; an object detection model is built based on the YOLO network, and the model parameters are set according to the preset ROI regions; the object detection model is trained using the training data, and the loss function and weight parameters are updated to obtain the pre-trained object detection model.

[0078] In the training process described above, 1000 images of various age groups were randomly selected. ROI regions were manually labeled using the TW method. 800 images were used to train the object detection network, and 200 images were used for testing. The object detection model used the YOLO network, with 18 categories (i.e., the number of categories in the preset ROI region). The predicted bounding boxes included the confidence level of the target category and the probability of each bounding box region across multiple categories. Redundant windows were removed using non-maximum suppression. The prepared training set and image-labeled categories (i.e., training data) were input into the designed deep convolutional neural network (i.e., the object detection model described above) for training. By reducing the loss function value and updating the network weight parameters, after several training iterations, the learned network weight parameters were obtained, thus generating the pre-trained object detection model.

[0079] Specifically, after the pre-trained object detection model identifies and crops the image to be processed, several sub-images containing ROI regions of preset categories need to be obtained. Image correction is then performed, specifically including:

[0080] S410: The image to be processed is identified using a pre-trained object detection model to obtain an image to be processed with multiple prediction boxes, wherein each prediction box corresponds to a ROI region of a preset category.

[0081] In the above steps, after the target detection model, the output image to be processed has a predicted bounding box. Each predicted bounding box corresponds to one of the ROI regions of a preset category. As mentioned above, if there are 18 ROI regions of the preset category, the predicted bounding box should contain 18.

[0082] S420: If the number of ROI regions in the image to be processed is lower than the preset value, discard it;

[0083] After the detection in step S410 above, each predicted box corresponds to a ROI region. Data with less than 14 ROI categories (i.e., the number of predicted boxes is less than 14) are deleted, thereby excluding some poorly developed or diseased images, i.e., discarding bad images to improve the accuracy of subsequent evaluation results.

[0084] S430: When the number of ROI regions in the image to be processed is not less than the preset value, the ROI regions of the preset category are cropped to obtain several sub-images.

[0085] Specifically, the 18 detected ROI regions are cropped from the corrected image (i.e., after the above-mentioned discarding operation), and after digital sampling, each ROI block (i.e., the image containing each ROI region) is unified to the same size: 64×64, so that feature extraction can be performed in the following step S500.

[0086] S500: A second convolutional network is used to extract features from each sub-image to obtain several local features;

[0087] In the above steps, the second convolutional network can be set to be the same as the first convolutional network, i.e., S-VGGNet, or it can be set to other neural networks capable of feature extraction. As described above, each sub-image includes a ROI region of a preset category. Therefore, by performing feature extraction on each sub-image, local features can be obtained, which can be denoted as...

[0088]

[0089] S600: Use the Transformer network to perform convolution and normalization on the global features and the local features to obtain global context local features;

[0090] In this embodiment, see Figure 4 Using the global features obtained in step S300 and the local features obtained in step S500, a Transformer is then used to extract the feature information between the global and local features to predict bone age. Specifically, performing convolution and normalization on the global and local features to obtain global context local features includes:

[0091] S610: Process each of the local features using a 1*1 convolutional layer to obtain multiple first feature data;

[0092] The above steps refer to the above features Performing a 1×1 convolution operation maps to a new space, denoted as .

[0093] S620: The global features are processed using a 1*1 convolutional layer to obtain the second feature data;

[0094] The above steps, namely feature f G Performing a 1×1 convolution operation maps to a space, denoted as .

[0095] S630: The second feature data is processed again using a 1*1 convolutional layer to obtain the third feature data;

[0096] The above steps are to transform the features Performing a 1×1 convolution operation maps to a space, denoted as . That is, for feature f G Performing two 1×1 convolution operations maps to two spaces, denoted as follows: (Second feature data) (Third characteristic data);

[0097] S640: Normalize the product of the first feature data and the second feature data corresponding to each local feature to obtain the global context feature corresponding to each local feature. Sum the global context features corresponding to each local feature and then multiply them with the third feature data to obtain the global context local feature.

[0098] In the above steps, after the convolution operations in S610-S630, we obtain... Then perform normalization, that is: That is, the global context feature corresponding to each of the above local features; the global context local features can be calculated by the following formula after considering the set of global context feature information corresponding to each local feature: All the above feature data can be summarized into matrix operations. Specifically, the global context local features can be represented as follows:

[0099] in, The matrix representation of the first feature data; The matrix representation of the second feature data; is the matrix representation of the third feature data; T represents the transpose of the matrix; i is the category index of the ROI region of the preset category; j is the position index of the image to be processed; d is the number of channels.

[0100] As can be seen, the number of channels mentioned above are parameters of the network, and the global context local features mentioned above are features that fuse local and global features. That is, the interaction between local and global features is realized through the Transform network. Specifically, the encoder-decode framework of the Transformer network is used to realize the fusion calculation of local and global features to generate a global-local collaborative Transformer network for subsequent bone age assessment. The calculation implemented by this network structure is continuously adjusted in the target assessment model using the training samples in S100 mentioned above until the parameters of the network structure are determined.

[0101] S700: After fusing each local feature with the global context local feature, the global feature and the local feature are connected, and processed through a fully connected layer to obtain the bone age assessment result.

[0102] In the above steps, global context local features can be obtained based on S600. These features are then fused with local features, and after concatenating the global and local features, they are mapped to a bone age range through a fully connected layer. The fully connected layer, located at the end of the entire model, is responsible for transforming the two-dimensional feature map output by the convolution into a one-dimensional vector or information, i.e., the bone age assessment result. It should also be noted that the fusion of local features with the global context local features, and the connection between the global and local features, is performed after the fused feature set. With F G-L After fusion, plus and Instead of performing feature fusion on all features, this improves the accuracy of the evaluation results.

[0103] This implementation provides a bone age assessment method based on a global-local collaborative Transformer, achieving efficient and fully automated bone age assessment. First, it detects and extracts the Region of Interest (ROI) that doctors focus on during image reading, i.e., local information, while the entire image represents global information. Convolutional networks (a first convolutional network and a second convolutional network) are used to extract global and local feature information respectively. Then, a Transformer is used to extract the feature information between the global and local information to predict bone age, improving the efficiency and accuracy of bone age assessment and reducing the burden on specialist physicians.

[0104] Example 2: The present invention also provides a bone age assessment device 8 based on the collaboration of global and local features, see reference. Figure 5 , include:

[0105] Training module 81 is used to establish an initial evaluation model and train it using training samples to obtain a target evaluation model; wherein, the target evaluation model includes a first convolutional network, an object detection model, a second convolutional network, and a Transformer network (i.e., hereinafter referred to as...). Figure 2 Global-local transformer blocks);

[0106] Specifically, the database can be formed from X-ray images collected from hospital radiology departments. It can be set to have an equal number of males and females, and the data can be classified in 12-month units according to the labeled bone age, such as [7,8) years old as one group, [8,9) years old as another group, thus generating a dataset containing several groups. 80% of the collected wrist bone data can be randomly selected as the training set, 10% as the validation set, and 10% as the test set to generate training samples for training.

[0107] The preprocessing module 82 is used to acquire bone images to be evaluated, and to obtain images to be processed after preprocessing, so as to input them into the target evaluation model;

[0108] Specifically, preprocessing operations include histogram equalization or digit sampling to enhance contrast, or adjusting image size for subsequent processing in the target evaluation model. These can also be set as standardized preprocessing steps.

[0109] The global feature extraction module 83 is used to extract features from the image to be processed using a first convolutional network to obtain global features;

[0110] Specifically, S-VGGNet can be used for feature extraction, and global features are those extracted from the entire image.

[0111] The local feature extraction module 84 is used to identify and crop the image to be processed using a pre-trained target detection model to obtain several sub-images containing ROI regions of preset categories; and to extract features from each sub-image using a second convolutional network to obtain several local features.

[0112] Specifically, the second convolutional network can be set to be the same as or different from the first convolutional network. The object detection network is mainly used to detect ROI regions of a preset category, so that the corresponding sub-images can be obtained based on the ROI regions to collect local features. After object detection, a processing image with a predicted bounding box containing the ROI region of the preset category is obtained. When the ROI region is lower than the preset value, it may be a poorly developed or diseased image. The image is discarded to reduce its impact on local feature extraction, thereby reducing the impact on the accuracy of the evaluation results.

[0113] The processing module 85 is used to perform convolution and normalization on the global features and the local features using a Transformer network to obtain global context local features; after fusing each local feature with the global context local features, the global features and local features are connected, and then processed through a fully connected layer to obtain bone age assessment results.

[0114] Specifically, a 1×1 convolution operation is performed sequentially on the local features, and two 1×1 convolution operations are performed on the global features to obtain the first feature data, the second feature data, and the third feature data, respectively. The first feature data corresponding to each local feature is multiplied by the second feature data and then normalized to obtain the global context feature corresponding to each local feature. The global context features corresponding to each local feature are summed and then multiplied by the third feature data to obtain the global context local features.

[0115] An initial assessment model is established and trained using a training module to obtain a target assessment module. Then, the bone images collected from the subjects are preprocessed in the preprocessing module for input into the target assessment module. In the target assessment model, local and global feature extraction modules are used to detect and extract regions of interest (ROIs) relevant to the doctor's image reading process, i.e., extracting local images (sub-images). The entire image is considered the global image. Convolutional networks (first and second convolutional networks) are used to extract global and local feature information respectively. Finally, in the processing module, a Transformer network is used to extract the feature information between global and local features to predict bone age, improving the efficiency and accuracy of bone age assessment.

[0116] Example 3: To achieve the above objectives, the present invention also provides a computer device 9. This computer device may include multiple computer devices. The components of the bone age assessment device 8 based on global and local feature collaboration in Example 2 may be distributed across different computer devices 9. The computer device 9 may be a smartphone, tablet, laptop, desktop computer, rack server, etc., executing a program. The computer device in this embodiment includes, but is not limited to: a memory 91, a processor 92, and the bone age assessment device 8 based on global and local feature collaboration, all of which can be interconnected via a system bus. See also... Figure 6 It should be pointed out that, Figure 6 Only computer devices with components are shown; however, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0117] In this embodiment, the memory 91 may include a program storage area and a data storage area. The program storage area may store the application programs required by the operating system and at least one function; the data storage area may store user data on the computer device. Furthermore, the memory 91 may include high-speed random access memory and non-volatile memory. In some embodiments, the memory 91 may optionally include memory 91 remotely located relative to the processor, and these remote memories can be connected via a network. Examples of such networks include, but are not limited to, the Internet and local area networks.

[0118] In some embodiments, processor 92 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. Processor 92 is typically used to control the overall operation of a computer device. In this embodiment, processor 92 is used to run program code stored in memory 91 or process data, for example, to run the bone age assessment device 8 based on global and local feature collaboration, to implement the bone age assessment method based on global and local feature collaboration of Embodiment 1.

[0119] It should be noted that only computer device 9 with components 91-92 is shown, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead.

[0120] Example 4:

[0121] To achieve the above objectives, the present invention also provides a computer-readable storage medium, which includes multiple storage media such as flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), disk, optical disk, server, etc., on which a computer program is stored. When the program is executed by processor 92, it performs corresponding functions. In this embodiment, the computer-readable storage medium is used to store a bone age assessment device 8 based on global and local feature collaboration. When executed by processor 92, it implements the bone age assessment method based on global and local feature collaboration of Embodiment 1.

[0122] It should be noted that the embodiments of the present invention have better implementability and are not intended to limit the present invention in any way. Any person skilled in the art may use the above-disclosed technical content to change or modify it into equivalent effective embodiments. However, any modifications or equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall still fall within the scope of the technical solution of the present invention.

Claims

1. A bone age assessment method based on the collaboration of global and local features, characterized in that, include: An initial evaluation model is established and trained using training samples to obtain a target evaluation model; wherein, the target evaluation model includes a first convolutional network, an object detection model, a second convolutional network, and a Transformer network; Obtain bone images to be evaluated, preprocess them to obtain images to be processed, and input them into the target evaluation model; A first convolutional network is used to extract features from the image to be processed to obtain global features; The image to be processed is identified and cropped using a pre-trained object detection model to obtain several sub-images containing ROI regions of preset categories; A second convolutional network is used to extract features from each sub-image to obtain several local features; The global and local features are convolutionally and normally processed using a Transformer network to obtain global context local features. After fusing each local feature with the global context local feature, the global feature and the local feature are connected, and then processed through a fully connected layer to obtain the bone age assessment result; The step of performing convolution and normalization on the global features and the local features to obtain global context local features includes: Each of the aforementioned local features is processed using a 1×1 convolutional layer to obtain multiple first feature data; The global features are processed using a 1×1 convolutional layer to obtain the second feature data; The second feature data is processed again using a 1×1 convolutional layer to obtain the third feature data; After multiplying the first feature data and the second feature data corresponding to each local feature, normalize the product to obtain the global context feature corresponding to each local feature. Then, sum the global context features corresponding to each local feature and multiply them with the third feature data to obtain the global context local feature. The local features of the global context can be represented as: ;in, The matrix representation of the first feature data; The matrix representation of the second feature data; is the matrix representation of the third feature data; T represents the transpose of the matrix; Category index for the ROI region of the preset category; The position index of the image to be processed; This represents the number of channels.

2. The evaluation method according to claim 1, characterized in that, The step of training the target evaluation model using training samples includes: Bone images with labeled bone age information are collected from a database and classified according to the bone age information in natural year cycle to form a dataset containing at least one set of bone age information in the interval [a, b), where a and b are positive integers. Histogram equalization and size adjustment are performed on each image in the dataset; Images from the dataset are randomly selected to generate training, validation, and test sets to generate training samples, and the initial evaluation model is trained to obtain the target evaluation model.

3. The evaluation method according to claim 2, characterized in that, Before generating training samples, the following is also included: Data augmentation is performed on the training set and the validation set; The data augmentation includes image transposition, horizontal mirroring, rotation, translation, scaling, or scale transformation.

4. The evaluation method according to claim 1, characterized in that, The step of using a pre-trained target detection model to identify the image to be processed, and training the target detection model, includes: Several bone images are obtained from a database, and ROI regions of a preset category are pre-labeled to generate training data; A target detection model is built based on the YOLO network, and the model parameters are set according to the ROI regions of the preset categories. The target detection model is trained using training data, and the loss function and weight parameters are updated to obtain a pre-trained target detection model.

5. The evaluation method according to claim 1, characterized in that, The image to be processed is identified and cropped using a pre-trained object detection model to obtain several sub-images containing ROI regions of preset categories, including: The image to be processed is identified using a pre-trained object detection model to obtain an image to be processed with multiple prediction boxes, wherein each prediction box corresponds to a ROI region of a preset category. If the number of Regions of Interest (ROIs) in the image to be processed is lower than a preset value, the image will be discarded. If the number of ROI regions in the image to be processed is not less than a preset value, then the ROI regions of the preset categories are cropped to obtain several sub-images.

6. An automatic bone age assessment device, characterized in that, include: The training module is used to establish an initial evaluation model and train it using training samples to obtain a target evaluation model; wherein, the target evaluation model includes a first convolutional network, an object detection model, a second convolutional network, and a Transformer network; The preprocessing module is used to acquire bone images to be evaluated, and to obtain images to be processed after preprocessing, so as to input them into the target evaluation model; A global feature extraction module is used to extract features from the image to be processed using a first convolutional network to obtain global features; The local feature extraction module is used to identify and crop the image to be processed using a pre-trained target detection model to obtain several sub-images containing ROI regions of preset categories; and to extract features from each sub-image using a second convolutional network to obtain several local features. The processing module is used to perform convolution and normalization on the global features and the local features using a Transformer network to obtain global context local features; after fusing each local feature with the global context local features, the global features and local features are connected, and then processed through a fully connected layer to obtain bone age assessment results; The step of performing convolution and normalization on the global features and the local features to obtain global context local features includes: Each of the aforementioned local features is processed using a 1×1 convolutional layer to obtain multiple first feature data; The global features are processed using a 1×1 convolutional layer to obtain the second feature data; The second feature data is processed again using a 1×1 convolutional layer to obtain the third feature data; After multiplying the first feature data and the second feature data corresponding to each local feature, normalize the product to obtain the global context feature corresponding to each local feature. Then, sum the global context features corresponding to each local feature and multiply them with the third feature data to obtain the global context local feature. The local features of the global context can be represented as: ;in, The matrix representation of the first feature data; The matrix representation of the second feature data; is the matrix representation of the third feature data; T represents the transpose of the matrix; Category index for the ROI region of the preset category; The position index of the image to be processed; This represents the number of channels.

7. A computer device, characterized in that, The computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the evaluation method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the evaluation method described in claims 1-5.

Images