X-ray image analysis method

Abstract

An X-ray image analysis method is performed via a computer, comprising: receiving an X-ray image; selecting at least one detection region in the X-ray image; performing image standardization processing on a target image in the detection region to obtain a to-be-analyzed image; and inputting the to-be-analyzed image into an image classification model to obtain a bone density analysis result.

Description

Technical Field

[0001] This invention relates to an image analysis method, and more particularly to a method for X-ray image analysis using a neural network model. Background Technology

[0002] Dual-energy X-ray absorptiometry (DXA), commonly known as a bone densitometer, generates X-rays of two different energies. Since X-rays of different energies are attenuated to different degrees by different media, the higher the density of the medium, the greater the attenuation of the X-rays. Therefore, it is possible to detect bone density in different parts of the body. However, DXA measurements typically take 15 to 20 minutes, and the equipment is more expensive than a regular X-ray machine, which hinders its widespread adoption by the public. Summary of the Invention

[0003] An embodiment of the present invention provides an X-ray image analysis method. The X-ray image analysis method, executed by a computer, includes: receiving an X-ray image; confirming whether the image quality of the X-ray image meets the requirements; selecting at least one detection area in the X-ray image; performing image standardization processing on the target image in the detection area to obtain an image to be analyzed; and inputting the image to be analyzed into an image classification model to obtain bone density analysis results.

[0004] The X-ray image analysis method according to embodiments of the present invention can automatically analyze bone density analysis results from X-ray images. According to some embodiments, risk values ​​can also be further analyzed. Attached Figure Description

[0005] Figure 1 This is a flowchart of an X-ray image analysis method according to an embodiment of the present invention.

[0006] Figure 2 This is a detailed flowchart of image standardization processing according to an embodiment of the present invention.

[0007] Figure 3 This is a detailed flowchart of image standardization processing according to another embodiment of the present invention.

[0008] Figure 4 This is a detailed flowchart of image classification processing according to an embodiment of the present invention.

[0009] Figure 5 This is a detailed flowchart of image classification processing according to another embodiment of the present invention.

[0010] Figure 6 This is a detailed flowchart of risk value prediction according to an embodiment of the present invention.

[0011] Figure 7This is a detailed flowchart of risk value prediction according to another embodiment of the present invention.

[0012] Figure 8 This is a detailed flowchart of risk value prediction according to another embodiment of the present invention.

[0013] Figure 9 This is a detailed flowchart of risk value prediction according to another embodiment of the present invention.

[0014] Figure 10A A schematic diagram of an X-ray image that meets the requirements.

[0015] Figure 10B This is a schematic diagram of an X-ray image that does not meet the requirements.

[0016] Figure 10C This is a schematic diagram of another X-ray image that does not meet the requirements.

[0017] The annotations in the attached figures are explained as follows:

[0018] 100 Receiving X-ray Images

[0019] 101. Confirm that the image quality of the X-ray image meets the requirements.

[0020] 200 Select detection area

[0021] 300 Image Standardization Processing

[0022] 301, 311 Image clarification processing

[0023] 302,312 Minimum side trimming

[0024] 303,315 Scaling

[0025] 313 Calculate high-texture feature regions

[0026] 314 Range Sampling

[0027] 400 inputs to the image classification model

[0028] 401 Input to triplet loss model

[0029] 402 Principal Component Analysis

[0030] 403. Obtain the analysis type based on the coordinate landing point.

[0031] 404,413 integrate all analysis types

[0032] 411 Input to the convolutional neural network

[0033] 412 Obtain the analysis type

[0034] 500 Bone mineral density analysis results obtained

[0035] 600 Feature Regularization

[0036] 700 Input Features to Risk Value Prediction Model

[0037] 800 Risk Value Detailed Implementation

[0038] Reference Figure 1 The following is a flowchart of an X-ray image analysis method according to an embodiment of the present invention. First, X-ray images are received (step 100).

[0039] In some embodiments, the X-ray images are spinal X-ray images, femoral X-ray images, clavicle X-ray images, or metacarpal X-ray images. By analyzing the characteristics of these spinal, femoral, clavicle, or metacarpal X-ray images, it is possible to determine whether subtle changes in the texture structure caused by bone loss have occurred in the corresponding areas, thereby estimating whether osteoporosis has occurred. The X-ray images are obtained using a diagnostic X-ray machine, a mobile X-ray machine, or a mobile X-ray vehicle, which reduces equipment costs and measurement time compared to traditional dual-energy X-ray absorptiometry.

[0040] In step 101, it is confirmed whether the image quality of the X-ray image meets the requirements. If it does, the subsequent steps continue; otherwise, the process ends. Specifically, this step can use operators such as Canny, Focus, Sobel, and Laplacian to check the image quality of the X-ray image. For example, a threshold can be set, and the result of applying one of the aforementioned operators to the X-ray image is compared with this threshold. If the result is lower than the threshold, it indicates that the image meets the requirements. For example, the Sobel operator can calculate the gradients in the horizontal and vertical directions. If the gradient value is too high, it indicates that the image contains excessive noise. Figure 10A The image shown is a schematic diagram of an X-ray image that meets the requirements; as shown Figure 10B The image shown is a schematic diagram of an X-ray image that does not meet the requirements. It can be seen that the image contains too many noise points; for example... Figure 10C The image shown is a schematic diagram of another X-ray image that does not meet the requirements. It can be seen that the image includes several horizontal lines. Therefore, it is possible to detect whether the texture in an X-ray image is clear enough, in order to filter out sufficiently clear images and avoid errors in subsequent judgments.

[0041] In some embodiments, multiple operator functions may be used, each corresponding to a threshold. When the calculation results of these operator functions are all lower than the corresponding thresholds, the image quality of the X-ray image is determined to meet the requirements.

[0042] In step 200, at least one detection area is selected from the X-ray image. For example, for a femoral X-ray image, the femoral neck is selected as the detection area.

[0043] In some embodiments, step 200 also provides a user interface for a user to select the detection area.

[0044] In some embodiments, step 200 is implemented using an object detection model. The object detection model may be, for example, Mask R-CNN, YOLO, or similar models. The object detection model needs to be pre-trained by inputting multiple sample images and corresponding labeled regions containing the target (e.g., the femoral neck) into the object detection model to train it to detect the femoral neck in femoral X-ray images.

[0045] In step 300, image standardization is performed on the target image within the detection area to obtain the image to be analyzed. For clarity, the detailed process of image standardization will be described later. Image standardization yields an image of suitable size and with the required level of detail, making it suitable for input into an image classification model.

[0046] In step 400, the processed image to be analyzed is input into the image classification model; then, in step 500, the bone mineral density analysis result is obtained based on the output of the image classification model. The image classification model is a neural network model, the implementation of which will be described in detail later. The bone mineral density analysis result may be, for example, whether osteoporosis is present or not, or the bone mineral density value, etc.

[0047] In some embodiments, the size of the detection region is determined based on the input specifications of the neural network model. For example, if the image size suitable for input to the neural network model is 224 pixels square, then the size of the detection region is also 224 pixels square.

[0048] Reference Figure 2 This is a detailed flowchart of image normalization processing according to an embodiment of the present invention. Image normalization processing step 300 includes image sharpening processing (step 301), minimum edge cropping (step 302), and scaling (step 303).

[0049] In step 301, sharpening or equalization (such as histogram equalization) can be used to make image details clearer. Before performing sharpening or equalization, grayscale processing is also included to convert the color target image into a grayscale image. If the target image is already a grayscale image, grayscale processing is not required.

[0050] In step 302, the target image is cropped. If the target image size does not meet the requirements of the aforementioned neural network model, the target image is cropped to the appropriate size. For example, if the target image is rectangular, the longer side is cropped based on the shorter side to obtain a square image.

[0051] In step 303, if the image size after processing in step 302 does not conform to the size of the aforementioned neural network model, it is scaled (proportionally reduced or enlarged) to obtain the required size of the aforementioned neural network model. After preprocessing the target image through the aforementioned steps 301 to 303, the image to be analyzed can be obtained.

[0052] Reference Figure 3 This is a detailed flowchart of image standardization processing according to another embodiment of the present invention. Compared to Figure 2 The image normalization processing in this embodiment also includes the step of calculating the high-texture feature region (step 313) and the step of range sampling (step 314). Steps 311, 312 and 315 are the same as steps 301, 302 and 303, respectively, and will not be described again here.

[0053] In step 313, an edge detection algorithm is used to detect texture in the image. Examples of edge detection algorithms include the Canny algorithm and the Sobel algorithm. Particularly for osteoporosis identification, step 313 can identify the region with the most bone texture.

[0054] In step 314, based on the center of the region with the most bone texture identified in step 313, a specific range is expanded, and multiple region images of the same size as the detection region are randomly sampled within this specific range for subsequent input into the image classification model in step 400. Here, since the sampled region images meet the size requirements of the neural network model, step 315 can be omitted.

[0055] Reference Figure 4 This is a detailed flowchart of image classification processing according to an embodiment of the present invention. In step 401, the aforementioned image or region image to be analyzed is input into the image classification model. Here, the image classification model is a triplet loss model. The triplet loss model is used to train a dataset with relatively small differences. The input data includes anchor examples, positive examples, and negative examples. By optimizing the model, the distance between the anchor example and the positive example is less than the distance between the anchor example and the negative example, thus realizing the similarity calculation of the samples. The anchor example is a sample randomly selected from the sample set. Positive examples and anchor examples belong to the same class, while negative examples and anchor examples belong to different classes. In this way, the image features can be clustered using the triplet loss model. For example, it can be distinguished into a cluster of people suffering from osteoporosis and a cluster of people not suffering from osteoporosis.

[0056] In step 402, the output of the triplet loss model is dimensionality-reduced using Principal Component Analysis (PCA). PCA finds a projection axis in the feature space for the data, and the projection yields the maximum variance of the data set. This effectively reduces the number of dimensions, but the overall variance is not significantly reduced. Thus, PCA can be used to reduce the dimensionality of the clustering results to obtain the distribution coordinate information of each cluster. After steps 401 and 402, the image or region image input to the triplet loss model can be converted into a coordinate point.

[0057] In step 403, based on the distribution coordinate information of each group obtained during the training process, it is determined which group the coordinate point falls within, in order to obtain which group (or analysis type) the image belongs to.

[0058] Step 404 involves aggregating all analysis types. This means aggregating the analysis types obtained for each image or region captured from the same X-ray image. For example, if three regions were captured from the same X-ray image, each region would yield an analysis type after steps 401 to 403. Step 404 aggregates these three analysis types so that a bone mineral density (BMD) analysis result can be obtained in step 500 based on the aggregation result. Specifically, the majority of the analysis types is used as the BMD analysis result. For instance, if two analysis types indicate osteoporosis and one indicates no osteoporosis, the majority vote determines the BMD analysis result indicating osteoporosis.

[0059] Reference Figure 5 This is a detailed flowchart of image classification processing according to another embodiment of the present invention. Figure 4 The difference lies in the fact that this embodiment uses Convolutional Neural Networks (CNNs) as the image classification model, such as Deep Residual Networks (ResNet), GoogleLeNet, and DenseNet. During model training, X-ray images used as training samples are used to obtain the image or region to be analyzed in the aforementioned manner, and the analysis type of the X-ray image is labeled and input into the model. The last layer of the convolutional neural network is a weight classifier (such as XGBoost) to predict the probability of possible categories based on extracted features. Therefore, when making a prediction, the X-ray image to be identified is used to obtain the image or region to be analyzed in the aforementioned manner and input into the model (step 411), and the predicted analysis type can be obtained (step 412). Step 413 is the same as step 404 described above and will not be repeated here.

[0060] Reference Figure 6 This is a detailed flowchart of risk value prediction according to an embodiment of the present invention. Continuing from the foregoing... Figure 5 In some embodiments, features extracted by the convolutional neural network can be reused. These extracted features are input into another neural network model (hereinafter referred to as the risk value prediction model) (step 700). Here, the risk value prediction model can be a multilayer perceptron (MLP). During training, the extracted features and corresponding risk values ​​corresponding to the training samples are input into the risk value prediction model, so that when making a prediction judgment, the risk value can be predicted based on the extracted features of the sample to be identified, thereby obtaining the predicted risk value (step 800). In osteoporosis identification applications, the risk value can be, for example, a T-score parameter or a fracture risk assessment (FRAX) parameter. In some embodiments, in addition to the features extracted from the convolutional neural network, other features can also be input into the risk value prediction model, such as personal data (e.g., gender, age), body data (e.g., body mass index (BMI), height, weight), medical information (medical history (e.g., whether or not one has diabetes, hypertension)), etc. These features can be input by the user through a user interface or obtained by reading a medical record database.

[0061] Reference Figure 7 This is a detailed flowchart of risk value prediction according to another embodiment of the present invention. Figure 6 The difference is that, before step 700, step 600 is also performed to normalize the extracted features to a numerical range between 0 and 1.

[0062] Reference Figure 8 This is a detailed flowchart of risk value prediction according to another embodiment of the present invention. Similar to the foregoing... Figure 6 Similarly, the features extracted by the triplet loss model can be reused and input into the aforementioned risk value prediction model (step 700). Step 800 is as described above and will not be repeated here.

[0063] In some embodiments, in addition to the features extracted from the triplet loss model, other features may also be input into the risk value prediction model, such as personal data (e.g., gender, age), body data (e.g., body mass index (BMI), height, weight), and medical information (e.g., medical history (e.g., presence or absence of diabetes, hypertension)). These features can be input by the user through a user interface or obtained by reading a medical record database.

[0064] Reference Figure 9This is a detailed flowchart of risk value prediction according to another embodiment of the present invention. Similar to the aforementioned... Figure 7 Similarly, before step 700, step 600 is also performed to normalize the extracted features to a numerical range between 0 and 1.

[0065] The aforementioned X-ray image analysis method is implemented by loading and executing a computer program product. The computer program product consists of multiple program instructions and is stored in a non-transitory computer-readable medium. The computer can be, for example, a personal computer, a server, or other computing device with computational capabilities. Computers generally have hardware resources such as processing units (e.g., central processing unit, graphics processing unit), memory, storage media (e.g., hard disk), input / output interfaces, and network interfaces.

[0066] In some embodiments, the computer may be linked to a medical image storage system (such as a Picture Archiving and Communication System, PACS) or a medical testing instrument to obtain X-ray images.

[0067] In summary, the X-ray image analysis method according to embodiments of the present invention can automatically analyze bone density results from X-ray images. According to some embodiments, risk values ​​can also be further analyzed.

Claims

1. A method for analyzing X-ray images, characterized in that, The X-ray image analysis method, executed by a computer, includes: Receive X-ray images; The image quality of the X-ray image is confirmed to meet the requirements by using an edge detection operator function. The result of performing the edge detection operator function on the X-ray image is compared with a threshold. If the result is lower than the threshold, it means that the image quality meets the requirements. An edge detection algorithm is used to detect texture in the X-ray image to find the region with the most bone texture. A specific range is expanded based on the center of the region with the most bone texture, and at least one detection region in the X-ray image is randomly selected from the specific range. Perform image normalization processing on the target image in the detection area to obtain the image to be analyzed; and The image to be analyzed is input into the triplet loss model to calculate the similarity of the samples, and the image features are grouped by the triplet loss model. The clustering results of the triplet loss model are used to find a projection axis in the feature space of the data through principal component analysis. After projection, the maximum variation of the data can be obtained, so as to reduce the dimensionality of the clustering results and convert the image to be analyzed input into the triplet loss model into coordinate points. The analysis type of the image to be analyzed is obtained based on the cluster range of the coordinate points located by the projection and dimensionality reduction of the clustering results; and The analysis types of all the images to be analyzed are integrated to obtain bone mineral density analysis results.

2. The X-ray image analysis method according to claim 1, characterized in that, Also includes: Multiple features extracted via the triplet loss model are input into the risk value prediction model to obtain the risk value.

3. The X-ray image analysis method according to claim 2, characterized in that, Before inputting the plurality of features into the risk value prediction model, the method further includes: normalizing the plurality of features.

4. The X-ray image analysis method according to claim 2, characterized in that, The risk value prediction model is a multilayer perceptron.

5. The X-ray image analysis method according to claim 1, characterized in that, The step of selecting the detection area is implemented through an object detection model.

Images