A Dual-View Vertebral Fracture Feature Detection Method Based on DR Images

Abstract

The present invention discloses a processing method based on dual-view DR images for detecting fracture characteristics of patients with pathological and osteoporotic vertebral compression fractures. The present invention realizes the acquisition of images from a DR device and the preprocessing of image data, determines the regions of interest of the anterior-posterior and lateral vertebrae, designs a neural network to extract image features, automatically locates and grades the severity of vertebral fractures, and provides an effective method for detecting vertebral fracture characteristics for orthopedic doctors to conduct clinical screening and formulate early diagnosis and treatment plans for patients.

Description

Technical Field

[0001] The present invention relates to a method for detecting vertebral fracture features from dual perspectives based on DR images, belonging to the field of computer image processing. Background Art

[0002] Vertebral fractures are common fracture types in clinical practice. Due to pathological reasons or osteoporosis, vertebral compression fractures (VCFs) are prone to occur, which can lead to potential destructive sequelae, resulting in a decline in the quality of life of patients, an increase in mortality, and high social and economic costs. In recent years, with the accelerating aging of the population in China, its incidence has been rising year by year. Approximately 1.4 million new cases are affected each year, and it has become an important health issue that has attracted widespread attention. Although the imaging technology of X-ray medical imaging equipment has been continuously improved, there are still problems such as noise artifacts and uneven gray levels in the images. It is inevitable that doctors will have deviations when identifying fracture sites with the naked eye, which affects the accuracy of fracture detection. Adopting an accurate and effective fracture feature detection method can significantly improve the treatment and prognosis of patients, and increase the survival rate and life expectancy of patients.

[0003] Currently, the commonly used semi-quantitative assessment method for analyzing vertebral fractures is based on doctors' visual observation. The fracture location is determined according to the shape of the vertebral body and the percentage of height reduction and endplate surface loss in the anterior, posterior, and middle parts of the vertebral body. And according to the degree of height reduction and surface loss, the severity of vertebral fractures is divided into Grand 1 (mild) to Grand 3 (severe), where Grand 1 is a height loss of 20% - 25% or a surface loss of 10% - 20%, Grand 2 is a height loss of 25% - 40% or a surface loss of 20% - 40%, and Grand 3 is a height loss or surface loss exceeding 40%. This is a practical and repeatable method for vertebral fracture assessment, which is easy to implement in clinical diagnosis and treatment, and is suitable for epidemiological studies and clinical efficacy trials. However, this method requires professional and experienced doctors.

[0004] X-ray DR plain film is a fast, widely used and inexpensive technology with relatively low radiation dose, and is usually the preferred examination method for spinal fractures. Due to the composition and characteristics of DR imaging equipment, there are systematic inherent noise and random noise in DR images, which reduce the contrast of the images and affect the feature detection of spinal fractures. Clinically, fractures are screened for patients by visually observing DR images, and different doctors may give different conclusions. Moreover, the semi-quantitative evaluation method requires a large amount of computational work, increasing the pressure on radiologists to process image data and resulting in deviations in the severity grading of spinal fractures. In view of the existing problems, the present invention simulates the process of doctors observing the plain films from two perspectives of the DR anteroposterior and lateral views, and proposes a method for detecting spinal fracture features based on dual-view DR images, for localizing spinal fractures and grading their severity, improving the accuracy of fracture feature detection by radiologists and orthopedic doctors, and formulating a more reasonable treatment plan for patients. Summary of the Invention

[0005] The purpose of the present invention is to provide a method for detecting spinal fracture features based on dual-view DR images, providing a reference for fracture feature detection and treatment plan formulation by radiologists and orthopedic doctors.

[0006] The present invention adopts the following technical solutions to solve the above problems:

[0007] The present invention provides a method for detecting spinal fracture features based on dual-view DR images, and the specific steps are as follows:

[0008] Step 1: Collect images of spinal fracture patients from a DR device and perform preprocessing (Imagepreprocessing) on the image data.

[0009] By using median filtering and anisotropic diffusion filtering methods, remove the random noise in the images collected from the DR imaging system, and filter out the noise points that cause blurring of the edges of the spinal vertebrae in the DR images. According to the DICOM format data tags or the maximum and minimum values of the original pixels, adjust and set the window width and window level of the DR images, and map the original pixels to 8-bit images. Perform image enhancement using contrast-limited adaptive histogram equalization to enhance the contrast of the DR images while suppressing noise, making the distinction between the spine and soft tissues more obvious.

[0010] Step 2: Use the preprocessed images and the annotations of radiologists with rich clinical experience to obtain the regions of interest (ROI) of different vertebrae in the dual-angle DR plain films of the anteroposterior and lateral views.

[0011] According to the DR image and the corresponding annotation information, the image patches containing the vertebral fracture ROI are cropped out (ROIpatches), and the image patches of the normal vertebral ROI are obtained from the adjacent positions. The image patches are resized to a uniform size of 224×224, and the image patches are assigned the classification labels y of fracture and normal.

[0012] Step 3: Input the ROI image block into the designed neural network, perform image block feature hybridization (Hybridpatch) after the pre-trained model (CNNmodel) trained on the ImageNet dataset, and then input it into the Multi-head transformer and Shifted-window transformer network modules for feature splicing, and finally obtain the prediction result of vertebral fracture.

[0013] Furthermore, the step 3 specifically includes the following steps: the ROI of the anteroposterior and lateral views includes a normal vertebral image block I N and fractured vertebral image block I F , the feature map f(I N ,I F ) and predict classification The classification loss function (Class loss) of normal or fracture can be calculated, that is, Where N is the total number of training samples. During the neural network training process, the classification loss function is used as a constraint to optimize the CNN model and fine-tune the parameters in the pre-trained network model so that it can maintain the representation ability of image features and adapt to the statistical distribution of vertebral fracture data.

[0014] The first stage CNN model extracts the feature map f(I N ,I F ), in the second stage, the H×W×C dimension f(I N ,I F ) is converted to H / 4×W / 4×2C dimensions and input to Multi-head transformer and Shifted-window transformer respectively to avoid losing useful feature information by downsampling, where H, W, and C represent the height, width, and number of channels of the feature map respectively. In Multi-head transformer, three 1×1 convolutions transform the input feature map Project to Flatten and transpose into a sequence of size n×d, where d is the dimension of the embedding, Q, K, V are flattened and transposed into a sequence of size n×d, where n=H×W, thus the output is This enables the model to jointly infer attention for fracture localization from different representation subspaces. In the Shifted-window transformer, self-attention is calculated within shifted local windows in the feature map, and the windows are evenly divided in a non-overlapping manner, reducing the huge computational cost of global attention. The output is where B is the shift bias. Let O MHT and O SWT be concatenated to obtain f cat = Concatenate(O MHT , O SWT ). Further, it can be known that the fracture position t = (t x , t y , t w , t h ) predicted by the neural network and the localization loss function where (x, y, w, h) are the horizontal and vertical coordinates, width, and height of the predicted bounding box, and smooth L1 is the smooth L1 loss function.

[0015] The unique feature of this method is the use of anteroposterior and lateral dual-view DR images and the design of parallel network modules of Multi-head transformer and Shifted-window transformer. Through the neural network training and optimization process, the method can obtain the prediction results of the probabilities of different severity grades of fractures and the possible lesion positions in the input ROI image patches.

[0016] Step 4: According to the fracture severity grade output by the neural network and the probability values of the predicted bounding boxes, the method realizes the fracture localization and severity grading of dual-view DR images.

[0017] Step 5: Clinical radiologists and orthopedic surgeons observe the actual DR images and analyze the spinal fracture features. Combining the results of spinal fracture localization and severity grading, they determine the patient's fracture condition and formulate an effective treatment plan.

[0018] The schematic diagram of the entire process of the invention is as shown in Figure 1 .

[0019] Compared with the prior art, the present invention uses dual-view DR images, combines the anteroposterior and lateral spinal imaging information, constructs a neural network of CNN model and transformer network modules, and gives full play to the advantages of the neural network in image representation, which helps to improve the detection rate of spinal fractures. The beneficial effects are as follows: The present invention realizes the automatic spinal fracture localization and severity grading in dual-view DR images, providing an effective method for spinal fracture feature detection for the clinical screening and treatment plan formulation of patients with pathological and osteoporotic vertebral compression fractures. Brief Description of the Drawings

[0020] Figure 1 It is a schematic diagram of the spinal fracture feature detection process of the present invention;

[0021] Figure 2 It is a framework diagram of the neural network in the method,

[0022] Figure 3 It is the result of spinal fracture localization and severity grading obtained by the neural network in the method;

[0023] Figure 4 It is a schematic diagram of the overall process of the present invention. Detailed Embodiments

[0024] The following further clarifies the present invention in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and not to limit the scope of the present invention. After reading the present invention, various equivalent modifications of the present invention by those skilled in the art fall within the scope defined by the appended claims of this application.

[0025] Embodiment 1: Refer to Figures 1-4 , the present invention provides a method for detecting spinal fracture features based on dual-view DR images, and the specific steps are as follows:

[0026] Step 1, collect images of spinal fracture patients from a DR device and preprocess the image data (Image preprocessing).

[0027] By using median filtering and anisotropic diffusion filtering methods, remove the random noise in the images collected from the DR imaging system, and filter out the noise points that cause the edges of the spinal vertebrae in the DR images to be blurred. According to the DICOM format data tags or the maximum and minimum values of the original pixels, adjust and set the window width and window level of the DR images, and map the original pixels to 8-bit images. Use contrast-limited adaptive histogram equalization for image enhancement to enhance the contrast of the DR images while suppressing noise, making the distinction between the spine and soft tissues more obvious.

[0028] Step 2, use the preprocessed images and the annotations of radiologists with rich clinical experience to obtain the regions of interest (ROI) of different vertebrae in the dual-angle DR plain films in the anteroposterior and lateral views,

[0029] According to the DR images and the corresponding annotation information, crop out the image patches (ROI patches) containing the spinal fracture ROI, and at the same time obtain the image patches of the normal vertebra ROI from adjacent positions, adjust the size of the image patches to be uniformly 224×224, and assign the classification labels y of fracture and normal to the image patches.

[0030] Step 3: Input the ROI image patches into the designed neural network (Neural network). After passing through the pre-trained model (CNN model) trained on the ImageNet dataset, perform hybrid patch of the image patch features, and then input them into the Multi-head transformer and Shifted-window transformer network modules respectively for feature splicing. Finally, obtain the prediction result of spinal fracture.

[0031] Further, step 3 specifically includes the following steps: The ROI of the anterior-posterior and lateral radiographs includes normal vertebral body image patches I N and fractured vertebral body image patches I F , and obtain the feature map f(I N ,I F ) and the prediction classification The classification loss function (Class loss) for normal or fractured can be calculated, that is where N is the total number of training samples. During the neural network training process, the classification loss function is used as a constraint term to optimize the CNN model, and the parameters in the pre-trained network model are fine-tuned to maintain the representation ability of the image features while adapting to the statistical distribution of spinal fracture data.

[0032] The feature map f(I N ,I F ) extracted by the CNN model from the ROI image patches in the first stage is, in the second stage, after hybrid of the feature image patches, the f(I N ,I F ) with the dimension of H×W×C is converted into the dimension of H / 4×W / 4×2C and input into the Multi-head transformer and Shifted-window transformer respectively to avoid losing useful feature information during downsampling, where H, W, and C respectively represent the height, width, and number of channels of the feature map. In the Multi-head transformer, three 1×1 convolutions project the input feature map onto flatten and transpose it into a sequence of size n×d, where d is the dimension of the embedding, and Q, K, V are flattened and transposed into a sequence of size n×d, where n = H×W, and thus the output This enables the model to jointly infer attention from different representation subspaces for fracture localization. In the Shifted-window transformer, the self-attention of the feature map is calculated within the shifted local windows, and the windows are evenly divided in a non-overlapping manner, reducing the huge computational amount of global attention, and the output is where B is the shift bias. Shift O MHT and O SWT are feature concatenated to obtain f cat = Concatenate(O MHT , O SWT ). Further, it can be known that the fracture position t = (t x , t y , t w , t h ) predicted by the neural network and the localization loss function where (x, y, w, h) are the horizontal and vertical coordinates, width, and height of the predicted bounding box, and smooth L1 is the smooth L1 loss function.

[0033] The use of anteroposterior and lateral dual-view DR images and the design of parallel network modules of Multi-head transformer and Shifted-window transformer are unique to this method. Through the neural network training and optimization process, the method can obtain the prediction results of the fracture severity grading probabilities and possible lesion positions in the input ROI image patches.

[0034] Step 4, according to the fracture severity grading and the probability values of the predicted bounding boxes output by the neural network, the method realizes the fracture localization and severity grading of the dual-view DR images.

[0035] Step 5, clinical radiologists and orthopedic surgeons observe the actual DR images and perform feature detection and analysis of spinal fractures, and combine the results of spinal fracture localization and severity grading to determine the fracture condition of the patient and formulate an effective treatment plan.

[0036] The schematic diagram of the entire process of the invention is as shown in Figure 1 .

[0037] Example 2: Refer to Figure 1 — Figure 3 , a method for detecting spinal fracture features based on dual-view DR images, the specific steps are as follows:

[0038] Step 1, collect the images of spinal fracture patients from the DR device and preprocess the image data.

[0039] Specifically, Step 1 includes: filtering out the noise points in the DR image by median filtering and anisotropic diffusion filtering methods. Setting the window width and window level of the DR image and performing image enhancement by adaptive histogram equalization with limited contrast.

[0040] Step 2, obtain the ROI of the spine in the anteroposterior and lateral dual-angle DR images from the preprocessed images and annotations.

[0041] Specifically, in step 2, according to the DR image and the corresponding annotation information, image patches containing the ROI of spinal fractures are cropped, and at the same time, image patches of normal spinal ROIs are obtained from adjacent positions, and the image patches are given classification labels y of fracture and normal.

[0042] In step 3, refer to Figure 2 , the ROI image patches are input into the designed neural network (Neural network). After passing through the pre-trained model (CNN model) trained on the dataset by ImageNet, image patch feature mixing (Hybrid patch) is performed, and then they are respectively input into the Multi-head transformer and Shifted-window transformer network modules for feature splicing. Through the neural network training and optimization process, the method can obtain the prediction results of the fracture severity grading probability and possible lesion locations in the input ROI image patches.

[0043] In step 4, according to the probability values of the fracture severity grading and the prediction box output by the neural network, the method realizes the fracture localization and severity grading of dual-view DR images.

[0044] In step 5, clinical radiologists and orthopedic surgeons observe the actual DR images, analyze the detection results of spinal fracture features, localize the spinal fracture lesions and grade the severity, and formulate an effective treatment plan after determining the patient's fracture condition.

[0045] Figure 2 It is the framework diagram of the neural network in the method, mainly including network modules such as CNN model, Multi-head transformer and Shifted-window transformer, as well as main links such as Classification, Hybrid patch, and Fracture prediction. According to the DR image and the corresponding annotation information, image patches containing spinal fractures (Fracture patches) are cropped, and together with the normal vertebral body image patches (Normal patches) at adjacent positions, they are used as the input of the CNN model. The predicted classification results are calculated with the true category to obtain the classification loss (Class loss). After the output feature image patches are mixed, they pass through the parallel branch network modules of the Multi-head transformer and Shifted-window transformer for splicing, predicting the location and severity grading of the spinal fracture lesions, and calculating the location loss function (Location loss) from the localization coordinates and the true annotation.

[0046] Figure 3The results of spinal fracture localization and severity grading obtained by the neural network in the method.

[0047] The left figure is the anteroposterior DR image: Grade 1 indicates a mild spinal fracture with a surface loss of 10% - 20%; Grade 3 indicates a severe spinal fracture with a surface loss exceeding 40%.

[0048] The right figure is the lateral DR image: Grade 1 indicates a mild spinal fracture with a height loss of 20% - 25%; Grade 3 indicates a severe spinal fracture with a height loss exceeding 40%.

[0049] Effect evaluation:

[0050] The present method invents a method for detecting spinal fracture features based on dual-view DR images, providing an effective method for detecting spinal fracture features for the clinical screening and early treatment of patients with pathological and osteoporotic vertebral compression fractures.

[0051] It should be noted that the above embodiments are only preferred embodiments of the present invention and do not limit the protection scope of the present invention. Equivalent substitutions or replacements made on the basis of the above technical solutions all fall within the protection scope of the present invention.

Claims

1. A method for detecting spinal fracture features from dual-view DR images, characterized in that, The specific steps are as follows: Step 1: Collect images of spinal fracture patients from DR devices and preprocess the image data; Step 2: Use the preprocessed images and the annotations of radiologists with rich clinical experience to obtain the regions of interest (ROIs) of different vertebrae in the anteroposterior and lateral dual-angle DR plain films; Step 3: Input the ROI image patches into the designed neural network. After passing through the pre-trained model CNNmodel trained on the ImageNet dataset, perform image patch feature mixing, then input them into the Multi-head transformer and Shifted-window transformer network modules respectively for feature splicing, and finally obtain the prediction results of spinal fractures; Step 4: Based on the fracture severity grading and the probability values of the prediction boxes output by the neural network, fracture localization and severity grading of dual-view DR images are achieved; Step 5: Clinical radiologists or orthopedic surgeons observe the actual DR images and, in combination with the results of spinal fracture feature detection, localize the fracture lesions and grade the severity.

2. The method for detecting spinal fracture features from dual-view DR images according to claim 1, characterized in that, Through median filtering and anisotropic diffusion filtering methods, remove the random noise in the DR images, filter out the noise points that cause blurring of the spinal vertebral margins in the DR images, set the window width and window level of the DR images according to the DICOM format data tags or the original pixel values, map the original pixels to 8-bit images, and perform contrast-limited adaptive histogram equalization for image enhancement to enhance the contrast of the DR images while suppressing noise, making the distinction between the spine and soft tissues more obvious.

3. The method for detecting spinal fracture features from dual-view DR images according to claim 1, characterized in that, According to the DR images and the corresponding annotation information, crop the image patches containing the spinal fracture ROIs, and at the same time obtain the image patches of normal vertebral ROIs from adjacent positions, adjust the image patch size, and assign classification labels of fracture and normal to the image patches.

4. The method for detecting spinal fracture features from dual-view DR images according to claim 1, characterized in that, The specific steps of step 3 are as follows: The ROIs of the anterior-posterior frontal and lateral radiographs include the normal vertebral body image block I N and the fractured vertebral body image block I F , and in the first stage, the feature map f(I N ,I F ) and the predicted classification extracted by the CNNmodel from the ROI image blocks In the second stage, through the feature image block mixing, f(I N ,I F ) with the dimension of H×W×C is converted into the dimension of H / 4×W / 4×2C and input into the Multi-head transformer and the Shifted-window transformer respectively, where H, W, and C represent the height, width, and number of channels of the feature map. In the Multi-head transformer, three 1×1 convolutions project the input feature map onto flatten and transpose it into a sequence of size n×d, where d is the dimension of the embedding. Q, K, and V are flattened and transposed into a sequence of size n×d, where n = H×W, and thus the output is obtained. This enables the model to jointly infer the attention from different representation subspaces for fracture localization. In the Shifted-window transformer, the self-attention of the feature map is calculated within the shifted local windows, and the windows are evenly divided in a non-overlapping manner. The output is where B is the shift bias. Concatenate O MHT and O SWT to obtain f cat = Cancate(O MHT ,O SWT ). Further, it can be known that the fracture position t = (t x ,t y ,t w ,t h ) predicted by the neural network and the localization loss function where (x, y, w, h) are the horizontal and vertical coordinates, width, and height of the predicted bounding box, and smooth L1 is the smooth L1 loss function. Adopting the anteroposterior and lateral dual-view DR images and designing the parallel network modules of Multi-head transformer and Shifted-window transformer are the unique features of this method. Through the neural network training and optimization process, the method can detect fracture features in DR images and obtain the lesion locations and severity grades of fractures.

Images