Lumbar spine abnormal image recognition method and system

Abstract

The present application relates to lumbar abnormal image recognition method and system, wherein the lumbar abnormal image recognition method comprises the following steps: acquiring the lumbar region MRI image, positioning the left and right apices of the intervertebral disc and the posterior apices of the spinal canal as key points; determining the region of interest in the lumbar image according to the positioned key points, and obtaining the shape image of the lumbar spinal canal through pretreatment; inputting the obtained lumbar spinal canal shape image into the trained classification network model, and judging whether the image contains the abnormal shape of lumbar spinal canal stenosis. By positioning the key points, the region of the lumbar spinal canal is roughly determined, and the most interesting lumbar center region in machine learning is sent into the classification network. Compared with the existing technology of sending the whole MRI image into the classification neural network, the irrelevant information in the network input can be reduced, the features related to the spinal canal stenosis can be more focused on, the abnormal shape image of the lumbar spinal canal stenosis existing in the image data can be accurately recognized, and the recognition accuracy is improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to a method and system for recognizing lumbar spine abnormalities. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Images of the lumbar spine reflect the structure of the human lower back. When abnormalities occur in the lumbar spine, symptoms such as lower back pain may appear. Correspondingly, the images will show an incorrect shape in a certain part of the lumbar spine. For example, lumbar degeneration leading to joint hyperplasia and ligament hypertrophy can compress the normal volume of the spinal canal. This morphological change is called spinal stenosis, which is a type of lumbar spine abnormality.

[0004] There are two main existing methods for identifying abnormal lumbar spine images: traditional image processing algorithms and machine learning methods such as deep neural networks. Traditional image processing algorithms are greatly affected by objective factors such as environment, shooting angle, and individual differences, resulting in poor robustness and difficulty in adapting to various environments. Deep learning methods, on the other hand, typically feed the entire MRI image directly into the network for classification. The entire MRI image contains a large amount of information about the lumbar structure, and abnormal information in the spinal canal is superimposed with abnormal information in other parts, leading to low accuracy. Summary of the Invention

[0005] To address the technical problems mentioned above, this invention provides a method and system for recognizing abnormal images of the lumbar spine. The method involves locating key points in an input lumbar MRI image of the lumbar region by identifying the left and right vertebrae of the intervertebral discs and the posterior vertebrae of the spinal canal; determining the ROI region of the lumbar spine based on the located key points; processing the ROI region using image processing techniques such as affine transformation; and then feeding the processed ROI image into a classification network model to identify whether the image contains abnormal morphology of lumbar spinal stenosis.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] The first aspect of the present invention provides a method for image recognition of lumbar spine abnormalities, comprising the following steps:

[0008] Acquire MRI images of the lumbar region and locate the left and right apexes of the lumbar intervertebral discs and the posterior apex of the spinal canal as key points;

[0009] Based on the key points of localization, the region of interest in the lumbar spine image is determined, and the morphological image of the lumbar spinal canal is obtained after preprocessing;

[0010] The obtained lumbar spinal canal morphology image is input into the trained classification network model to determine whether the image contains abnormal morphology of lumbar spinal canal stenosis.

[0011] MRI images of the lumbar spine region were acquired, and the left and right vertebrae of the intervertebral discs and the posterior vertebrae of the spinal canal were located as keypoints based on the trained keypoint localization network, including:

[0012] Pre-label the left and right apexes of the lumbar intervertebral discs and the posterior apex of the spinal canal in lumbar MRI images;

[0013] The annotation results are converted into a format and a heatmap with annotation results is output. The heatmap is used as a label to train the key point localization network.

[0014] Using lumbar MRI images as input, key points are output through a key point localization network. All output key points occupy a single dimension of the network, thus obtaining the desired key points.

[0015] The region of interest in the lumbar spine image is determined based on the key points of localization, including:

[0016] Find at least three regions of maximum value in the network output as three key points;

[0017] The obtained key points are sorted according to their coordinate values ​​to locate the left and right vertices of the lumbar intervertebral disc and the posterior vertices of the spinal canal. Specifically, among the two points with larger y values, the one with the smaller x value is the left vertices of the lumbar intervertebral disc, denoted as point 1; the one with the smaller x value is the right vertices of the lumbar intervertebral disc, denoted as point 2; the remaining point with the smallest y value is the posterior vertices of the spinal canal, denoted as point 3; the y value is the vertical axis coordinate, and the x value is the horizontal axis coordinate.

[0018] Based on the coordinates of the key points, the length, width, center point coordinates, and rotation angle of the region of interest in the lumbar spinal canal can be obtained.

[0019] The morphological image of the lumbar spinal canal obtained after preprocessing includes:

[0020] Based on the coordinates of the center point of the region of interest in the lumbar spinal canal, the image is straightened through affine transformation;

[0021] The affine transformation image is extracted as the output image to obtain the lumbar spinal canal morphology image.

[0022] A second aspect of the present invention provides a system for implementing the above-described method, comprising:

[0023] The key point detection module is configured to: acquire MRI images of the lumbar region and locate the left and right vertebrae of the lumbar intervertebral disc and the posterior vertebrae of the spinal canal as key points;

[0024] The region of interest segmentation module is configured to: determine the region of interest in the lumbar spine image based on the key points of the location, and obtain the morphological image of the lumbar spinal canal after preprocessing;

[0025] The classification prediction module is configured to input the obtained lumbar spinal canal morphology image into the trained classification network model and determine whether the image contains abnormal morphology of lumbar spinal canal stenosis.

[0026] A third aspect of the present invention provides a computer-readable storage medium.

[0027] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the lumbar spine abnormality image recognition method described above.

[0028] A fourth aspect of the present invention provides a computer device.

[0029] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps in the lumbar spine abnormality image recognition method described above.

[0030] Compared with existing technologies, one or more of the above technical solutions have the following beneficial effects:

[0031] By locating key points to roughly determine the region of the lumbar spinal canal, the lumbar central region of most interest in machine learning is fed into the classification network. Compared with the existing technology of feeding the entire MRI image into the classification neural network, this can reduce irrelevant information in the network input and focus more on extracting features related to spinal stenosis. It can accurately identify abnormal morphological images of lumbar spinal stenosis in the image data, thereby improving the accuracy of recognition. Attached Figure Description

[0032] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0033] Figure 1 This is a schematic diagram of the lumbar spine abnormality image recognition process provided in one or more embodiments of the present invention;

[0034] Figure 2 This is a schematic diagram of the left and right apexes of the lumbar intervertebral disc and the posterior apex of the spinal canal marked in a lumbar spinal MRI image provided by one or more embodiments of the present invention;

[0035] Figure 3 This is a Gaussian heatmap based on annotation results provided by one or more embodiments of the present invention;

[0036] Figure 4 This is a schematic diagram of the results after key point detection provided by one or more embodiments of the present invention;

[0037] Figure 5This is a schematic diagram of the ROI region determined based on key points, provided by one or more embodiments of the present invention;

[0038] Figures 6(a)-6(d) These are all schematic diagrams of ROI regions provided by one or more embodiments of the present invention after affine transformation and subsequent cropping;

[0039] Figure 7 This is an example of annotation prior to classification prediction provided by one or more embodiments of the present invention. Detailed Implementation

[0040] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0041] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0042] The ROI (Region of Interest) is a region of interest defined in machine vision and image processing. It is a region of interest defined by outlining the area to be processed within an image using shapes such as rectangles, circles, ellipses, and irregular polygons.

[0043] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0044] As described in the background section, due to the complex structure of the human lumbar region and the superposition of multiple abnormal morphologies, current deep learning methods for identifying lumbar spine abnormalities often suffer from problems such as excessive irrelevant pixel features and unclear key features when directly inputting the entire image into the classification network for recognition, since the lumbar spinal canal only occupies a small part of the MRI image. This results in low accuracy.

[0045] Therefore, the following embodiments provide a method and system for lumbar spine abnormality image recognition. For the input lumbar spine region MRI image, a key point detection model is used to locate the left and right vertebrae of the lumbar intervertebral disc and the posterior vertebrae of the spinal canal. The located key points determine the lumbar spine ROI region, and image processing techniques such as affine transformation are used to process the ROI region. The processed ROI image is then fed into a classification network model for diagnosis to determine whether the image contains abnormal morphology of lumbar spinal canal stenosis.

[0046] Example 1:

[0047] like Figure 1 The lumbar spine abnormality image recognition method shown includes the following steps:

[0048] Acquire MRI images of the lumbar region and locate the left and right apexes of the lumbar intervertebral discs and the posterior apex of the spinal canal as key points;

[0049] Based on the key points of localization, the region of interest in the lumbar spine image is determined, and the morphological image of the lumbar spinal canal is obtained after preprocessing;

[0050] The obtained lumbar spinal canal morphology image is input into the trained classification network model to determine whether the image contains abnormal morphology of lumbar spinal canal stenosis.

[0051] Specifically:

[0052] Part 1: Key point localization based on Unet network. Key point localization is to find the approximate location of the lumbar spinal canal in the input lumbar MRI image, in order to prepare for subsequent ROI region division. This embodiment is achieved by using a trained segmentation network (Unet network) in conjunction with pre-annotated lumbar MRI images.

[0053] Acquire lumbar MRI images and label the left and right apexes of the lumbar intervertebral discs and the posterior apex of the spinal canal in the lumbar MRI images. The three labeled points are as follows: Figure 2 As shown.

[0054] After labeling, a Python script is used to preprocess the labeled tags, converting them into a format usable by the network. The network requires tags in heatmap format: in this example, a 1×512×512 image. In the heatmap, if the coordinate point is a key point, the output is 1; otherwise, the output is 0.

[0055] In localization tasks, it is difficult to accurately locate a target point to a specific pixel location. If points surrounding a pixel are directly defined as negative samples, it may interfere with network training. To address this issue, this embodiment uses a Gaussian heatmap for labeling. Assuming the keypoint coordinates are P = (x0, y0), then the value of a point (x, y) on the Gaussian heatmap is:

[0056]

[0057] Where, σ 2 The variance is the Gaussian distribution. Using a Gaussian heatmap can improve network convergence. In this embodiment, the Gaussian heatmap is as follows: Figure 3 As shown.

[0058] A keypoint localization network was selected. The Unet network, which performs well in the medical field, was used, and its output layer was modified. The input size remained the same as the original network (N×3×512×512), while the output was changed to N×1×512×512, where N is the number of input images. The network output has a dimension of 1. Compared to the method where each keypoint occupies one dimension, this embodiment uses a method where all keypoints share one dimension. Experiments have shown that this results in better model convergence and more accurate keypoint localization. The network model output is shown below. Figure 4 As shown, the black dots represent the network model output, and the white dots represent the labeled areas. The labeled areas surround the output.

[0059] Part 2: Image processing techniques are used to process key points and extract Regions of Interest (ROIs). The extracted ROIs contain the lumbar spinal canal. After the key point locations are determined, the coordinate values ​​of the key points are used to determine the parameters of the ROIs, thereby obtaining a lumbar spinal canal morphology image centered on the lumbar spinal canal, which facilitates subsequent classification and prediction.

[0060] I. Keypoint Finding. Three maximum value regions are found in the output of the Unet network to obtain three keypoints. The specific algorithm formula is as follows:

[0061] 1. Let the output result be heatmap(x,y).

[0062] 2. Find the maximum value point P on the output heatmap. max =(x max ,y max ).

[0063] 3. Define the output result: P(x,y)=min(heatmap);

[0064] Where, x∈[x max -σ,x max +σ],y∈[y max -σ,y max +σ].

[0065] 4. Go back to step two and repeat the process three times to find three key points.

[0066] II. Sorting of Key Points. The three key points of the lumbar spinal canal found need to be further sorted. The key points obtained in step two are sorted in ascending order of their y-values. Among the two points with larger y-values, the one with the smaller x-value is the left vertex of the lumbar intervertebral disc, denoted as point 1; the one with the smaller x-value is the right vertex of the lumbar intervertebral disc, denoted as point 2; the remaining point with the smallest y-value is the posterior vertex of the spinal canal, denoted as point 3.

[0067] 3. Calculate ROI parameters. The length, width, center point coordinates, and rotation angle of the lumbar spinal canal ROI region need to be calculated. This prepares for subsequent affine transformations.

[0068] 1. Take point P1(x1,y1) and point P2(x2,y2) as points 1 and 2 respectively. Draw a straight line through the two points, denoted as L1:y=k1·x+b2.

[0069] in:

[0070] 2. From k1, we can obtain the angle of image rotation θ = arctan(k1).

[0071] 3. Given the equation of line L1 and the coordinates of point 3 P3(x3,y3), the width and height of the ROI region can be calculated.

[0072] in

[0073] 4. Calculate the midpoint P between points 1 and 2. mid1 (x mid1 ,y mid1 ).

[0074] in Then calculate the value passing through point P. mid1 (x mid1 ,y mid1 The equation of the line perpendicular to L1 is L2: y = k²·x + b². b2 = y mid -k2·x mid The coordinates of the center point P can be easily obtained based on the distance and the equation of the straight line. center (x c ,y c ).

[0075] By obtaining the coordinates of the center point, the width, height, and angle of the image, the Region of Interest (ROI) in the center of the lumbar spinal canal can be located. For example... Figure 5 As shown.

[0076] IV. Affine Transformation to Straighten the Image. In step three, the obtained ROI region is defined by P... center (x c ,y c The image is a rectangular region centered at point θ, rotated at an angle θ. To allow the image to be fed into the subsequent classification neural network for classification and diagnosis, an affine transformation needs to be performed on the image to straighten the ROI region.

[0077] Affine transformation matrix Then the transformed image position matrix M output =M·M input .

[0078] Then cut This can be used as the output image. Multiplying by 1.2 in the formula expands the region outwards by a certain size, preserving more edge information and preventing insufficient cropping from affecting the subsequent classification and diagnostic network performance. The cropped image is shown below. Figures 6(a)-6(d) As shown.

[0079] Part 3: Classification Prediction Using the EfficientNet Network

[0080] EfficientNet is a deep neural network that takes a stretched image and feeds it into a well-trained neural network. It can automatically output the probability that the image is lumbar spinal stenosis and determine whether the image is an abnormal morphology of spinal stenosis based on the probability value (greater than 0.5).

[0081] EfficientNet is a supervised learning model that relies on manually labeled data during training. The network outputs two probability values: lumbar spinal stenosis and no lumbar spinal stenosis. During training, a label is automatically generated for each image based on the pre-labeled data: [0,1] for stenosis and [1,0] for no stenosis. The straightened image is then fed into the network, which updates its parameters using gradient descent based on the corresponding label, thus completing the training process.

[0082] Specifically:

[0083] 1. Beforehand, doctors and other professionals from relevant departments labeled the lumbar MRI images to indicate the presence or absence of lumbar spinal stenosis. MRI images showing lumbar spinal stenosis were labeled with 1, while those not showing stenosis were labeled with 0. The output images from the second part, combined with the labels, were used to train the EfficientNet network. Due to the relatively low image resolution and to achieve real-time detection speed, this embodiment uses the EfficientNet-B0 classification neural network. An example of data labeling is shown below. Figure 7 As shown.

[0084] 2. During the detection process, the lumbar spinal canal ROI image output from the second part, after straightening, is resized to 224×224 and then fed into the EfficientNet classification network for prediction. The output of the classification network is the final case judgment result. A result of 0 indicates the absence of lumbar spinal stenosis, while a result of 1 indicates the presence of lumbar spinal stenosis.

[0085] Compared to existing technologies that feed the entire MRI image into a classification neural network, this invention roughly determines the region of the lumbar spinal canal by locating key points and only feeds the central lumbar region of interest to the machine learning into the classification network. This reduces irrelevant information in the network input and allows for a greater focus on extracting features related to spinal stenosis. This enables accurate identification of abnormal morphological images related to lumbar spinal stenosis in the image data, thereby improving the accuracy of identification.

[0086] The method achieved a detection accuracy of 95% when tested on the test set.

[0087] Example 2:

[0088] A system for implementing the above method includes:

[0089] The key point detection module is configured to: acquire MRI images of the lumbar region and locate the left and right vertebrae of the lumbar intervertebral disc and the posterior vertebrae of the spinal canal as key points;

[0090] The region of interest segmentation module is configured to: determine the region of interest in the lumbar spine image based on the key points of the location, and obtain the morphological image of the lumbar spinal canal after preprocessing;

[0091] The classification prediction module is configured to input the obtained lumbar spinal canal morphology image into the trained classification network model and determine whether the image contains abnormal morphology of lumbar spinal canal stenosis.

[0092] Example 3:

[0093] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the lumbar spine abnormality image recognition method described in Embodiment 1 above.

[0094] Example 4:

[0095] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the lumbar spine abnormality image recognition method described in Embodiment 1 above.

[0096] The steps or modules involved in Embodiments 2 to 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0097] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A lumbar abnormality image recognition method, characterized by, The method comprises the following steps: Obtain an MRI image of a lumbar region, and locate left and right apices of a lumbar intervertebral disc and a posterior apex of a spinal canal as key points; Determine a region of interest in the lumbar image according to the located key points, and obtain a morphological image of the lumbar spinal canal through preprocessing; The determination of the region of interest in the lumbar image according to the located key points comprises: Find at least three maximum value regions on the output of the network as three key points; Sort the obtained key points according to coordinate values, and locate the left and right apices of the lumbar intervertebral disc and the posterior apex of the spinal canal; Obtain a length, a width, a center point coordinate and a rotation angle of the region of interest of the lumbar spinal canal according to the coordinate values of the key points; Input the obtained morphological image of the lumbar spinal canal into a trained classification network model, and determine whether the image contains an abnormal morphology of lumbar spinal canal stenosis.

2. The lumbar abnormality image recognition method according to claim 1, wherein Obtain an MRI image of a lumbar region, and locate left and right apices of a lumbar intervertebral disc and a posterior apex of a spinal canal as key points based on a trained key point positioning network.

3. The lumbar abnormality image recognition method according to claim 2, wherein The locating of the left and right apices of the lumbar intervertebral disc and the posterior apex of the spinal canal as key points based on the trained key point positioning network comprises: Pre-label the left and right apices of the lumbar intervertebral disc and the posterior apex of the spinal canal in the MRI image of the lumbar region; Convert the labeling result into a format, output a heat map with the labeling result, and use the heat map as a label to train the key point positioning network; Use the MRI image of the lumbar region as input, output the key points through the key point positioning network, and obtain the required key points, wherein all the output key points jointly occupy a dimension of the network.

4. The lumbar abnormality image recognition method of claim 1, wherein, Sort the obtained key points according to coordinate values, and locate the left and right apices of the lumbar intervertebral disc and the posterior apex of the spinal canal; specifically: In the first two points with larger y values, the one with smaller x value is the left apex of the lumbar intervertebral disc, denoted as point No. 1; the one with smaller x value is the right apex of the lumbar intervertebral disc, denoted as point No. 2; and the point with the smallest y value is the posterior apex of the spinal canal, denoted as point No. 3; the y value is a vertical axis coordinate, and the x value is a horizontal axis coordinate.

5. The lumbar abnormality image recognition method according to claim 1, wherein The morphological image of the lumbar spinal canal obtained through preprocessing comprises: performing affine transformation to correct the image according to the center point coordinate of the region of interest of the lumbar spinal canal.

6. The lumbar abnormality image recognition method according to claim 5, wherein The morphological image of the lumbar spinal canal obtained through preprocessing further comprises: cutting the image after the affine transformation as an output image to obtain the morphological image of the lumbar spinal canal.

7. A lumbar abnormality image recognition system characterized by comprising: The method comprises: A key point detection module is configured to obtain an MRI image of a lumbar region, and locate left and right apices of a lumbar intervertebral disc and a posterior apex of a spinal canal as key points; A region of interest division module is configured to determine a region of interest in the lumbar image according to the located key points, and obtain a morphological image of the lumbar spinal canal through preprocessing; The determination of the region of interest in the lumbar image according to the located key points comprises: Find at least three maximum value regions on the output of the network as three key points; Sort the obtained key points according to coordinate values, and locate the left and right apices of the lumbar intervertebral disc and the posterior apex of the spinal canal; Obtain a length, a width, a center point coordinate and a rotation angle of the region of interest of the lumbar spinal canal according to the coordinate values of the key points; A classification prediction module is configured to input the obtained morphological image of the lumbar spinal canal into a trained classification network model, and determine whether the image contains an abnormal morphology of lumbar spinal canal stenosis.

8. A computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the lumbar abnormality image recognition method according to any one of claims 1 to 6.

9. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the lumbar abnormality image recognition method according to any one of claims 1 to 6 when executing the program.

Images