Spinal ligament ossification image recognition method and system

Abstract

The present application relates to a spinal ligament ossification image recognition method and system, comprising the following steps: acquiring a CT image of a set region of the spine, positioning key points in the set region of the spine, and determining the region where the ligament is located; determining the position of the reference line in the region where the ligament is located according to the positioned key points, and intercepting a region of interest containing the reference line to obtain a binary image after preprocessing; based on the obtained binary image, according to the reference line and the pixel value distribution on both sides, the case where the reference line passes through the spinal skeleton part is removed, and the ligament ossification region in the image is recognized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a spinal ligament ossification image recognition method and system. BACKGROUND

[0002] The statements in this section merely provide background information related to the present application and do not necessarily constitute the prior art.

[0003] There are long ligaments in the human spine to strengthen the stability of each vertebra, and the ligaments will ossify due to excessive fatigue or aging, thereby compressing the spinal cord and nerves.

[0004] Ligament ossification usually requires medical imaging and medical judgment criteria, and although the prior art can replace manual identification of whether the spinal ligament in the image has ossification morphology through machine learning or image detection, due to the complex structure of the human spine and the influence of objective factors such as environment, shooting angle and individual differences, the baseline for identifying ossification in the algorithm is easy to pass through the spinal bone, causing misjudgment and interfering with the final recognition result, resulting in low accuracy. SUMMARY

[0005] In order to solve the technical problems existing in the background art, the present application provides a spinal ligament ossification image recognition method and system, which roughly determines the region of the ligament in the spine by key point positioning, and after the region of interest is intercepted, the bone and ossification part in the image is obtained by binary processing, the misjudgment influence of the image itself is eliminated by using the distribution of pixel values on both sides of the baseline, which can reduce irrelevant information in network input and focus more on extracting features related to ligament ossification, and can accurately identify images with ligament ossification morphology in image data, thereby improving the accuracy of recognition.

[0006] In order to achieve the above purpose, the present application adopts the following technical scheme:

[0007] The first aspect of the present application provides a spinal ligament ossification image recognition method, comprising the following steps:

[0008] Obtain the CT image of the set region of the spine, position the key points in the set region of the spine, and determine the region of the ligament;

[0009] According to the position of the key points, the position of the baseline in the region of the ligament is determined, and the region of interest containing the baseline is intercepted, and a binary image is obtained after preprocessing;

[0010] Based on the obtained binary image, according to the pixel value distribution on both sides of the baseline, the case that the baseline passes through the spinal bone part is eliminated, and the ligament ossification region in the image is identified.

[0011] The CT image of the set region of the spine is acquired, and the key points in the set region of the spine are located based on the trained key point positioning network, comprising:

[0012] The key bone points in the CT image of the set region of the spine are pre-labeled;

[0013] The label result conversion format is converted, and the heat map with the label result is outputted, and the heat map is used as a label to train the key point positioning network;

[0014] The CT image of the set region of the spine is used as input, the key points are outputted through the key point positioning network, all the outputted key points jointly occupy a network dimension, and the required key points are obtained.

[0015] The position of the reference line in the ligament region is determined according to the located key points, comprising:

[0016] At least four maximum value regions are found on the output of the network, as the corresponding four key points, and the obtained key points are sorted according to the size of the vertical axis coordinates, specifically: the two points with smaller y values are selected, the point with smaller x value is the No. 1 point, and the point with larger x value is the No. 2 point; the two points with larger y values are selected, the point with smaller x value is the No. 3 point, and the point with larger x value is the No. 4 point; wherein, the y value is the vertical axis coordinate value, and the x value is the horizontal axis coordinate value.

[0017] The midpoint of the No. 1 point and the No. 2 point, and the midpoint of the No. 3 point and the No. 4 point are taken, and the two midpoints are connected, which is the reference line.

[0018] Based on the obtained binary image, according to the reference line and the pixel value distribution on both sides, the case that the reference line passes through the spine skeleton part is removed, and the ligament ossification region in the image is identified; comprising:

[0019] If there is a white pixel region on the reference line, it is suspected to be ossified, that is, the reference line may pass through the skeleton part of the spine or the ossification region of the ligament; if not, it is an image without suspected ossification;

[0020] When suspected to be ossified, it is judged whether the remaining pixels in the N grids on the left and right sides of the white pixel region in the paragraph of the reference line are continuous white pixels;

[0021] If the white pixels on the left side are not continuous and the white pixels on the right side are continuous, the reference line passes through the skeleton part but not the ossification part, which needs to be removed; otherwise, the region passed through by the reference line is the ossification region of the ligament, and the image is outputted as the identification result of the ligament ossification form image.

[0022] The second aspect of the present application provides a system required for realizing the above-mentioned method, comprising:

[0023] The key point detection module is configured to: acquire a CT image of a set region of a spine, locate a key point in the set region of the spine, and determine a region where a ligament is located;

[0024] The image preprocessing module is configured to: determine a position of a reference line in the region where the ligament is located according to the located key point, and intercept a region of interest containing the reference line to obtain a binary image after preprocessing.

[0025] The ossification region identification module is configured to: based on the obtained binary image, according to a pixel value distribution condition on both sides of the reference line, eliminate a case where the reference line passes through a spine skeleton part, and identify a ligament ossification region in the image.

[0026] The third aspect of the present application provides a computer readable storage medium.

[0027] A computer readable storage medium, which stores a computer program, the program being executed by a processor to implement the steps in the spinal ligament ossification image identification method described above.

[0028] The fourth aspect of the present application provides a computer device.

[0029] A computer device, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor executing the program to implement the steps in the spinal ligament ossification image identification method described above.

[0030] Compared with the prior art, the above one or more technical solutions have the following beneficial effects:

[0031] By using the CT image of the set region of the spine, the approximate region of the ligament in the image is determined through key point positioning, the skeleton part and the ossification part in the image are obtained through binary processing after the region of interest is intercepted, the false judgment influence caused by the image itself is eliminated by using the distribution condition of the pixel values on both sides of the reference line, so as to distinguish the skeleton part and the ossification part passed through by the reference line, the irrelevant information in the network input can be reduced, more attention can be paid to extracting the features related to the ligament ossification, the image with the ligament ossification form in the image data can be accurately identified, and the accuracy of the identification is improved. BRIEF DESCRIPTION OF DRAWINGS

[0032] The drawings accompanying the specification of the present application form a part of the present application and serve to provide a further understanding of the present application, the illustrative embodiments of the present application and their descriptions serve to explain the present application, and do not constitute an improper limitation on the present application.

[0033] Figure 1 is a spinal ligament ossification image identification flowchart provided by one or more embodiments of the present application;

[0034] Figure 2 is a key point annotation diagram provided by one or more embodiments of the present application in a cervical vertebra CT image;

[0035] Figure 3 is a Gaussian heat map based on an annotation result provided by one or more embodiments of the present application;

[0036] Figure 4 is a result diagram after key point detection provided by one or more embodiments of the present application;

[0037] Figure 5 is a reference line (K line) diagram determined according to a key point provided by one or more embodiments of the present application;

[0038] Figure 6 is a diagram after ROI region binarization provided by one or more embodiments of the present application;

[0039] Figure 7 is a diagram of a cervical vertebra bone passing through a reference line (K line) provided by one or more embodiments of the present application;

[0040] Figure 8 and Figure 9 are both diagrams for judging ossification results provided by one or more embodiments of the present application. DETAILED DESCRIPTION

[0041] The present application is further described below in conjunction with the accompanying drawings and embodiments.

[0042] It should be noted that the following detailed description is exemplary in nature and is intended to provide further description of the present application. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs.

[0043] The ROI region refers to a region of interest. In machine vision and image processing, a region that needs to be processed is outlined in a box, circle, ellipse, and irregular polygon from the processed image, which is called a region of interest.

[0044] It should be noted that the terms used herein are only for the purpose of describing specific embodiments, and are not intended to limit the exemplary embodiments according to the present application. As used herein, the singular form is intended to include the plural form unless the context clearly indicates otherwise, and furthermore, it should be understood that when the terms "comprise" and / or "include" are used in the specification, they indicate the presence of a feature, step, operation, device, component, and / or combination thereof.

[0045] An adult spine has 7 cervical vertebrae (C1-C7), 12 thoracic vertebrae (T1-T12), 5 lumbar vertebrae (L1-L5), 1 sacrum and 1 coccyx, which are connected by ligaments, joints and intervertebral discs. The ligaments can be ossified due to overfatigue or aging, thereby compressing the spinal cord and nerves. The existing technology is not ideal in identifying the ossification morphology of the ligament, and cannot distinguish whether the baseline passes through the ossification area of the ligament or the bone area of a part of the spine.

[0046] The following embodiment gives a spine ligament ossification image recognition method and system. Taking the cervical vertebrae as an example, the input cervical vertebrae CT image is processed by using a neural network model to locate the cervical vertebrae bone C2 and C7 key points and take the line connecting the C2 and C7 midpoints as the K line (baseline). After the image is binarized, the ROI region is intercepted to obtain the ossification part in the cervical vertebrae and ligament. The cervical vertebrae image with the K line is excluded to avoid misjudgment. Then it is judged whether the ossification part reaches the K line. If it reaches, it is negative (there is an ossification part in the ligament), otherwise it is positive (there is no ossification part in the ligament).

[0047] Embodiment one:

[0048] As shown in the spine ligament ossification image recognition method, the following steps are included: Figure 1 Obtain the CT image of the set region of the spine, locate the key points in the set region of the spine, and determine the region where the ligament is located;

[0049] According to the position of the baseline in the region where the ligament is located determined by the located key points, the region of interest containing the baseline is intercepted, and the binarized image is obtained after preprocessing;

[0050] Based on the obtained binarized image, according to the baseline and the pixel value distribution on both sides, the case where the baseline passes through the spine bone part is excluded, and the ligament ossification region in the image is recognized.

[0051] Specifically:

[0052] First part: based on Unet network to locate key points, the key point positioning is to find the approximate position of the cervical ligament in the input cervical image, so as to determine the position of the baseline (K line); this embodiment is realized by using the trained segmentation network (Unet network) and the pre-labeled image.

[0053] The C2 and C7 key bone points in the cervical vertebrae CT image are labeled in advance, a total of 4 points, as shown in

[0054] Figure 2

[0055] ​​After labeling, the labeled labels are preprocessed using a python script to convert the labels into a format that can be used by the network. The label required by the network is in the form of a heatmap: in this embodiment, a 1x512x512 image. In the heatmap, if the coordinate point is a key point, the output is 1, otherwise the output is 0.

[0056] In the positioning task, it is difficult to accurately position the target point to a certain pixel position, that is, it is difficult to accurately position. If the points around the pixel point are directly defined as negative samples, it may interfere with the training of the network. In order to solve this problem, the embodiment uses a Gaussian heatmap method for the label. Assuming that the labeled key point coordinates are P=(x0, y0), the value of a point (x, y) on the Gaussian heatmap is:

[0057]

[0058] where σ 2 is the variance of the Gaussian distribution. Using a Gaussian heatmap can make the network better converge. In this embodiment, the Gaussian heatmap is as shown in Figure 3 .

[0059] Select the key point positioning network. The key point positioning network uses the Unet network which performs well in the medical industry, and modifies the output layer of the Unet network. The input size of the network remains the original network size of Nx3x512x512, and the output is changed to Nxlx512x512, where N is the number of input images. The output dimension of the network is 1, compared with the method of one key point occupying one dimension, this embodiment adopts the method of all key points occupying one dimension, which can make the model better converge and more accurately locate the key points. The output of the network model is as shown in Figure 4 , wherein the area formed by the white points is the output result of the network model, and the black points are the labeling results. The area where the labeling results are located surrounds the output result. In actual application, both the model output result and the labeling result are colored, and this embodiment uses black and white as an example for explanation.

[0060] Second part: using image processing technology to process key points and output recognition results.

[0061] I. Key point search. Find 4 maximum value regions on the output of the Unet network to get 4 key points. The specific algorithm formula is as follows:

[0062] 1 Let the output result be heatmap(x, y).

[0063] 2 Find the maximum value point P max =(x max ,y max ) on the output heatmap.

[0064] 3 Define the output result: P(x, y) = min(heatmap);

[0065] Wherein, x ∈ [x max -σ, x max +σ], y ∈ [y max -σ, y max +σ].

[0066] 4 Back to the second step, loop 4 times in turn, find 4 key points.

[0067] Two, key point sorting. The four cervical vertebra key points found need further sorting work. Set from the left point of C2 cervical vertebra to the right point of C7 cervical vertebra, the label is 1 to 4 in turn. Four points are sorted by the size of y coordinate value. First select the two points with smaller y value, the point with smaller x value in the two points is No. 1 point, and the larger one is No. 2 point; Then select the remaining two points with larger y value, the one with smaller x value is No. 3 point, and the one with larger x value is No. 4 point.

[0068] Here, left side refers to the front side of the human body, and right side refers to the back side of the human body. The cervical vertebra CT image selected in this embodiment is a cross section under the side view angle of the human cervical vertebra part.

[0069] Three, get K line (reference line). Take No. 1 point P1(x1, y1) and No. 2 point P2(x2, y2), make the midpoint P mid1 (x mid1 ,y mid1 ) of the two, the formula is as follows:

[0070]

[0071] Similarly, the midpoint P mid2 (x mid2 ,y mid2 ) of No. 3 point and No. 4 point can be obtained. Select the two midpoints to make a line, which is K line. As shown in Figure 5 .

[0072] Four, image binarization. It is needed to judge whether the degree of cervical vertebra ossification exceeds K line. First, extend the four key points to the surrounding 20 pixel points, and take the ROI region from the original image. Then, the ROI region is subjected to OTSU dynamic threshold binarization algorithm to obtain the binarized image of the ROI region. As shown in Figure 6 .

[0073] Wherein, the ROI region is taken again after binarization processing, in order to send the cervical ligament region most interested in machine learning into the classification network, so as to focus more on extracting the features related to ligament ossification, thereby reducing the irrelevant information in the network input.

[0074] In the binarized ROI image, the bone and ossification part is white with pixel value 255, and the rest is black with pixel value 0. Whether the pixel point where the K-line is located has a pixel value of 255 (white) can determine whether the ligament contains an ossification region.

[0075] V. Excluding bone error. In some images, due to the influence of environment, shooting angle and other factors, the K-line may pass through the cervical vertebra, causing false judgment and interfering with the algorithm result. As shown in Figure 7

[0076] For this situation, the influence of the cervical vertebra crossed by the K-line on the final result needs to be removed before the ossification is judged.

[0077] By observing and comparing the ossification part and the cervical vertebra image causing false judgment, it is found that the right image of the cervical vertebra causing false judgment is often continuous white, and the left side stops extending to black. The right side of the ossification part is often discontinuous white, that is, it extends to the right for a distance, and encounters a black area, and the left side is large area white.

[0078] Therefore, according to the above image features and rules, the embodiment detects the distribution of pixels on both sides of the reference line (K-line). If a white pixel area is detected on the K-line, it is suspected to be ossification. Whether the left and right ends of the pixel in the paragraph are continuous white is determined. If the left side is discontinuous and the right side is continuous, it is considered that this place is the cervical vertebra, which is a false judgment, and this part of the area needs to be excluded. In other cases, it is considered that the ossification area is found.

[0079] Specifically as follows:

[0080] 1. Design two arrays, one array pointslist is used to store the coordinates of the points on the K-line, and the other two-dimensional array pointsvaluelist is used to store the pixel values of the K-line and its left and right N grids. Wherein, N is a parameter, in this embodiment, N is 10.

[0081] 2. Operate the pointsvaluelist array, and create a new array blocks with a size of 2N+1, which is used to record the interval of the point with pixel value 255 along the K-line direction on the K-line and its left and right N grids. Each line blocks[i] has M small intervals, which are used to record the starting point y begin and the end point y end of the white area in the interval.

[0082] ​Query the blocks array. First query blocks[K] that is the block on the K line whether it is empty. If it is empty, it means that there is no interval of points with pixel value 255 (no white area), the K line does not pass through the cervical vertebrae or the ossification area of ligament, that is, the image is correct.

[0083] If it is not empty, it means that there is an interval of points with pixel value 255, the K line may pass through the cervical vertebrae or the ossification area of ligament, that is, there is a suspected ossification area, and further false query is needed. Extend to the left, and query whether there is an interval in the mk-1 intervals of blocks[K-1] that borders any interval in the mk intervals of the previous block blocks[K].

[0084] Here, extending to the left means that, at the pixel level, the array blocks[K] with suspected ossification area is queried in turn blocks[K-1], blocks[K-2]...blocks[K-N], that is, traversed to the left. Similarly, extending to the right is blocks[K+1], blocks[K+2]...blocks[K+N].

[0085] 3. If there is, continue to loop and continue to judge whether the intervals K-2 and K-1 border. Until K-N. If the left N intervals are continuous, record flag left =1. If not, some two adjacent blocks do not border, which means that the left side of the K line is discontinuous, and record flag left =0.

[0086] 4. According to the same method, judge whether the right side of the K line is continuous. If the right side is continuous, record flag right =1, otherwise record flag right =0.

[0087] 5. Judge two flag bits. If flag left =0 and flag right =1, that is, the left is discontinuous and the right is continuous, it is considered that this area is the cervical vertebrae area, not the ossification area, which is a false judgment, that is, the K line passes through the cervical vertebrae but does not pass through the ossification area of ligament.

[0088] Otherwise, it is considered to be an ossification area, and the ossification area of ligament exceeds the K line. As shown in Figure 8 and Figure 9 , Figure 8 The area framed in the K line is the ossification area of cervical ligament, and the K line in this area is marked in color (for example, red) in actual application, Figure 9The region framed on the K-line is the cervical vertebra region, which is a misjudged image that needs to be removed. The K-line of this region is marked in color (for example, blue) in actual application.

[0089] In actual application, the part of the K-line that passes through the ligament ossification region and the part of the K-line that passes through the bone are distinguished by different colors. The present embodiment is only an example that uses black and white.

[0090] Embodiment Two

[0091] A system for implementing the above method comprises:

[0092] A key point detection module configured to: acquire a CT image of a set region of a spine, locate key points in the set region of the spine, and determine a region where a ligament is located;

[0093] An image preprocessing module configured to: determine a position of a reference line in the region where the ligament is located according to the located key points, and intercept a region of interest containing the reference line to obtain a binary image after preprocessing;

[0094] An ossification region identification module configured to: based on the obtained binary image, according to a distribution of pixel values on both sides of the reference line, remove a case where the reference line passes through a part of the spine bone, and identify a ligament ossification region in the image.

[0095] Embodiment Three

[0096] The present embodiment provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the spinal ligament ossification image identification method of Embodiment One.

[0097] Embodiment Four

[0098] The present embodiment provides a computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the spinal ligament ossification image identification method of Embodiment One when executing the program.

[0099] The steps or modules involved in Embodiments Two to Four correspond to Embodiment One, and the specific implementation can be referred to the relevant description of Embodiment One. The term "computer-readable storage medium" should be understood to include a single medium or multiple media of one or more instruction sets; it should also be understood to include any medium that can store, encode, or carry instruction sets for execution by a processor and cause the processor to perform any method in the present application.

[0100] The above merely provides the preferred embodiments of the present application, and is not used to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modifications, equivalent replacements, improvements, etc. made within the principles and technical scope of the present application shall fall into the scope of the present application.

Claims

1. A method of identifying a spondylosis image, characterized by, The method comprises the following steps: Obtain CT images of a set region of a spine, locate key points in the set region of the spine, and determine a region where ligaments are located; Determine the position of a reference line in the region where the ligaments are located according to the located key points, and obtain a binary image after preprocessing by intercepting a region of interest containing the reference line; Find at least four maximum value regions on the output of the network as four corresponding key points, and sort the obtained key points according to the size of the vertical axis coordinates, and determine the position of the reference line according to the sorting result, comprising: Sort according to the size of the y coordinate value, and the y value is the vertical axis coordinate value and the x value is the horizontal axis coordinate value; Select two points with smaller y values, and the point with smaller x value is point No. 1 and the point with larger x value is point No. 2; select the remaining two points with larger y values, and the point with smaller x value is point No. 3 and the point with larger x value is point No. 4; Determine the midpoint of point No. 1 and point No. 2, and the midpoint of point No. 3 and point No. 4, and select the two midpoints as a connecting line, which is the reference line; Based on the obtained binary image, according to the distribution of pixel values on both sides of the reference line, the case where the reference line passes through the spine skeleton part is removed, and the ligament ossification region in the image is identified.

2. The method of identifying a spondylosis image according to claim 1, wherein Obtain CT images of a set region of a spine, locate key points in the set region of the spine based on a trained key point positioning network, comprising: Pre-label key bone points in the CT images of the set region of the spine; Convert the labeling result format, output a heat map with the labeling result, and use the heat map as a label to train the key point positioning network; Use the CT images of the set region of the spine as input, and output the key points through the key point positioning network, all the output key points occupy a network dimension, and the required key points are obtained.

3. The method of identifying a spondylosis image according to claim 1, wherein Based on the obtained binary image, according to the distribution of pixel values on both sides of the reference line, the case where the reference line passes through the spine skeleton part is removed, and the ligament ossification region in the image is identified; comprising: If there is a white pixel region on the reference line, it is suspected to be ossified, that is, the reference line may pass through the bone part of the spine or the ossification region of the ligament; if not, it is an image without suspected ossification.

4. The method of identifying a spondylosis image according to claim 1, wherein Based on the obtained binary image, according to the distribution of pixel values on both sides of the reference line, the case where the reference line passes through the spine skeleton part is removed, and the ligament ossification region in the image is identified; further comprising: When suspected ossification, judge whether the remaining pixels in the N grids on the left and right sides of the white pixel region segment on the reference line are continuous white pixels; If the white pixels on the left are not continuous but the white pixels on the right are continuous, the reference line passes through the bone part but not the ossification part, which needs to be removed; otherwise, the region passed through by the reference line is the ossification region of the ligament.

5. The method of identifying a vertebral ligament ossification image according to claim 4, wherein When the region passed through by the reference line is the ossification region of the ligament, output the image as the identification result of the image containing the ligament ossification form.

6. A vertebral ligament ossification image recognition system characterized by, Comprise The key point detection module is configured to obtain CT images of a set region of a spine, locate key points in the set region of the spine, and determine a region where ligaments are located; An image preprocessing module is configured to determine the position of the reference line in the region where the ligament is located according to the located key points, and to intercept a region of interest containing the reference line, to obtain a binary image after preprocessing. Finding at least four maximum value regions on the output of the network as the corresponding four key points, and sorting the obtained key points according to the size of the vertical axis coordinates, determining the position of the reference line according to the sorting result, including: Sorting according to the size of the y coordinate value, the y value being the vertical axis coordinate value and the x value being the horizontal axis coordinate value; Selecting two points with smaller y values, the point with smaller x value being point No. 1 and the point with larger x value being point No. 2; selecting the remaining two points with larger y values, the point with smaller x value being point No. 3 and the point with larger x value being point No. 4; Determining the midpoint of point No. 1 and point No. 2 and the midpoint of point No. 3 and point No. 4, and selecting the two midpoints to be connected, i.e. the reference line; A bone region identification module is configured to, based on the obtained binary image, according to the pixel value distribution on both sides of the reference line, eliminate the case where the reference line passes through the spinal bone part, and identify the ligament ossification region in the image.

7. A computer readable storage medium having stored thereon a computer program, which, when executed by a processor, implements the steps of the spinal ligament ossification image identification method according to any one of claims 1-5.

8. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the spinal ligament ossification image identification method according to any one of claims 1-5 when executing the program.

Images