A target tracking method in medical images based on improved frame difference
By improving the inter-frame differential method, combining discrete points and barium meal basin segmentation, the problem of poor tracking of weakly deformed objects in medical images is solved, and more accurate and efficient target tracking is achieved, suitable for swallowing disorders Barium meal movement trajectory tracking in diagnosis.
Patent Information
- Application Number
- CN202110323497.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-03-26
- Publication Date
- 2024-11-05
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing inter-frame differential method has poor tracking of weakly deformed objects in medical images, especially in the diagnosis of swallowing disorders. The error is large, making it difficult to accurately track the movement trajectory of barium meals, and the grayscale characteristics are not obvious enough, resulting in The tracking effect is not ideal.
An improved inter-frame differential method is proposed, combining discrete points and barium meal basin segmentation, through image preprocessing such as bilateral filtering and grayscale stretching, the noise influence is reduced and grayscale distinction is enhanced, and morphological opening and closing operations and dynamic thresholds are used. Adjust, remove voids and line areas, determine target locations, and use dynamic tracking box and basin segmentation methods to improve inter-frame differential method to improve tracking accuracy.
It significantly improves the accuracy and completeness of target tracking in medical images, reduces errors, and is suitable for multi-object tracking and recognition in swallowing contrast video sequences, improves processing efficiency and accuracy, and is suitable for real-time applications.
Smart Images

Figure CN115131396B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of medical imaging. Background Art
[0002] Medical imaging is an important research direction in the field of medical research. Many scholars and researchers have devoted themselves to the study of medical images for many years. The problem of target recognition and detection in medical images has always received great attention. Many traditional medical image recognition tasks are based on manual processing methods, which are time-consuming and labor-intensive, and the accuracy depends largely on the professional knowledge of medical personnel. Therefore, with the rapid development of computer technology today, we hope to achieve computer-assisted rapid and effective recognition of medical images; target tracking technology is a key scientific issue in the path tracking of mobile targets in medical imaging. The inter-frame motion signal of the mobile target is used for differential analysis and image recognition to achieve the purpose of mobile target trajectory tracking and regional positioning.
[0003] Motion tracking in medical images aims to derive the trajectory of moving organs and body parts over time, and in some cases the trajectory of medical devices. At present, many studies on target tracking in medical images are mainly based on the results of medical image segmentation, which gives segmentation methods and comparisons based on adjustable thresholds, regions and edges. Accordingly, many object tracking methods have been proposed for medical image processing and analysis, but there are many types of medical images, and there is no powerful method to cope with the test of various scenarios.
[0004] Classic frame difference method
[0005] The inter-frame difference method is a method for obtaining the contour of a moving target by performing a difference operation on two consecutive frames of a video image sequence. When abnormal target motion occurs in the monitoring scene, there will be a more obvious difference between two adjacent frames. The two frames are subtracted to obtain the absolute value of the pixel value difference at the corresponding position of the image, and it is determined whether it is greater than a preset threshold, thereby analyzing the object motion characteristics of the video or image sequence. Its mathematical formula is described as follows:
[0006]
[0007] D(x,y) is the difference image between two consecutive frames of images, F(t) and F(t-1) are the images at time t and t-1 respectively, T is the preset threshold selected when the difference image is binarized, D(x,y)=1 represents the foreground, and D(x,y)=0 represents the background.
[0008] The advantages of this algorithm are: simple algorithm implementation, low program design complexity, fast running speed; strong adaptability to dynamic environments, and insensitive to changes in scene light. However, there are also obvious disadvantages: "hole" phenomenon (the grayscale values inside the moving object are similar); "double shadow" phenomenon (the edge contour of the differential image object is coarse); the complete area of the moving object cannot be extracted, only the contour can be extracted; the algorithm effect is heavily dependent on the selected frame time interval and segmentation threshold.
[0009] Closest existing technology and problem description
[0010] The inter-frame difference method has certain advantages for tracking slightly deformed objects, but the tracking effect is not optimistic for targets with obvious deformation, especially in medical images, where the grayscale features are often not obvious enough. In the process of diagnosing a patient's swallowing disorder, the doctor will use X-ray fluoroscopy to observe the movement of the swallowed material in the patient's mouth and throat, and then determine whether there is a problem with swallowing. Mosher first proposed and adopted swallowing X-ray fluoroscopy (VFSS), which is known as the "gold standard" for diagnosing swallowing disorders and determining the mechanism of oropharyngeal dysfunction.
[0011] In medical images obtained by X-ray, doctors feed patients barium meal to make them appear in the image. Tracking the trajectory of the barium meal and obtaining the kinematic data of the barium meal is of great significance to medical research. Based on the classic inter-frame difference method, the movement of the barium meal in certain segments can be obtained, but the error is large and it is often calibrated to the organ part. Summary of the invention
[0012] The purpose of the present invention is to overcome the shortcomings of the prior art. For the video sequence of swallowing angiography, an improved inter-frame difference method is proposed, which combines the advantages of discrete points and barium meal watershed segmentation to more effectively track and identify moving targets in video images.
[0013] Technical Solution
[0014] A target tracking method in medical images based on improved inter-frame difference, characterized in that the data source is medical data in DICOM format, and the process is:
[0015] Part I: Image Preprocessing
[0016] Step 1: Read the data and convert it into a matrix using the simpleITK open source software library to facilitate subsequent processing;
[0017] Step 2: After reading the data, use the grayscale information to find the exact location of the target in the first frame, and capture a suitable search box to include the entire target area. Compared with processing the entire image, focusing on a specific area will greatly improve processing efficiency and reduce the impact of errors;
[0018] Step 3: bilateral filtering the target area can effectively reduce the impact of noise on image processing;
[0019] Step 4: Grayscale stretching increases the span of regions with similar grayscales, making it easier to determine and segment.
[0020] Part II: Swallowed Target Localization Process
[0021] Step 5: Using the improved inter-frame difference method on the preprocessed image, the morphological opening and closing operations are applied to the obtained target area to remove the holes and line areas, and the position of the target is determined by combining the discrete point position and grayscale information. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] Figure 1 The target tracking flow chart of the frame difference method is improved in the embodiment;
[0023] Figure 2 The target tracking result diagram of the improved inter-frame difference method of the embodiment;
[0024] Figure 3 This is a diagram describing the inter-frame difference algorithm of the present invention;
[0025] Figure 4 Intercept the grayscale stretch map for the region;
[0026] Figure 5 Figure 2 shows the target tracking effect. (a) A single target is split into multiple targets. (b) An incomplete area. (c) The tracking and segmentation effect of the barium meal watershed. DETAILED DESCRIPTION
[0027] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.
[0028] The data source of this embodiment is medical data in DICOM format.
[0029] Part I: Image Preprocessing
[0030] Step 1: Read the data and convert it into a matrix using the simpleITK open source software library to facilitate subsequent processing.
[0031] Step 2, after reading the data, use the grayscale information to find the exact position of the target in the first frame, and capture a suitable search box to include the entire target area. Compared with processing the entire image, focusing on a specific area will greatly improve processing efficiency and reduce the impact of errors.
[0032] Step 3, performing bilateral filtering on the target area can effectively reduce the impact of noise on image processing.
[0033] Step 4: Grayscale stretching increases the span of regions with similar grayscale, making it easier to determine and segment.
[0034] Part II: Swallowed Target Localization Process
[0035] Step 5: Using the improved inter-frame difference method on the preprocessed image, the morphological opening and closing operations are applied to the obtained target area to remove the holes and line areas, and the position of the target is determined by combining the discrete point position and grayscale information.
[0036] The above overall processing process is as follows Figure 1 The results are shown as Figure 2 .
[0037] In the first part:
[0038] Generally speaking, the noise of medical images is relatively large. In order to improve the accuracy of contour extraction, bilateral filtering is used for denoising. Bilateral filtering is a nonlinear filtering method. It is a compromise between the spatial proximity and pixel value similarity of the image. It also considers spatial information and grayscale similarity to achieve the purpose of edge preservation and denoising. The advantage of bilateral filtering is that it can do edge preserving. Gaussian filtering for denoising will obviously blur the edges, and the protection effect for high-frequency details is not obvious. As the name implies, the bilateral filter has an additional Gaussian variance compared to the Gaussian filter. It is a Gaussian filter function based on spatial distribution. Therefore, near the edge, the pixels farther away will not affect the pixel values on the edge too much, thus ensuring the preservation of the pixel values near the edge. However, due to the preservation of too much high-frequency information, the bilateral filter cannot cleanly filter out the high-frequency noise in the color image, and can only perform noise filtering on the low-frequency information.
[0039] Grayscale stretching is also called contrast stretching. It is the most basic grayscale transformation. It uses the simplest piecewise linear transformation function. Its main idea is to improve the dynamic range of grayscale during image processing. Grayscale stretching is more effective in medical image processing because medical images are often grayscale images with low resolution, so it is difficult to capture the edge information of the target. Grayscale stretching can effectively reduce the impact of grayscale proximity.
[0040] To use grayscale stretching, you must first capture the key area, select the appropriate grayscale upper limit t_u and lower limit t_d, and limit the range of pixels in the area.
[0041]
[0042] If the grayscale range of the points in this area is between 10 and 150, and the grayscale range of the target is approximately between 30 and 50, then t_u=100 can be taken, because the points with grayscale greater than 100 have little effect on the recognition of the target. At this time, the grayscale value of the target area becomes 10 to 100. After stretching it to the range of 0 and 255, as shown in Figure 4 , the target area is better determined and the edge is easier to extract.
[0043] In the second part:
[0044] Principle of improved inter-frame difference method:
[0045] In step 5, the present invention adopts an improved inter-frame difference method, adopts watershed segmentation, discrete point tracking, and uses a dynamic tracking box (a rectangular area containing the target area) method to improve the shortcomings of the difference method. After multiple attempts in the swallowing angiography video sequence, better results were achieved compared to the classic difference method. The main algorithm is as follows:
[0046] For the region of interest in the first frame, a rectangular search box is obtained to include this region. The purpose of this is to reduce the amount of data and noise and speed up processing. The current frame intercepts the region with the same position and size as the search box of the previous frame, and then performs a difference with the previous frame to obtain the difference matrix d_frame, which is binarized:
[0047]
[0048] Among them, T_diff is the preset binarization threshold. If it is less than this value, it means that the change between frames is small. The value of the matrix d_frame at this point is 0, which is determined to be a background point; the area with a value of 1 in d_frame is extracted as seg. Although the points in this area have a large change relative to the previous frame, they cannot be directly determined as foreground points, because after the target moves, the grayscale value of the original position also changes greatly. At this time, the points in seg are projected to the position of the original image of the current frame. If the point Image[x,y] corresponding to the original image is greater than the preset projection threshold T_p, let seg[x,y]=1, indicating that this point is a background point and needs to be subtracted in the following steps. Otherwise, let seg[x,y]=0, and name the matrix d_seg; add the region of interest seg_p of the previous frame to d_frame, and then subtract d_seg to get the region of interest seg_c in the current frame:
[0049] seg_c=(d_frame+seg_p)-d_seg (3)
[0050] Get the center of the current region of interest, update seg_p = seg_c, and get the new search box and the average threshold of the region of interest, and continue to the next frame. Figure 3 The algorithm idea is demonstrated. Using this method for inter-frame difference can effectively eliminate the influence of residual images, and combined with watershed segmentation, the contour and area of the moving target can be obtained more accurately.
[0051] During the swallowing process of the patient, multiple swallows and swallowing residues may occur, causing a single target to be transformed into multiple targets. In order to solve the problem of tracking and identifying multiple targets in a video sequence, the present invention proposes a method for locating the target area with discrete points. When the target area of a certain frame is split into several areas, the points in the small areas are recorded separately, and a single target recognition operation is performed on each small area in the next frame. In addition to being helpful in multi-target tracking, discrete points can also be combined with watershed segmentation to correct target edge information. The discrete points selected in this embodiment are based on the number of pixels occupied by the target. 1 / 3 of the number of pixels in the target area is selected and its center position is calculated (used to record the movement trajectory of the target). In the d_frame obtained by inter-frame difference, the findtours method of opencv can be used to easily find the set S of each area and number them, and delete the area in S whose area is less than the preset threshold k (for example: in a 480*480 medical image, k is 20, which can well remove small areas). Because the swallowing process is not an irregular movement, it is often like Figure 5 The two modes of a (the dotted box is the original position, the pattern is the moving target, and the solid color is other areas). By extracting the first five larger areas in the set S, the main moving direction of the target can be determined. Then, according to the position relationship between the discrete points of the current frame and the previous frame, the area in the non-critical moving direction can be removed, such as Figure 5 In the case where the target may exceed the search box (the number of discrete points in the target area of the current frame is significantly less than the number of discrete points in the previous frame), the watershed segmentation is combined to find the part of the watershed of the discrete points outside the area, such as Figure 5 b, 5c, after which the position and size of the search box need to be repositioned so that it completely contains the target area.
[0052] Dynamic threshold adjustment
[0053] In medical image analysis, many video sequences will have target splitting or target area changes, which means the target's grayscale value will also change. Using a fixed threshold parameter may incorrectly divide the target and background areas, resulting in continuous errors. The accumulation of errors makes target tracking particularly difficult. Therefore, when there is noise or the background changes greatly, it is necessary to dynamically adjust the threshold.
[0054] At present, a global binarization algorithm is widely used. It is based on the grayscale characteristics of the image, finds its maximum inter-class variance, and divides the image into two parts: foreground and background. When the optimal threshold is taken, the difference between the two parts should be the largest. The standard used to measure the difference in the OTSU algorithm is the more common maximum inter-class variance. Based on this, the present invention proposes a dynamic threshold adjustment method for medical images. In order to reduce the amount of calculation, the advantages of the search box and discrete point tracking method are combined, and the average grayscale of the discrete points in the target area in the previous frame is used as a reference for the threshold of the current frame. If the average grayscale value of the discrete points in the previous frame is G1, the grayscale of the pixels in the search box of the current frame is stretched, and G1 is also stretched to the same scale to obtain G2. The OTSU algorithm is used in the search box to find the optimal threshold T. Considering the existence of single target splitting, the area with an average grayscale lower than the threshold T and in the interval [G2-k, G2+k] is determined as the foreground target area, and k is generally 10. The discrete points and average grayscale of these areas are recorded and brought into the calculation of the next frame. Performance analysis and verification of implementation examples
[0055] In order to better verify the performance of the algorithm, this embodiment selects two algorithms, GMM and VIBE, for comparative study. The enhanced Gaussian mixture model (EGMM) proposed in Z. Yaniv, B. C. Lowekamp, H. J. Johnson, and R. Beare, "SimpleITK Image-Analysis Notebooks: a Collaborative Environment for Education and Reproducible Research", J Digit Imaging., 31(3): 290-303, 2018. is representative and is a deterministic parameter method for background subtraction. B. C. Lowekamp, D. T. Chen, and L. The VIBE algorithm proposed in D.Blezek, "The Design of SimpleITK", Front.Neuroinform., 7:45., 2013. describes the boundary of foreground objects more clearly. This time, we did not compare it with the classical algorithms (frame difference, optical flow method, etc.) because the classical algorithms are not sensitive enough to background changes and only a few data are of reference value, so we did not compare them. The data samples selected this time are from the swallowing contrast video sequence of the hospital radiology department. The details are as follows Table 1
[0056] Table 1 Sample information
[0057]
[0058] The evaluation indicators are: Average Pixel Error (APE), which is the pixel distance between the predicted target center and the true position as the error value. The larger the value, the greater the error. Average Overlap Rate (AOR), which is expressed as the overlap rate between the target area and the original annotated area. Table 2 shows the performance comparison of the three algorithms in side view and front view.
[0059] Table 2 Algorithm performance comparison
[0060]
[0061] Note: AOR = AO / (A1+A2-AO), AO is the overlapping part of the two areas
[0062] The three algorithms are all fast, but the VIBE and EGMM algorithms are not sensitive to residual stationary targets and are prone to incomplete recognition, resulting in their inferior precision and accuracy to the improved inter-frame difference method of the present invention. For grayscale images of medical images, the foreground and background colors are similar, and the latter two algorithms may misjudge the foreground as the background, resulting in large errors. Especially in the front view, the area of the moving target is small, and there is no obvious movement for a long time. Once the movement occurs, it is relatively violent, which makes the results obtained by the three algorithms unsatisfactory. The improved inter-frame difference method proposed in the present invention takes into account the residue of the moving object, and the results in the front view still have a great advantage. Both EGMM and VIBE need to model the entire background, while the improved inter-frame difference method processes the image in the search box and does not need to save the background information, which greatly saves the amount of memory, improves the performance, and is more suitable for occasions with real-time requirements.
Claims
1. A target tracking method in medical images based on frame difference improvement, characterized in that: The data source is medical data in DICOM format, and the process is: Part I: Image Preprocessing Step 1, read the data and convert it into a matrix using the simpleITK open source software library; Step 2, after reading the data, use the grayscale information to find the exact location of the target in the first frame, and intercept a suitable search box to include the entire target area; Step 3, perform bilateral filtering on the target area; Step 4: Grayscale stretching increases the span of regions with similar grayscales; Part II: Swallowed Target Localization Process Step 5: Using the improved inter-frame difference method on the preprocessed image, the morphological opening and closing operation is applied to the obtained target area to remove the holes and line areas, and the position of the target is determined by combining the discrete point position and grayscale information; The improved inter-frame difference method comprises: For the region of interest in the first frame, a rectangular search box is obtained to contain this region; the current frame intercepts the region with the same position and size as the search box of the previous frame, and then interpolates it with the previous frame to obtain the difference matrix d_frame, which is binarized: Where T_diff is the preset binarization threshold; extract the area with a value of 1 in d_frame as seg; project the points in seg to the position of the original image of the current frame. If the point Image[x,y] corresponding to the original image is greater than the preset projection threshold T_p, set seg[x,y]=1 to indicate that this point is a background point. Otherwise, set seg[x,y]=0 and name the matrix d_seg; add the region of interest seg_p of the previous frame to d_frame, and then subtract d_seg to get the region of interest seg_c in the current frame: seg_c=(d_frame+seg_p)-d_seg (3) Get the center of the current region of interest, update seg_p = seg_c, get a new search box and the average threshold of the region of interest, and continue processing the next frame; The step of combining the discrete point position and the grayscale information to determine the position of the target includes: When the target area of a certain frame is split into several areas, the points in the small areas are recorded separately, and a single target recognition operation is performed on each small area in the next frame; the selected discrete points are based on the number of pixels occupied by the target, and 1 / 3 of the number of pixels in the target area is selected and their center positions are calculated; in the d_frame obtained by inter-frame difference, the findtours method of opencv is used to find the set S of each area and number them, and the areas in S with an area smaller than the preset threshold k are deleted; the first five larger areas in the set S are extracted to determine the main movement direction of the target, and the areas in non-critical movement directions are removed according to the positional relationship between the discrete points of the current frame and the previous frame; in the case where the target exceeds the search box, the watershed segmentation is combined to find the part of the watershed of the discrete points outside the area, and then the position and size of the search box are repositioned to make it completely contain the target area.
