A method for extracting feature points from capsule endoscope images

Abstract

The present invention discloses a method for extracting feature points of capsule endoscope images, and belongs to the interdisciplinary technical field of computer vision and medicine. The present invention studies the problem that feature points are difficult to extract due to small grayscale changes in capsule endoscope images, divides the image into blocks, calculates the grayscale difference of pixels in the regional blocks using the coefficient of variation to obtain a local adaptive threshold, realizes the adaptive extraction of FAST corner points, screens key points through a quadtree, further obtains evenly distributed key points, describes the key points based on the BEBLID feature point descriptor, completes the extraction of feature points of capsule endoscope images, and solves the problem that feature points of capsule endoscope images are difficult to extract due to small grayscale differences.

Description

Technical Field

[0001] The invention relates to a method for extracting characteristic points of capsule endoscope images, and belongs to the technical field of cross-disciplinary subjects combining computer vision with medicine. Background Art

[0002] Magnetically controlled wireless capsule endoscopes (WCE) have been widely used as clinical imaging and diagnostic tools for the human digestive system that are non-invasive, painless, and cross-infection-free. However, WCE cannot currently achieve self-controlled adaptation, including effective diagnosis of abnormalities found by analyzing WCE videos, which requires the positioning and three-dimensional restoration of the capsule in the digestive tract. The current SLAM method that can achieve good camera positioning and the SFM method for three-dimensional structure restoration are almost ineffective for intracavitary videos, mainly because the features caused by the high light on the inner surface of the gastrointestinal cavity and the low frame rate of the video frame cannot be extracted. This greatly limits the applicability and universality of capsule endoscopes.

[0003] Feature points are points where the grayscale value of an image changes dramatically or where the curvature is large on the edge of the image. The purpose of extracting feature points is to refine and express some important areas in the image, which is the basis for subsequent visual tasks such as feature matching, pose estimation, and three-dimensional structure recovery. Failure to extract feature points will result in the subsequent capsule endoscopy positioning, gastrointestinal three-dimensional structure recovery, and lesion identification being unable to proceed smoothly.

[0004] At present, the methods used for extracting feature points of capsule endoscopy images mainly include SIFT, SURF, ORB, etc. Spyrou et al. compared and evaluated various feature extraction methods on capsule endoscopy images [E. Spyrou, DKIakovidis, S. Niafas, et al. Comparative assessment of feature extraction methods for visual odometry in wireless capsule endoscopy [J]. Computers in Biology & Medicine, 2015, 65: 297-307.] and found that the grayscale changes of capsule endoscopy images are very small, resulting in these grayscale-based methods being unable to extract enough feature points in capsule endoscopy image areas with weak textures. Summary of the invention

[0005] In order to solve the problem that the existing capsule endoscope image has small grayscale changes and is difficult to extract feature points, the present invention provides a method for extracting feature points of a capsule endoscope image, comprising the following steps:

[0006] Step 1: Acquire a capsule endoscope image, construct an image pyramid, and divide the image of each layer of the image pyramid into image blocks of the same size to achieve local division;

[0007] Step 2: Calculate the coefficient of variation of the grayscale value of each pixel in the image block in step 1 as the threshold for extracting FAST corner points in the current image block;

[0008] Step 3: According to the threshold value calculated in step 2, FAST corner points are extracted from the image block as feature points, and the extracted feature points are uniformly screened using a quadtree;

[0009] Step 4: Use the BEBLID feature descriptor to describe the feature points processed in step 3.

[0010] Optionally, the step 2 is specifically as follows:

[0011] When calculating the adaptive threshold using the coefficient of variation, the image block is first divided into l×h grids according to the set grid size, and each grid is used as an independent image area. The threshold of the image block is:

[0012]

[0013] Among them, D i Represents the standard deviation of the grayscale value of the pixel in grid i, M i represents the average grayscale value of the pixels in the grid i, and α is the grayscale value variation coefficient of the pixels in the image block.

[0014] Optionally, the gray value variation coefficient α of the pixel points in the image block is:

[0015]

[0016]

[0017] Among them, M′ i is the truncated mean difference of the grayscale values ​​of the pixels in grid i, n i represents the number of pixels in grid i, f i (x, y) represents the gray value at (x, y) in grid i, f i (x, y) max represents the maximum gray value in grid i, f i (x, y) min Represents the minimum grayscale value in grid i.

[0018] Optionally, the step 3 is specifically as follows:

[0019] Step 31: According to the total number of image pyramid layers and the total number of feature points to be extracted, the number of feature points to be extracted in each layer of the image pyramid is calculated, and then the feature points are evenly distributed to each layer of the image pyramid according to the area;

[0020] Step 32: extracting FAST corner points from the image blocks divided in step 1, and screening them using the threshold T of the image blocks;

[0021] Step 33: Use quadtree to average and distribute the feature points in each layer of the image pyramid. Take the entire image as the initial node of the quadtree to get the initial quadtree structure. If the number of feature points in the node is greater than 1, divide the node into four nodes. If the feature point in the node is empty, delete it; if the number of feature points in the newly divided node is greater than 1, split it into 4 nodes again; when the total number of nodes is greater than the number of feature points N that should be allocated to the layer α Stop splitting when the node reaches 0, or when no further splitting is possible, and finally select a FAST corner point with the highest response value from each node.

[0022] Optionally, the step 31 includes:

[0023] Assuming that the length and width of the 0th layer image are L and W respectively, the area is C = L × W, the scale factor is s, 0 < s < 1, then the total area of ​​the entire pyramid is as shown in formula (4):

[0024]

[0025]

[0026] Then, the number of feature points per unit area is as shown in formula (5):

[0027]

[0028] Then, the number of feature points that should be allocated to the 0th layer is as shown in formula (6):

[0029]

[0030] The number of feature points that should be allocated to the αth layer is:

[0031]

[0032] Where N is the total number of feature points that need to be extracted.

[0033] Optionally, the step 4 is specifically as follows:

[0034] BEBLID calculates the average gray value difference within a region. A fixed number of square region pairs of different sizes are selected around the feature point. Each pair of square regions has the same size. The difference in the average gray value of the pixels in each pair of square blocks is calculated to describe the feature point.

[0035] The extraction function f(x) of the BEBLID feature descriptor is:

[0036]

[0037] Where I(q) and I(r) are the grayscale values ​​of pixel points q and r, R(p1, s) and R(p2, s) are square frames centered at p1 and p2, and their size is s.

[0038] Optionally, in step 1, the image pyramid has a total of 8 layers, and the scaling ratio of each layer is 1.2.

[0039] Optionally, the step 1 divides each layer of the pyramid image into image blocks of 30*30 pixels.

[0040] The second object of the present invention is to provide a capsule endoscope image classification method, which utilizes the above-mentioned capsule endoscope image feature point extraction method to extract image features, and then performs image classification based on the extracted image features.

[0041] A third object of the present invention is to provide an endoscope image classification system, comprising:

[0042] An image acquisition module, used for acquiring capsule endoscope images;

[0043] a data processing module, which uses the above-mentioned method for extracting characteristic points of capsule endoscope images to extract image features and / or the above-mentioned method for classifying capsule endoscope images to process the image acquired by the image acquisition module;

[0044] The output display module is used to output the processing results of the data processing module.

[0045] The beneficial effects of the present invention are:

[0046] The present invention divides the image into blocks, calculates the grayscale difference of pixels in the regional block by using the coefficient of variation to obtain a local adaptive threshold, realizes adaptive extraction of FAST corner points of capsule endoscope images, screens key points by quadtree, further obtains evenly distributed key points, describes the key points based on the BEBLID feature point descriptor, completes the extraction of feature points of capsule endoscope images, and solves the problem of difficulty in extracting feature points of capsule endoscope images caused by small grayscale difference. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0048] Figure 1 It is a schematic diagram of feature points extracted from the left figure in the second embodiment of the present invention.

[0049] Figure 2 This is a schematic diagram of feature points extracted from the right figure in Embodiment 2 of the present invention.

[0050] Figure 3 This is a matching result diagram of feature points extracted from the left and right images in the second embodiment of the present invention. DETAILED DESCRIPTION

[0051] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0052] Embodiment 1:

[0053] This embodiment provides a method for extracting feature points of a capsule endoscope image, comprising the following steps:

[0054] Step 1: Acquire a capsule endoscope image, construct an image pyramid, and divide the image of each layer of the image pyramid into image blocks of the same size to achieve local division;

[0055] Step 2: Calculate the coefficient of variation of the grayscale value of each pixel in the image block in step 1 as the threshold for extracting FAST corner points in the current image block;

[0056] Step 3: According to the threshold value calculated in step 2, FAST corner points are extracted from the image block as feature points, and the extracted feature points are uniformly screened using a quadtree;

[0057] Step 4: Use the BEBLID feature descriptor to describe the feature points processed in step 3.

[0058] Embodiment 2:

[0059] This embodiment provides a method for extracting feature points of a capsule endoscope image, comprising the following steps:

[0060] Step 1: Construct an image pyramid with 8 layers in total. The scaling ratio of each layer is 1.2. Each layer of the pyramid is divided into image blocks of 30*30 pixels.

[0061] Step 2: When using the coefficient of variation to calculate the adaptive threshold, first divide the image into l×h grids according to the set grid size, and each grid is used as an independent image region. The calculation of the threshold T in each grid is shown in formula (1):

[0062]

[0063] Among them, D i Represents the standard deviation of the grayscale value of the pixel in grid i, M i Represents the average grayscale value of the pixels in grid i.

[0064]

[0065]

[0066] Among them, M′ i is the truncated mean difference of the grayscale values ​​of the pixels in grid i, n i represents the number of pixels in grid i, f i (x, y) represents the gray value at (x, y) in grid i, f i (x, y) max represents the maximum gray value in grid i, f i (x, y) min Represents the minimum grayscale value in grid i.

[0067] Step 3: The specific steps for extracting FAST corner points are as follows:

[0068] Step 31: According to the total number of image pyramid layers and the total number of feature points to be extracted, calculate the number of feature points to be extracted in each layer of the image pyramid, and then evenly distribute the feature points to each layer of the image pyramid according to the area. Assuming that the length and width of the image at layer 0 are L and W respectively, the area is C = L × W, and the scale factor is s (0 < s < 1), the total area of ​​the entire pyramid is as shown in formula (4):

[0069]

[0070] Then, the number of feature points per unit area is as shown in formula (5):

[0071]

[0072] Then, the number of feature points that should be allocated to the 0th layer is as shown in formula (6):

[0073]

[0074] Then the number of feature points that should be allocated to the αth layer is deduced as shown in formula (7):

[0075]

[0076] Step 32: Extract FAST corner points from the blocks divided in step 1 and filter them using the adaptive threshold T of the current block.

[0077] Step 33: Use a quadtree to average and distribute the feature points in each layer of the image pyramid. Use the entire image as the initial node of the quadtree to obtain the initial quadtree structure. If the number of feature points in a node is greater than 1, divide the node into four nodes. If the feature point in the node is empty, delete it. If the number of feature points in the newly divided node is greater than 1, split it into 4 nodes again. In this way, split it all the way. When the total number of nodes is greater than the number of feature points N that should be allocated to the layer α Stop splitting when the node reaches 0, or the node cannot be split any further. Finally, select a FAST corner point with the highest response value from each node.

[0078] Step 4: BEBLID calculates the average grayscale value difference within a region. A fixed number of square region pairs of different sizes are selected around the feature point, and each pair of square regions is of the same size. The difference in the average grayscale value of the pixels in each pair of square blocks is calculated to describe the feature point.

[0079] The extraction function f(x) of BEBLID feature descriptor is defined as shown in formula (8):

[0080]

[0081] Where I(q) and I(r) are the grayscale values ​​of pixel points q and r, R(p1, s) and R(p2, s) are square frames centered at p1 and p2, and their size is s.

[0082] This embodiment is completed using VS2019 and OpenCV under the Linux operating system. The hardware environment is a laptop with a 3.20GHz i7 processor and 8GB of running memory. The experimental process is relatively stable.

[0083] The experimental results are shown in the attached figure. Figure 1 , Figure 2 It is an image obtained by extracting feature points from two adjacent frames of the video taken by the capsule endoscope. The small circles represent the extracted feature points. Figure 3 Yes Figure 1 , Figure 2 The image is matched with the feature points extracted from the two images, and the line represents the matching relationship between the feature points in the two images. It can be seen from the figure that the image feature point extraction method of the present invention can accurately extract the feature points in the capsule endoscopy image.

[0084] Some steps in the embodiments of the present invention may be implemented using software, and the corresponding software program may be stored in a readable storage medium, such as a CD or a hard disk.

[0085] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for extracting feature points of capsule endoscope images, characterized in that: It includes the following steps: Step 1: Obtain the capsule endoscope image, construct an image pyramid, and divide the images of each layer of the image pyramid into image blocks of the same size to achieve local division; Step 2: Calculate the coefficient of variation of the gray values of the pixel points in each image block in Step 1 as the threshold for extracting FAST corner points in the current image block; Step 3: According to the threshold calculated in Step 2, extract FAST corner points in the image block as feature points, and use a quadtree to uniformly screen the extracted feature points; Step 4: Use the BEBLID feature descriptor to describe the feature points processed in Step 3; The specific content of Step 2 is as follows: When calculating the threshold of the image block using the coefficient of variation, first divide the image block into l×h grids according to the set grid size, and each grid is used as an independent image area. The threshold of the image block is: Among them, D i Represents the standard deviation of the grayscale value of the pixel in grid i, M i represents the average grayscale value of the pixels in the grid i, and α is the grayscale value variation coefficient of the pixels in the image block; The coefficient of variation α of the gray values of the pixel points in the image block is: Among them, M′ i is the truncated mean value of the grayscale value of the pixel in grid i, n i represents the number of pixels in grid i, f i (x,y) represents the gray value at (x,y) in grid i, f i (x,y) max represents the maximum gray value in grid i, f i (x,y) min Represents the minimum gray value in grid i; The specific content of Step 3 is as follows: Step 31: According to the total number of layers of the image pyramid and the total number of feature points to be extracted, calculate the number of feature points to be extracted in each layer of the image pyramid, and then evenly distribute the feature points to each layer of the image pyramid according to the area; Step 32: Extract FAST corner points in the image blocks divided in Step 1 and screen them using the threshold T of the image block; Step 33: Use a quadtree to average and distribute the feature points in each layer of the image pyramid in step 31; use the entire image as the initial node of the quadtree to obtain the initial quadtree structure. If the number of feature points in a node is greater than 1, divide the node into four nodes. If the feature point in the node is empty, delete it; if the number of feature points in the newly divided node is greater than 1, split it into 4 nodes again; when the total number of nodes is greater than the number of feature points N that should be allocated to the layer α Stop splitting when the node reaches 0, or when it is no longer possible to split, and finally select a FAST corner point with the largest response value from each node; The content of Step 31 includes: Assume that the length and width of the image in the 0th layer are L and W respectively, the area is C = L×W, and the scale factor is s, 0 < s < 1. Then the total area of the entire pyramid is shown in Equation (4): Then, the number of feature points per unit area is shown in Equation (5): Then, the number of feature points to be allocated in the 0th layer is shown in Equation (6): The number of feature points to be allocated in the αth layer is: where N is the total number of feature points to be extracted.

2. The method according to claim 1, characterized in that The specific content of Step 4 is as follows: BEBLID calculates the average gray value difference in a region. Select a fixed number of pairs of square regions of different sizes around the feature point. Each pair of square regions has the same size, and calculate the difference in the average gray values of the pixel points in each pair of square blocks to describe the feature point; The extraction function f(x) of the BEBLID feature descriptor is: where I(q) and I(r) are the gray values of pixel points q and r, and R(p1, s), R(p2, s) are square frames centered at p1 and p2 with a size of s.

3. The method according to claim 1, characterized in that In the image pyramid in Step 1, the pyramid has a total of 8 layers, and the scaling ratio of each layer is 1.

2.

4. The method according to claim 1, characterized in that: In Step 1, each layer of the image in the pyramid is divided into image blocks with a size of 30*30 pixels.

5. A capsule endoscope image classification method, characterized in that: The image classification method extracts image features using a method for extracting feature points of a capsule endoscope image according to any one of claims 1-4, and then performs image classification according to the extracted image features.

6. An endoscope image classification system, characterized in that: The system includes: An image acquisition module for acquiring capsule endoscope images; A data processing module that extracts image features using a method for extracting feature points of a capsule endoscope image according to any one of claims 1-4 and / or processes the images acquired by the image acquisition module using a method for classifying capsule endoscope images according to claim 5; The output display module is used to output the processing results of the data processing module.

Images