A Method for Extracting Image Features of Capsule Endoscope Based on Improved Harris Corner

Abstract

The present invention discloses a method for extracting capsule endoscope image features based on improved Harris corners, including: calculating the partial derivatives and gradient values of each pixel point in the capsule endoscope image, and filtering out the pixel points with gradient values lower than the set gradient threshold; improving the Harris response function, calculating the improved Harrsi response function values of each pixel point, and obtaining potential feature points through the response function values; combining the improved Harris response function values with the window width and sliding distance of the sliding window to obtain an adaptive sliding window and performing non-maximum suppression on the potential feature points; determining the direction of each feature point, taking this direction as the main direction of the improved Harris corner, constructing an rBRIEF descriptor, and using the descriptor for feature point matching. The present invention can not only obtain a large number of feature points, but also maintain high accuracy; it can quickly and effectively perform feature extraction, and can maintain the accuracy of feature matching while obtaining a sufficient number of feature points in the capsule endoscope image.

Description

Technical Field

[0001] The present invention relates to a method for extracting capsule endoscope image features based on improved Harris corners, belonging to the interdisciplinary technical field of the combination of computer vision and medicine. Background Art

[0002] Wireless capsule endoscopes equipped with miniature cameras have become a hot medical tool for non-invasive examination of the gastrointestinal tract. However, how to achieve autonomous, non-repetitive, and non-missing photography remains a difficult problem to date. Solving this problem is inseparable from autonomous navigation and positioning technology, and three-dimensional reconstruction of the gastrointestinal structure based on capsule endoscope video is the basis for realizing navigation and positioning. At present, two major method systems, SfM (Structure from Motion) and SLAM (Simultaneous Localization And Mapping), have been formed for three-dimensional reconstruction of videos. The implementation processes of SfM and SLAM are highly consistent, and the three-dimensional reconstruction of the gastrointestinal tract based on capsule endoscope video also follows this method system, that is, first, feature extraction and matching are performed on multiple frames of images, and then the matched two-dimensional point pairs in the images are triangulated to obtain three-dimensional points to represent the three-dimensional structure of the object or scene. That is to say, SfM and SLAM can effectively restore the three-dimensional structure of the gastrointestinal tract and perform subsequent work such as navigation and positioning only on the premise of ensuring sufficient and stable feature point extraction. Existing research and the applicant's experiments have shown that common mainstream feature extraction methods do not work well or even fail on capsule endoscope environmental images, mainly manifested as problems in feature point extraction. The root cause is that the capsule endoscope scene is completely different from the natural scene, that is, the internal environment of the gastrointestinal soft tissue where the capsule endoscope is located is deformed as a whole and coexists with weak textures. Coupled with the difference in illumination and wide baseline in the internal environment, it increases the difficulty of feature extraction and tracking, resulting in the failure or even inability to initialize SfM and SLAM.

[0003] In natural scenes, currently mainstream feature extraction and matching algorithms such as SIFT (Scale-Invariant Feature Transform), SURF (Speeded-Up Robust Features), and ORB (Oriented FAST and Rotated BRIEF) perform excellently. However, in the capsule endoscope environment, these methods will have a serious problem, that is, the problem that the number of feature points and the matching accuracy cannot be obtained simultaneously. When using the default threshold of the algorithm, the accuracy of feature matching is relatively good, but the number of feature points is very small, which has a fatal defect for three-dimensional reconstruction work. When the threshold is lowered to obtain more feature points, the quality of feature matching will be greatly reduced, affecting subsequent steps of three-dimensional reconstruction.

[0004] To solve the problem that the number of feature points and the matching accuracy cannot be obtained simultaneously in capsule endoscopes, some relatively effective methods have emerged. By improving the FAST algorithm, ORB algorithm, SIFT algorithm, and SURF algorithm, the number of extracted feature points and the quality of feature points can be increased, and the feature point extraction speed can be accelerated. However, the problem that the number of feature points and the accuracy of feature point matching cannot coexist still remains unsolved. Summary of the Invention

[0005] To solve the above problems, the present invention provides a method for extracting capsule endoscope image features based on improved Harris corners. This method can perform feature extraction quickly and effectively, and can maintain the accuracy of feature matching while obtaining a sufficient number of feature points in capsule endoscope images.

[0006] On the one hand, the present invention provides a method for extracting capsule endoscope image features based on improved Harris corners, and the method includes:

[0007] Step 1: Obtain two frames of capsule endoscope images, calculate the partial derivatives of each pixel point in the horizontal and vertical directions of the image, calculate the gradient value of each pixel point according to the partial derivatives in these two directions, set the gradient threshold and partial derivative threshold, and filter out the pixel points with gradient values lower than the gradient threshold, filtering out the wrinkle texture, blood vessel texture areas, and surrounding pixel points in the capsule endoscope image;

[0008] Step 2: Improve the Harris response function, calculate the improved Harrsi response function value of each pixel point according to the partial derivative in the horizontal direction, the partial derivative in the vertical direction, and the gradient value obtained in Step 1, and obtain potential feature points through the response function value;

[0009] Step 3: Combine the improved Harris response function value with the window width and sliding distance of the sliding window to obtain an adaptive sliding window, and perform non-maximum suppression on the potential feature points obtained in Step 2 through the adaptive sliding window to obtain the feature points with the largest response function value within a local range;

[0010] Step 4: Use the gray centroid method to determine the direction of each feature point obtained in Step 3, use this direction as the main direction of the improved Harris corner, and construct an rBRIEF descriptor based on this, and use the descriptor to perform feature point matching.

[0011] In one implementation, calculating the improved Harrsi response function value for each pixel point in step 2 includes: applying different weight values w(x, y) to the feature points within the 3×3 region of each pixel point to calculate the autocorrelation matrix M, and calculating the determinant det(M) and the trace trace(M) of matrix M according to M; using the determinant and the trace to calculate the improved Harris response function value, and the calculation formula is as follows:

[0012]

[0013] Wherein, w(x, y) represents the weight at point (x, y), and I x (x, y), I y (x, y) respectively represent the partial derivative in the horizontal direction and the partial derivative in the vertical direction at point (x, y), and the two eigenvalues of matrix M are λ 1 , λ 2 , and the expressions of det(M) and trace(M) are respectively det(M) = λ 1 *λ 2 , trace(M) = λ 1 +λ 2 .

[0014] In one implementation, step 3 specifically includes the following steps:

[0015] Step 31: Divide the image into different regions according to the magnitudes of the improved Harrsi response function values of each pixel point in the image calculated in step 2;

[0016] Step 32: For each region, calculate the adaptive window width and sliding distance according to the magnitude of the response function value; when the response function value is relatively low, set a smaller window width and sliding distance; when the response function value is relatively high, set a larger window width and sliding distance;

[0017] Step 33: Perform non-maximum suppression on the image using the adaptive window width and sliding distance, and step 33 specifically includes the following steps:

[0018] Step 331: With each pixel point as the center, delimit a sliding window using the adaptive window width;

[0019] Step 332: Move the sliding window along the adaptive sliding distance, and calculate the response function value of each pixel point within the sliding window;

[0020] Step 333: Within the sliding window, retain the pixel point with the maximum response function value and filter out other pixel points;

[0021] Step 34: Repeat step 33 until the entire image is processed.

[0022] In one of the embodiments, the window width W of the sliding window and the sliding distance d of the sliding window in step 3 satisfy the following relationship:

[0023] W = 2d - 1

[0024] where W represents the window width of the sliding window, d represents the sliding distance of the sliding window, and the expression of d is R is the value of the Harrsi response function calculated in step 2.

[0025] In one of the embodiments, step 4 specifically includes the following steps:

[0026] Step 41: Use the gray centroid method to determine the direction of each feature point obtained in step 3, so that the feature point has rotation invariance, and use this direction as the main direction of the improved Harris corner point;

[0027] Step 42: Divide the area centered on the feature point, compare the pixel values of the fixed position points to obtain the binary rBRIEF descriptor, and the length of the descriptor is related to the divided area;

[0028] Step 43: Perform Gaussian filtering on the image to reduce noise interference;

[0029] Step 44: After obtaining the rBRIEF descriptor of each feature point, match the feature descriptors by the brute-force matching method to obtain the rough matching result.

[0030] In one of the embodiments, the potential feature points in step 2 are: obtaining the high-response points of the folds and blood vessel regions by using the improved Harris response function, including flat region points, edge points, and corner points with differences; in step 4, a 256-dimensional rBRIEF descriptor is used to construct a descriptor for matching.

[0031] In one of the embodiments, there is also step 5, and step 5 includes: using the feature rough matching retained in step 4, and using the RANSAC method to screen and eliminate the mismatched point pairs.

[0032] In one of the embodiments, there is also step 6, and step 6 includes: calculating the accuracy of the feature matching by using the number of accurate matches and the number of rough matches obtained by RANSAC.

[0033] On the other hand, the present invention also provides a three-dimensional reconstruction method for the digestive tract of a capsule endoscope image based on the improved Harris corner point. The three-dimensional reconstruction method for the digestive tract uses the capsule endoscope image feature extraction method based on the improved Harris corner point to extract image feature points, and then performs three-dimensional reconstruction according to the extracted feature points.

[0034] On the other hand, the present invention also provides a three-dimensional reconstruction system for the digestive tract of capsule endoscope images based on improved Harris corners. The three-dimensional reconstruction system for the digestive tract includes:

[0035] An image acquisition module for acquiring the capsule endoscope images and performing preprocessing;

[0036] A feature extraction module that uses the method for extracting features of capsule endoscope images based on improved Harris corners to extract features from the preprocessed capsule endoscope images;

[0037] A three-dimensional reconstruction module for performing three-dimensional reconstruction based on the extracted features and outputting the reconstructed digestive tract model.

[0038] Beneficial effects

[0039] A method for extracting features of capsule endoscope images based on improved Harris corners provided by the present invention mainly targets the fold textures and blood vessel textures in the capsule endoscope and uses them as the source of feature points. Folds and blood vessel textures are the most stable features in the cavity, so obtaining feature points from them is stable and effective. This method first extracts feature points on the capsule endoscope images using an improved Harris response function and adaptive non-maximum suppression, adds the main direction information to the obtained feature points, constructs an rBRIEF descriptor based on this, and then uses the descriptor for feature point matching. Experimental results show that this implementation method can not only obtain a large number of feature points but also maintain high accuracy; it can perform feature extraction quickly and effectively, and can maintain the accuracy of feature matching while obtaining a sufficient number of feature points in the capsule endoscope images. In capsule endoscope images, the accuracy of obtaining sufficient feature matching points is about 20% higher than that of mainstream feature extraction methods such as SIFT, SURF, and ORB. Description of the drawings

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0041] Figure 1 is a flowchart of the method for extracting features of capsule endoscope images based on improved Harris corners of the present invention.

[0042] Figure 2is the image set for testing the present invention, where: (1.1) and (1.2) are adjacent frame pictures of the gastric wall; (2.1) and (2.2) are adjacent frame pictures of the duodenum; (3.1) and (3.2) are adjacent frame pictures before small intestine peristalsis; (4.1) and (4.2) are adjacent frame pictures after small intestine peristalsis; (5.1) and (5.2) are adjacent frame pictures of the large intestine.

[0043] Figure 3 is part of the feature point map obtained by the present invention from the Figure 2 adjacent frame dataset, where: Figure 3 the (1.1) in Figure 2 is the feature point map corresponding to (1.1) in Figure 3 the (1.2) in Figure 2 is the feature point map corresponding to (1.2) in Figure 3 the (2.1) in Figure 2 is the feature point map corresponding to (2.1) in Figure 3 the (2.2) in Figure 2 is the feature point map corresponding to (2.2) in Figure 3 the (3.1) in Figure 2 is the feature point map corresponding to (3.1) in Figure 3 the (3.2) in Figure 2 is the feature point map corresponding to (3.2) in Figure 3 the (4.1) in Figure 2 is the feature point map corresponding to (4.1) in Figure 3 the (4.2) in Figure 2 is the feature point map corresponding to (4.2) in Figure 3 the (5.1) in Figure 2 is the feature point map corresponding to (5.1) in Figure 3 the (5.2) in Figure 2 is the feature point map corresponding to (5.2) in

[0044] Figure 4 is the feature matching result map of the present invention based on the Figure 3 feature points in, where: Figure 4 the (a) in Figure 2 is the matching result map of (1.1) in Figure 2 and (1.2) in Figure 4 the (b) in Figure 2 is the matching result map of (2.1) in Figure 2 and (2.2) in Figure 4 the (c) in Figure 2 is the matching result map of (3.1) in Figure 2 and (3.2) in Figure 4 the (d) in Figure 2in (4.1) and Figure 2 matching result diagram of (4.2) in; Figure 4 in (e) is Figure 2 in (5.1) and Figure 2 matching result diagram of (5.2) in;

[0045] Figure 5 For the present invention in Figure 2 matching precision result diagram of multiple groups of adjacent frame pictures and different response function values. Specific implementation manner

[0046] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0047] Embodiment 1:

[0048] The present invention provides a method for extracting capsule endoscope image features based on improved Harris corners, including the following steps:

[0049] Step 1: Obtain two frames of capsule endoscope images, calculate the partial derivatives of each pixel point in the horizontal and vertical directions of the image, calculate the gradient value of each pixel point according to the partial derivatives, set a gradient threshold and a partial derivative threshold, and filter out the pixel points with gradient values lower than the gradient threshold;

[0050] In this step 1, in order to reduce the computational complexity of subsequent steps, points with relatively low gradient values are filtered out by setting a gradient threshold and a partial derivative threshold. After filtering by the threshold, not only can the calculation of the subsequent Harris response function value be accelerated, but also the remaining points with higher gradient values are usually the main feature information such as the texture and edge contours in the capsule endoscope image; among them, the partial derivative can be used to measure the degree of change of the image in a specific direction; the gradient value can be used to measure the degree of change of the image in all directions.

[0051] Step 2: Improve the Harris response function so that it can simultaneously obtain high-quality edge points, corner points, and a part of flat area points with differences, and also solve the problem that the Harris corners are affected by parameters; calculate the improved Harris response function value of each pixel point according to the partial derivative in the horizontal direction, the partial derivative in the vertical direction, and the gradient value obtained in step 1, and obtain potential feature points through the response function value;

[0052] Step 3: Perform non-maximum suppression on potential feature points through a sliding window. The purpose is to obtain the feature points with the largest response function value within a local range, and at the same time control the feature points within a dynamic range through non-maximum suppression; combine the improved Harris response function value with the window width and sliding distance of the sliding window to obtain an adaptive sliding window; when the response function value is relatively low, the sliding distance is small and more points are filtered. To prevent too many points from being filtered, a smaller window width is selected; conversely, when the response function value is larger, the sliding distance increases accordingly and the window width also increases accordingly; perform non-maximum suppression on the potential feature points obtained in Step 2 through the adaptive sliding window to obtain the feature points with the largest response function value within a local range;

[0053] Step 4: To enable better matching of feature points after rotation, use the gray centroid method to determine the direction of each feature point obtained in Step 3, and use this direction as the main direction of the improved Harris corner points, so that the feature points have rotation invariance; combine the improved Harris corner points with rBRIEF to form a feature matching method that can obtain a large number of feature points and ensure the matching accuracy rate in the capsule endoscope.

[0054] It should be noted that rBRIEF is an improved version of the BRIEF algorithm. rBRIEF improves the robustness and rotation invariance of the feature descriptor by modifying the point pair selection method and comparison method used in the BRIEF algorithm. The rBRIEF algorithm uses the moment method to determine the direction of the key points, and then rotates the point pairs according to the direction of the key points so that the direction of the point pairs is aligned with the direction of the key points. In addition, rBRIEF also uses the method of statistical learning to reselect the point pair set to reduce the distinguishability and correlation of the feature values. The rBRIEF algorithm is applicable to various image processing tasks, such as object detection, image classification, shape recognition, etc.

[0055] Example Two:

[0056] This example provides a capsule endoscope image feature extraction and matching method based on improved Harris corner points. See Figure 1 , including the following steps:

[0057] Step 1: Obtain two capsule endoscope pictures, solve the partial derivatives of each pixel point in the horizontal and vertical directions through convolution, calculate the gradient value of each pixel point according to the partial derivatives, set the gradient threshold and partial derivative threshold, and filter out the pixel points with gradient values lower than the gradient threshold, filtering out the wrinkle texture, blood vessel texture areas and surrounding pixel points in the capsule endoscope image. This Step 1 can not only initially filter out the relatively stable points in the capsule endoscope, but also speed up the subsequent calculation of the Harris response function value. The calculated partial derivative I in the horizontal direction x(x, y) and the partial derivative I in the horizontal direction y (x, y) is stored for subsequent step calculations;

[0058] Step 2: Using all the points in Step 1, calculate the improved Harris response function values in the image one by one; apply different weight values w(x, y) to the feature points within the 3×3 region of each pixel point to calculate the autocorrelation matrix M, and calculate the determinant det(M) and the trace trace(M) of the matrix according to M; use the determinant and the trace to calculate the improved Harris response function values, and the calculation formula is as follows:

[0059]

[0060] Among them, w(x, y) represents the weight at the point

[0061] (x, y), and I x (x, y), I y (x, y) represent the partial derivative in the horizontal direction and the partial derivative in the vertical direction at the point (x, y) respectively. The two eigenvalues of the matrix M are λ 1 , λ 2 , and the expressions of det(M) and trace(M) are respectively:

[0062] det(M) = λ 1 *λ 2 , trace(M) = λ 1 +λ 2 ;

[0063] Using the improved Harris response function, high-response points in the wrinkle and blood vessel regions can be obtained, including flat region points, edge points, and corner points with differences. These points are used as potential feature points;

[0064] Step 3: When the improved Harris response function value is small, there will be more potential feature points, and correspondingly, more points need to be filtered through non-maximum suppression; when the response function value is small, fewer points need to be filtered at this time; there are two factors affecting the non-maximum suppression effect, one is the window width of the sliding window, and the other is the sliding distance; the larger the window width, the fewer points are retained; the larger the sliding distance, the more points are retained; combine the improved Harris response function value with the window width and sliding distance of the sliding window to obtain an adaptive sliding window; when the response function value is relatively low, the sliding distance is small and more points are filtered. In order to prevent too many points from being filtered, a smaller window width is selected. On the contrary, when the response function value is large, the sliding distance increases correspondingly, and the window width also increases accordingly;

[0065] The specific steps of Step 3 are as follows:

[0066] Step 31: Divide the image into different regions according to the magnitude of the improved Harrsi response function value of each pixel point in the image calculated in Step 2;

[0067] Step 32: For each region, calculate the adaptive window width and sliding distance according to the magnitude of the response function value; when the response function value is relatively low, set a smaller window width and sliding distance; when the response function value is relatively high, set a larger window width and sliding distance;

[0068] Step 33: Perform non-maximum suppression on the image using the adaptive window width and sliding distance. The specific steps of Step 33 are as follows:

[0069] Step 331: With each pixel point as the center, delimit a sliding window using the adaptive window width;

[0070] Step 332: Move the sliding window along the adaptive sliding distance, and calculate the response function value of each pixel point within the sliding window;

[0071] Step 333: Within the sliding window, retain the pixel point with the maximum response function value and filter out other pixel points;

[0072] Step 34: Repeat Step 33 until the entire image is processed;

[0073] Through the above steps, the adaptive sliding window can dynamically adjust the window width and sliding distance according to the magnitude of the improved Harris response function value, so as to achieve non-maximum suppression of the feature points. This method can better retain the feature points in the image, while filtering out redundant pixel points, improving the accuracy and efficiency of feature extraction.

[0074] Step 4: In order to enable better matching of the feature points after rotation, assign a main direction to the improved Harris corner points to obtain the improved O-Harris corner points. The main direction of the feature points can be determined through the gray centroid method, so that the feature points have rotational invariance. In order to calculate the rBRIEF descriptor, first divide the region with the feature point as the center, and compare the pixel values of the fixed position point pairs to obtain the binary rBRIEF descriptor. In order to reduce noise interference, it is necessary to perform Gaussian filtering on the image first. After obtaining the descriptor of each feature point, perform brute-force matching on the feature descriptors to obtain a rough match;

[0075] The specific steps of Step 4 are as follows:

[0076] Step 41: Use the gray centroid method to determine the direction of each feature point obtained in Step 3, so that the feature points have rotational invariance, and use this direction as the main direction of the improved Harris corner points;

[0077] Step 42: Divide the region centered on the feature point, and compare the pixel values of the fixed-position points to obtain the binary rBRIEF descriptor. The length of the descriptor is related to the divided region. Among them, when calculating the descriptor, it is necessary to select an appropriate region size and point pairs to capture sufficient information of the feature point while reducing the length of the descriptor.

[0078] Step 43: Perform Gaussian filtering on the image to reduce noise interference. Among them, the parameters of Gaussian filtering need to be adjusted according to specific situations. If the filtering is excessive, the feature points may be smoothed out; if the filtering is insufficient, too much noise may be retained.

[0079] After obtaining the rBRIEF descriptor of each feature point, perform matching on the feature descriptors through the brute-force matching method to obtain the rough matching result. Among them, when performing brute-force matching, it is necessary to pay attention to the balance between matching accuracy and computational efficiency. If the matching accuracy is not high enough, incorrect matches may occur; if the computational efficiency is too low, it may affect real-time performance.

[0080] Through the improved O-Harris corner feature extraction and matching algorithm, the feature points in the image can be effectively identified and accurately matched. This method can be applied to various image processing tasks, such as object tracking, image registration, 3D reconstruction, etc.

[0081] Step 5: Use the rough feature matches retained in Step 4, and use the RANSAC (Random Sample Consensus) method to screen and eliminate the mismatched point pairs. Using the RANSAC method can further improve the accuracy and robustness of the matching. RANSAC is a common method for eliminating mismatches. It fits a model by randomly selecting some point pairs in the dataset, and then determines whether other point pairs are mismatched point pairs according to the distance between them and the model. In this way, the mismatched point pairs caused by factors such as noise, occlusion, and deformation can be eliminated, thereby improving the accuracy of feature matching.

[0082] Step 6: Calculate the accuracy of the feature matching by using the number of refined matches and the number of rough matches obtained by RANSAC to verify the actual effect of the present invention. Calculating the accuracy of the feature matching is one of the important indicators for evaluating the performance of the feature matching algorithm. Usually, indicators such as accuracy, recall rate, and F1 score can be used to evaluate the accuracy of the feature matching. These indicators can be calculated by comparing the rough matching result and the matching result after RANSAC screening. Through these indicators, the performance of the improved Harris corner feature extraction and matching algorithm can be quantitatively evaluated to verify its actual effect.

[0083] Based on the above specific embodiments, combined with the adjacent-frame image dataset, such as Figure 2in: (1.1) and (1.2), (2.1) and (2.2), (3.1) and (3.2), (4.1) and (4.2), (5.1) and (5.2) as shown, conduct test experiments to verify the effect of the present invention:

[0084] In the algorithm process, first input two capsule endoscope images, perform improved Harris corner feature extraction on the two input images, and the resulting feature point result diagram is as Figure 3 in: (1.1) and (1.2), (2.1) and (2.2), (3.1) and (3.2), (4.1) and (4.2), (5.1) and (5.2) as shown, and perform feature matching based on the obtained feature points, and the resulting matching result diagram is as Figure 4 in: (a), (b), (c), (d), (e) as shown.

[0085] Run and test the method of the present invention on the capsule endoscope image dataset. According to the differences in images and response function values, the present invention has the following precision performance on Figure 2 in: (1.1) and (1.2), (2.1) and (2.2), (3.1) and (3.2), (4.1) and (4.2), (5.1) and (5.2) as Figure 5 shown. The average precision of the present invention is above 70%, which is higher than that of other mainstream methods and has good adaptability in the capsule endoscope environment.

[0086] This embodiment is completed using Visual Studio 2022 and OpenCV4.5.5 under the Win10 operating system. The hardware environment is a laptop with a 2.3GHz i5 processor and 16GB of running memory, and the experimental process is relatively stable.

[0087] On the other hand, the present invention also provides a three-dimensional reconstruction method for the digestive tract of capsule endoscope images based on improved Harris corners. The three-dimensional reconstruction method for the digestive tract extracts image feature points using the above-mentioned method for extracting capsule endoscope image features based on improved Harris corners, and then performs three-dimensional reconstruction based on the extracted feature points.

[0088] On yet another aspect, the present invention also provides a three-dimensional reconstruction system for the digestive tract of capsule endoscope images based on improved Harris corners. The three-dimensional reconstruction system for the digestive tract includes:

[0089] An image acquisition module for acquiring the capsule endoscope images and performing preprocessing;

[0090] The feature extraction module uses the capsule endoscope image feature extraction method based on the improved Harris corner to extract features from the preprocessed capsule endoscope image;

[0091] The three-dimensional reconstruction module is used to perform three-dimensional reconstruction based on the extracted features and output the reconstructed digestive tract model.

[0092] Some steps in the embodiments of the present invention can be implemented by software, and the corresponding software program can be stored in a readable storage medium, such as an optical disc or a hard disk, etc.

[0093] Although the present invention has been described in detail with reference to the foregoing embodiments, for those skilled in the art, they can still modify the technical solutions described in the foregoing embodiments, or equivalently replace some of the technical features. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for extracting capsule endoscope image features based on improved Harris corners, characterized in that, the method includes: Step 1: Obtain two frames of capsule endoscope images, calculate the partial derivatives of each pixel point in the horizontal and vertical directions of the image, calculate the gradient value of each pixel point according to the partial derivatives in these two directions, set a gradient threshold and a partial derivative threshold, and filter out the pixel points with gradient values lower than the gradient threshold, filtering out the wrinkled texture, blood vessel texture area and surrounding pixel points in the capsule endoscope image; Step 2: Improve the Harris response function, calculate the improved Harrsi response function value of each pixel point according to the partial derivative in the horizontal direction, the partial derivative in the vertical direction and the gradient value obtained in Step 1, and obtain potential feature points through the response function value; Step 3: Combine the improved Harris response function value with the window width and sliding distance of the sliding window to obtain an adaptive sliding window, and perform non-maximum suppression on the potential feature points obtained in Step 2 through the adaptive sliding window to obtain the feature points with the largest response function value within a local range; Step 4: Use the gray centroid method to determine the direction of each feature point obtained in Step 3, use this direction as the main direction of the improved Harris corner, and construct an rBRIEF descriptor based on this, and use the descriptor to perform feature point matching; The calculation of the improved Harrsi response function value of each pixel point in Step 2 includes: applying different weight values w(x,y) to the feature points within the 3×3 area of each pixel point to calculate the autocorrelation matrix M, calculating the determinant det(M) and the trace trace(M) of M according to M; using the determinant and the trace to calculate the improved Harris response function value, and the calculation formula is as follows: Among them, w(x, y) represents the weight at the point (x, y), and I x (x, y), I y (x, y) represent the partial derivatives in the horizontal direction and the vertical direction at the point (x, y) respectively. The two eigenvalues of the matrix M are λ 1 , λ 2 , and the expressions of det(M) and trace(M) are det(M) = λ 1 *λ 2 , trace(M) = λ 1 +λ 2 .

2. The method for extracting capsule endoscope image features based on improved Harris corners according to claim 1, characterized in that, Step 3 specifically includes the following steps: Step 31: Divide the image into different regions according to the magnitude of the improved Harrsi response function value of each pixel point calculated in Step 2; Step 32: For each region, calculate the adaptive window width and sliding distance according to the magnitude of the response function value; When the response function value is relatively low, set a smaller window width and sliding distance; when the response function value is relatively high, set a larger window width and sliding distance; Step 33: Use the adaptive window width and sliding distance to perform non-maximum suppression on the image. Step 33 specifically includes the following steps: Step 331: With each pixel point as the center, delimit a sliding window using the adaptive window width; Step 332: Move the sliding window along the adaptive sliding distance, and calculate the response function value of each pixel point within the sliding window; Step 333: Within the sliding window, retain the pixel point with the largest response function value and filter out other pixel points; Step 34: Repeat Step 33 until the entire image is processed.

3. The method for extracting capsule endoscope image features based on improved Harris corners according to claim 2, characterized in that, In step 3, the window width W of the sliding window and the sliding distance d of the sliding window satisfy the following relationship: W = 2d - 1 Where W represents the window width of the sliding window, d represents the sliding distance of the sliding window, and the expression of d is R is the Harrsi response function value calculated in step 2.

4. The method for extracting capsule endoscope image features based on improved Harris corners according to claim 3, wherein, step 4 specifically includes the following steps: Step 41: Use the gray centroid method to determine the direction of each feature point obtained in step 3, so that the feature point has rotational invariance, and use this direction as the main direction of the improved Harris corner; Step 42: Divide the area centered on the feature point, compare the pixel values of fixed-position point pairs to obtain the binary rBRIEF descriptor, and the length of the descriptor is related to the divided area; Step 43: Perform Gaussian filtering on the image to reduce noise interference; Step 44: After obtaining the rBRIEF descriptor of each feature point, match the feature descriptors through the brute-force matching method to obtain the rough matching result.

5. The method for extracting capsule endoscope image features based on improved Harris corners according to claim 4, wherein, the potential feature points in step 2 are: using the improved Harris response function to obtain high-response points in the folded and vascular regions, including flat region points, edge points and corner points with differences; in step 4, a 256-dimensional rBRIEF descriptor is used to construct a descriptor for matching.

6. The method for extracting capsule endoscope image features based on improved Harris corners according to claim 5, wherein, it further includes step 5, and step 5 includes: using the feature rough matching reserved in step 4, and using the RANSAC method to screen and remove mismatched point pairs.

7. The method for extracting capsule endoscope image features based on improved Harris corners according to claim 6, wherein, it further includes step 6, and step 6 includes: calculating the accuracy of feature matching by using the number of accurate matches and the number of rough matches obtained by RANSAC.

8. A three-dimensional reconstruction method for the digestive tract of a capsule endoscope image based on improved Harris corners, wherein, the three-dimensional reconstruction method for the digestive tract uses the method for extracting capsule endoscope image features based on improved Harris corners according to any one of claims 1-7 to extract image feature points, and then performs three-dimensional reconstruction according to the extracted feature points.

9. A three-dimensional reconstruction system for the digestive tract of a capsule endoscope image based on improved Harris corners, wherein, the three-dimensional reconstruction system for the digestive tract includes: an image acquisition module for acquiring the capsule endoscope image and performing preprocessing; a feature extraction module that uses the method for extracting capsule endoscope image features based on improved Harris corners according to any one of claims 1-7 to extract features from the preprocessed capsule endoscope image; a three-dimensional reconstruction module for performing three-dimensional reconstruction according to the extracted features and outputting the reconstructed digestive tract model.

Images