Image stabilization correction method for capsule gastroscopy image acquisition

Abstract

The present application discloses an image stabilization correction method for capsule gastroscopy image acquisition, which relates to the technical field of medical diagnosis, and includes the following steps: image acquisition: taking the original image of the stomach wall through a capsule endoscope; image preprocessing: performing data enhancement processing on the original image, and then performing plane projection processing to obtain a two-dimensional projection plane image; generating a coordinate sequence: extracting a feature point sequence of the two-dimensional projection plane image, and generating a matching feature point pair coordinate sequence; correcting the lens posture: by calculating the lens posture, generating an image data sequence after posture correction, and correcting the lens posture; generating a stable image sequence: by calculating the path of the device, forming a lens space posture control sequence, thereby eliminating images that deviate from the predetermined path, and forming stable and reliable image data. The present application can make the acquired image more stable and reduce the impact of rotation and shaking on the image.

Description

Technical Field

[0001] The present application relates to the technical field of medical diagnosis, and in particular to an image stabilization correction method for capsule gastroscopy image acquisition. Background Art

[0002] Medical endoscopes play an important role in the diagnosis and treatment of digestive tract diseases. Traditional endoscopy usually requires doctors to insert a long cable into the patient's body to image key areas and provide diagnosis and treatment plans. However, this wired endoscope may cause discomfort and pain to patients due to its invasive nature. In addition, the use of such endoscopes also carries the risk of potential complications, such as infection, perforation, and tearing.

[0003] Capsule Endoscopy (CE) is an innovative medical device used in the field of modern gastroenterology diagnosis and treatment. It is usually composed of a miniature camera, a housing, a lighting system, a control circuit, a wireless communication module, and a battery. It examines the digestive tract in a painless, non-invasive way. Among them, capsule gastroscopy refers to a capsule endoscope specifically used for exploring the stomach area, which can scan and photograph the stomach area in detail. However, during each detection process, the capsule gastroscopy needs to move close to the stomach wall, which will inevitably generate a torque caused by friction, causing the lens to rotate and shake with the capsule, bringing difficulties to subsequent image acquisition and stitching. Summary of the invention

[0004] The embodiment of the present application provides an image stabilization correction method for capsule gastroscopy image acquisition, which solves the problem that the lens of the capsule endoscope rotates and shakes with the capsule, causing difficulties in subsequent image acquisition and stitching.

[0005] In order to solve the above technical problems, this application is implemented as follows:

[0006] An image stabilization correction method for capsule gastroscopy image acquisition is provided, which comprises the following steps:

[0007] Image acquisition: The original image of the inside of the stomach wall is captured by capsule endoscope;

[0008] Preprocessing the image: performing data enhancement processing on the original image, and then performing plane projection processing to obtain a two-dimensional projection plane image;

[0009] Generate coordinate sequence: extract the feature point sequence of the two-dimensional projection plane image and generate the coordinate sequence of matching feature point pairs;

[0010] Correcting the lens posture: by calculating the lens posture, generating a posture-corrected image data sequence, and correcting the lens posture;

[0011] Generate a stable image sequence: By calculating the path of the device, a lens space posture control sequence is formed, and images that deviate from the predetermined path are eliminated to form stable and reliable image data.

[0012] In one of the embodiments, in the step of performing data enhancement processing on the original image: the original image data collected in the image acquisition step is used as input and sequentially input into the image preprocessing stage, the original image data is subjected to noise reduction processing to reduce interference when retrieving feature points, and the image is subjected to color space transformation.

[0013] In one embodiment, in the step of performing planar projection processing on the original image:

[0014] Calculate the sampling distance h of the capsule endoscope: ,in is the scaling factor, ranging from ;

[0015] After obtaining the sampling distance, the original coordinates Transform to new coordinates:

[0016] ;

[0017] Taking the center of the image as the origin, crop the length and width of the image to 0.71 of the original ,in .

[0018] In one embodiment, in the step of extracting a sequence of feature points of a two-dimensional projection plane image: the image processed by plane projection is taken as input, and the sequence of these images is recorded as: , a total of M elements, corresponding to M image data collected at M moments;

[0019] Use SIFT, SURF or ORB algorithm to extract feature points, image sequence Generate M feature point sequences, recorded as: .

[0020] In one embodiment, in the step of generating a matching feature point pair coordinate sequence:

[0021] Take the extracted feature point sequence as input, and use FLANN to combine the matching feature points of two adjacent image data into an array match to store matching feature point pairs:

[0022] , where k represents the time, corresponding to the sequence number in FP, and the feature points corresponding to M times constitute M-1 matching feature point pairs:

[0023] MATCH=[ ], where each element The coordinates are expressed as:

[0024]

[0025] Where m represents the number of coordinate pairs of each set of matching points, that is There are m matching point pairs. All coordinates in the center are transformed, that is, ,in The subtable is the width and height of the image, there are M-1 The resulting queue is called a matching feature point pair coordinate sequence, denoted as MATCH.

[0026] In one embodiment, in the step of correcting the lens posture:

[0027] Eliminate axial displacement: solve the axial displacement that minimizes the distance between the corresponding feature points of two images at adjacent moments, obtain the axial displacement of the lens at the current moment, then eliminate the axial displacement and align the rotation center;

[0028] Eliminate the rotation angle: Find the rotation angle that minimizes the distance between the corresponding feature points of two images at adjacent moments, obtain the rotation angle of the lens at the current moment, and then eliminate the rotation angle;

[0029] Correct the tilt posture: solve the tilt angle that minimizes the distance between the corresponding feature points of two images at adjacent moments, obtain the tilt angle of the lens at the current moment, and then correct the tilt posture.

[0030] In one embodiment, in the step of eliminating axial displacement:

[0031] Calculate the axial displacement: Take the obtained matching feature point pair coordinate sequence MATCH as input, calculate the axial displacement of two adjacent images along the X-axis and Y-axis, and calculate the displacement of each element in MATCH. The coordinates of the matching point pairs of the two images at adjacent moments in the image are used to calculate the displacement of the two images along the X-axis direction at adjacent moments: , the displacement difference along the Y axis is: ;

[0032] Eliminate axial displacement: Take the X and Y axis displacements obtained in the step of calculating the axial displacement and the matching point pair coordinate spectrum sequence MATCH obtained in the step of generating the matching feature point pair coordinate sequence as input, and replace each element in MATCH The characteristic point coordinates at the k+1 moment eliminate the axial displacement , stored in MATCH_NORM, that is:

[0033] ,

[0034] Next, the displacement is eliminated. All coordinates in are normalized and stored in MATCH_NORM, which is used as the rotation center for the aligned image.

[0035] In one embodiment, in the step of eliminating the rotation angle:

[0036] Calculate the rotation angle: Take MATCH_NORM as input and assume that the image at time k+1 is rotated If the position of the regularized feature point after degrees coincides with the regularized feature point of the image at time k, it means that the image at time k has been rotated - Get the image at time k+1 and use Newton's method to solve it The minimum value of is the Gaussian coefficient: , The value range is , yes The coordinates of the regularized feature points of the image at time k+1 Rotation After getting, through: Calculated;

[0037] The rotation angle of the image at adjacent moments is obtained Then, rotate the image at time k+1 - To keep the rotation angle unchanged, each The calculated angle use Mark, considering the accumulation of rotation, the actual rotation correction angle is , save the Gaussian coefficient sequence: ;

[0038] Eliminate rotation: Take the obtained rotation angle as input and replace each The image rotation at the corresponding k+1 moment ,

[0039] , record the new sequence as MATCH_ANTIROT;

[0040] The Gaussian coefficients saved in the step of calculating the rotation angle are compared with each in MATCH_ANTIROT The corresponding relationship is:

[0041] ,

[0042] When the Gaussian coefficient of the corresponding feature point , delete the corresponding The coordinate pairs of the matching feature points in are obtained to obtain the MATCH_ANTIROT after the rows are deleted.

[0043] In one embodiment, in the step of correcting the tilt posture:

[0044] Calculate the tilt angle: Take the obtained center-aligned and rotation-corrected coordinate sequence MATCH_ANTIROT as input, perform deflection correction on the image, and assume that the image at time k+1 is tilted by ,in Indicates the tilt around the X axis, Indicates the tilt around the Y axis, solved using Newton's method

[0045] The minimum value of is the tilt angle of the image at time k+1, each Calculated tilt angle use Mark, the actual corrected tilt angle is , tilt angle sequence SL= ;

[0046] Eliminate tilt: Take the obtained tilt angle sequence SL as input, and for each Coordinates within Multiply by the corresponding To correct the tilt,

[0047] , update the coordinates of the corrected tilt angle in MATCH_ADJ.

[0048] In one embodiment, in the step of generating a stable image sequence:

[0049] Calculate the relative path: The scanning path of the capsule gastroscope on the inner wall of the stomach cavity is controlled by the device, and the posture of the device is controlled by a preset posture parameter sequence. The posture parameter sequence is expressed as: , each element represents the displacement value at time j, is the relative displacement at time j, relative to the initial position The absolute displacement of: , about the absolute pose sequence at time j:

[0050] , transform the device coordinate system to relative to the initial position The world coordinate system of , the world coordinate corresponding to time j is:

[0051] , with the initial position The center of the device is the origin, and the origin is the world coordinate at time j The line regularization of , we get the unit pose vector: , the world coordinate sequence can be expressed as: , the displacement difference between adjacent moments is: , and get n-1 shift difference sequences: , the reference plane normal vector of the displacement from time j to time j+1 is: , the reference plane is: ;

[0052] Image sequence filtering: The relative path and the corresponding reference plane, as well as the tilt-corrected MATCH_ADJ are used as input. Each time the device generates a displacement , capsule endoscopy collection The corresponding relationship between the number of elements in the image sequence IMG and the number of elements in the device displacement difference sequence DP is: , the jth displacement Collected from The image data should be along the The direction of ,vertical And it is The negative direction is recorded as , the displacement vector The calculation formula is:

[0053] , taking the displacement difference as output, calculate the previous The sum of the X and Y axis displacements of the image: ,in Indicates before MATCH_ADJ Coordinate displacement difference within the match element, correction angle The following formula can be used for calculation: , the relative displacement difference of the device at time j after correction is: , for time j The displacement difference of the adjacent moments of the image data, that is, the displacement difference of the adjacent moments of the image data in MATCH_ADJ The displacement difference recorded in: ,in , the deviation of each displacement The following formula can be used for calculation: Given a threshold greater than 0.5, , any Corresponding image Put it into IMG_LIST, the list IMG_LIST is the image sequence finally required.

[0054] In an embodiment of the present application, real-time correction is performed for the rotation and displacement generated by the capsule when collecting stomach image data. By matching and calculating the feature points in the images at adjacent moments, the position change of the capsule is accurately estimated and adjusted accordingly, so that the collected image is more stable and the impact of rotation and shaking on the image is reduced. At the same time, the present application uses a plane projection transformation in the image preprocessing stage to stretch the spherical image into a plane image, highlight the feature information in the stomach wall environment, and ensure the accuracy of subsequent feature point matching. The present application eliminates unnecessary interference in the gastric cavity environment through data enhancement, noise reduction and other technologies, improves the extraction and matching efficiency of image feature points, and enables the algorithm to maintain good performance in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0056] Figure 1 is a flowchart of the steps of the image stabilization correction method for capsule gastroscopy image acquisition of the present application;

[0057] Figure 2 It is a flow chart of an embodiment of the present application. DETAILED DESCRIPTION

[0058] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0059] See also Figure 1 , which is a flowchart of the steps of the image stabilization correction method for capsule gastroscopy image acquisition of the present application; as shown in the figure, the image stabilization correction method S for capsule gastroscopy image acquisition includes the following steps S1 to S5. Among them:

[0060] Step S1, image acquisition: the original image of the inside of the stomach wall is captured by a capsule endoscope. In the image acquisition stage, the original image of the inside of the stomach wall is first captured by a capsule endoscope. The image is captured in such a way that the capsule endoscope moves along a predetermined path along the stomach wall under the traction of magnetic force, and the built-in lens is shot with the assistance of a fill light LED to obtain a sequence of stomach wall images. However, in practice, there are many interference factors in the scanning route of the capsule. First, the capsule needs to be close to the stomach wall, which will inevitably generate friction, causing the capsule to deflect along the path of travel and rotate along its own axis. Secondly, there are a large number of wrinkles on the stomach wall, and the protruding wrinkles cause the capsule to slip tangentially or deflect the lens in the vertical direction. At the same time, the liquid-filled environment in the stomach cavity will also cause the capsule posture to fluctuate with the intermittent peristalsis of the stomach wall. Therefore, in order to facilitate image stitching, the image sequence needs to be screened and corrected in subsequent steps S2 to S5.

[0061] Step S2, preprocessing the image: perform data enhancement processing on the original image, and then perform plane projection processing to obtain a two-dimensional projection plane image. Data enhancement is to eliminate noise and reduce unexpected interference when selecting feature points. At the same time, due to the particularity of the gastric cavity environment, the differences in contour, color, and texture are not obvious, and it is necessary to highlight the image features suitable for the gastric cavity environment through data enhancement. The process of the capsule taking images on the inner side of the stomach wall can be regarded as taking pictures inside a sphere. Therefore, it is necessary to project the sphere into a plane, and crop and extract the data within a certain range of the middle part of the image as the effective area, so as to reduce the size distortion problem caused by the projection.

[0062] In the data enhancement step, the image data collected in step S1 is used as input and sequentially input into the image preprocessing stage. In order to enhance the recognition ability of feature points, the original image data needs to be subjected to noise reduction processing to reduce interference when retrieving feature points. At the same time, in view of the single color of the internal environment of the gastric cavity, the image needs to be transformed in color space, such as brightness adjustment, contrast enhancement, etc.

[0063] In the plane projection step, the image after the above data enhancement is used as input for plane projection transformation. The image captured by the lens in the capsule endoscope is a spherical surface, and projection transformation is needed to stretch the spherical image into a plane image. First, calculate the sampling distance h of the capsule endoscope: ,in is the scaling factor, ranging from , You can select an appropriate value based on the actual situation. After obtaining the sampling distance, the original coordinates Transform to new coordinates:

[0064] ;

[0065] After stretching the original coordinates to the new coordinates, there will be "gaps", which are filled by linear interpolation. Taking the center of the image as the origin, the length and width of the image are cropped to 0.71 of the original ,in , It can be selected according to actual situation.

[0066] In step S3, a coordinate sequence is generated: a feature point sequence of the two-dimensional projection plane image is extracted, and a coordinate sequence of matching feature point pairs is generated. In the step of extracting a feature point sequence of the two-dimensional projection plane image, the image processed by the plane projection in step S2 is taken as input, and these image sequences are recorded as: , a total of M elements, corresponding to M image data collected at M times, using SIFT, SURF or ORB algorithm to extract feature points, image sequence Generate M feature point sequences, recorded as: .

[0067] In the step of generating a coordinate sequence of matching feature point pairs, the extracted feature point sequence is used as input, and FLANN is used to combine the matching feature points of two adjacent image data into an array match to store matching feature point pairs: , where k represents the time, corresponding to the sequence number in FP, and the feature points corresponding to M times constitute M-1 matching feature point pairs:

[0068] MATCH=[ ], where each element The coordinates are expressed as: , where m represents the number of coordinate pairs of each set of matching points, that is There are m matching point pairs. All coordinates in the center are transformed, that is, ,in The subtable is the width and height of the image, there are M-1 The resulting queue is called a matching feature point pair coordinate sequence, denoted as MATCH.

[0069] Step S4, correcting the lens posture: by calculating the lens posture, generating an image data sequence after posture correction, and correcting the lens posture. First, eliminate the axial displacement: solve the axial displacement that minimizes the distance between the corresponding feature points of the two images at adjacent moments, obtain the axial displacement of the lens at the current moment, then eliminate the axial displacement, and align the rotation center. Next, eliminate the rotation angle: solve the rotation angle that minimizes the distance between the corresponding feature points of the two images at adjacent moments, obtain the rotation angle of the lens at the current moment, and then eliminate the rotation angle. Finally, correct the tilt posture: solve the tilt angle that minimizes the distance between the corresponding feature points of the two images at adjacent moments, obtain the tilt angle of the lens at the current moment, and then correct the tilt posture.

[0070] In the step of eliminating axial displacement: First, calculate the axial displacement: take the matching feature point pair coordinate sequence MATCH obtained in step S3 as input, calculate the axial displacement of two adjacent images along the X-axis and Y-axis, and calculate the axial displacement of each element in MATCH. The coordinates of the matching point pairs of the two images at adjacent moments in the image are used to calculate the displacement of the two images along the X-axis direction at adjacent moments: , the displacement difference along the Y axis is: Next, eliminate the axial displacement: take the X and Y axis displacements obtained in the step of calculating the axial displacement and the matching point pair coordinate spectrum sequence MATCH obtained in the step of generating the matching feature point pair coordinate sequence in step S3 as input, and replace each element in MATCH The characteristic point coordinates at the k+1 moment eliminate the axial displacement , stored in MATCH_NORM, that is:

[0071] , then, the displacement is eliminated All coordinates in are normalized and stored in MATCH_NORM. The significance of the process of obtaining MATCH_NORM can be regarded as the rotation center of the aligned image, that is, MATCH_NORM is used as the rotation center of the aligned image.

[0072] In the step of eliminating the rotation angle: First, calculate the rotation angle: take MATCH_NORM as input, assuming that the image at time k+1 is rotated If the position of the regularized feature point after degrees coincides with the regularized feature point of the image at time k, it means that the image at time k has been rotated - Get the image at time k+1. Solve using Newton's method The minimum value of is the Gaussian coefficient: , The value range is , It can be selected according to actual situation. yes The coordinates of the regularized feature points of the image at time k+1 Rotation After getting, through: Calculate and get the rotation angle of the image at adjacent moments Then, rotate the image at time k+1 - To keep the rotation angle unchanged, each The calculated angle use Mark, considering the accumulation of rotation, the actual rotation correction angle is , save the Gaussian coefficient sequence: Next, eliminate the rotation: take the obtained rotation angle as input and replace each The image rotation at the corresponding k+1 moment ,

[0073] , and record the new sequence as MATCH_ANTIROT. The Gaussian coefficients saved in the step of calculating the rotation angle are the same as each one in MATCH_ANTIROT. The corresponding relationship is:

[0074] Given a threshold greater than 0.99, here we take , when the Gaussian coefficient of the corresponding feature point , delete the corresponding The coordinate pairs of the matching feature points in are obtained to obtain the MATCH_ANTIROT after the rows are deleted.

[0075] In the step of correcting the tilt posture: First, calculate the tilt angle: use the obtained center-aligned and rotation-corrected coordinate sequence MATCH_ANTIROT as input to perform deflection correction on the image. "Deflection" can be understood as tilting. When the capsule slides over the stomach wall, due to obstruction or friction, the tilt angle [insert picture] between its own axis and the stomach wall will change. The change in the tilt angle of the capsule's axis can be decomposed into two tilt angle components about the X and Y axes. Because the input coordinate data has been center-aligned, only the influence of the tilt needs to be considered. Suppose the image at time k+1 is tilted compared to the image at time k. ,in Indicates the tilt around the X axis, Indicates the tilt around the Y axis, solved using Newton's method The minimum value of That is, the tilt angle of the image at time k+1. Calculated tilt angle use Mark, the actual corrected tilt angle is , tilt angle sequence SL = Next, eliminate the tilt: take the obtained tilt angle sequence SL as input, and for each Coordinates within Multiply by the corresponding To correct the tilt,

[0076] , update the coordinates of the corrected tilt angle in MATCH_ADJ. At this time, the coordinate pairs in MATCH_ADJ have all been corrected for posture.

[0077] In step S5, a stable image sequence is generated: by calculating the path of the device, a lens space posture control sequence is formed, and images that deviate from the predetermined path are eliminated accordingly to form stable and reliable image data. First, the relative path is calculated: the scanning path of the capsule gastroscope on the inner wall of the stomach cavity is controlled by the device, and the posture of the device is controlled by a preset posture parameter sequence, which is expressed as: , each element represents the displacement value at time j, is the relative displacement at time j, relative to the initial position The absolute displacement of: , about the absolute pose sequence at time j:

[0078] , transform the device coordinate system to relative to the initial position The world coordinate system of , the world coordinate corresponding to time j is:

[0079] , with the initial position The center of the device is the origin, and the origin is the world coordinate at time j The line regularization of , we get the unit pose vector: , the world coordinate sequence can be expressed as: , the displacement difference between adjacent moments is: , and get n-1 shift difference sequences: , the reference plane normal vector of the displacement from time j to time j+1 is: , the reference plane is: ;

[0080] Next, the image sequence is filtered: the relative path and the corresponding reference plane, as well as the tilt-corrected MATCH_ADJ are used as input, and the device generates a displacement every time , capsule endoscopy collection The corresponding relationship between the number of elements in the image sequence IMG and the number of elements in the device displacement difference sequence DP is: , the jth displacement Collected from The image data should be along the The direction of ,vertical And it is The negative direction is recorded as , the displacement vector The calculation formula is:

[0081] Considering the initial posture of the gastroscope capsule, the rotation angle of the capsule lens is required to correct the displacement in the world coordinate system. Relative displacement direction. Take the displacement difference as output and calculate the The sum of the X and Y axis displacements of the image: ,in Indicates before MATCH_ADJ Coordinate displacement difference within the match element, correction angle The following formula can be used for calculation: , the relative displacement difference of the device at time j after correction is: , for time j The displacement difference of the adjacent moments of the image data, that is, the displacement difference of the adjacent moments of the image data in MATCH_ADJ The displacement difference recorded in: ,in , the deviation of each displacement The following formula can be used for calculation: Given a threshold greater than 0.5, , any Corresponding image Put it into IMG_LIST, the list IMG_LIST is the image sequence finally required.

[0082] As described above, the image stabilization correction method for capsule gastroscopy image acquisition of this embodiment performs real-time correction for the rotation and displacement generated by the capsule when collecting gastric image data, and accurately estimates the change in the position of the capsule and makes corresponding adjustments by matching and calculating the feature points in the images at adjacent moments, so that the collected image is more stable and the impact of rotation and shaking on the image is reduced. At the same time, the present application uses a plane projection transformation in the image preprocessing stage to stretch the spherical image into a plane image, highlight the feature information in the gastric wall environment, and ensure the accuracy of subsequent feature point matching. This embodiment eliminates unnecessary interference in the gastric cavity environment through data enhancement, noise reduction and other technologies, improves the efficiency of extracting and matching image feature points, and enables the algorithm to maintain good performance in complex environments.

[0083] The image stabilization correction method for capsule gastroscopy image acquisition of this embodiment has the following advantages: 1. No additional hardware is required to achieve stable image output: At present, many capsule gastroscopy stabilization technologies on the market rely on external sensors, mechanical structures or expandable materials and other stabilization components, which increase the complexity and cost of the equipment. However, this embodiment adopts a pure image processing method to achieve image stability through feature point matching and algorithm calculation, without changing the hardware structure of the capsule gastroscopy. This not only greatly reduces the production and maintenance costs, but also makes the equipment more lightweight and suitable for a variety of medical scenarios.

[0084] 2. Higher precision and stronger adaptability: This embodiment uses Newton's method to solve the rotation angle of adjacent image frames, which can minimize the error in the image feature point matching process. Compared with the traditional rotation correction method, Newton's method can determine the rotation angle more accurately to ensure that the corrected image will not produce obvious deviation. In addition, through the feature point center alignment and tilt correction technology, this embodiment can effectively cope with the complex and changeable environment inside the gastric cavity and ensure the stability of image output.

[0085] 3. High algorithm efficiency and strong real-time performance: In the process of medical diagnosis, the real-time performance of images is crucial. The algorithm of this implementation can complete the processing of each frame of images in a short time by matching and correcting feature points, meeting the real-time requirements. Compared with the solution that requires external sensor data assistance, this implementation relies entirely on the image's own information, reducing the time for data transmission and processing and improving the working efficiency of the entire system.

[0086] 4. Improve the accuracy of image stitching and subsequent processing: Traditional capsule gastroscopy often causes the collected image sequence to be misaligned and overlapped during the stitching process due to rotation and shaking, affecting the doctor's diagnosis of gastric diseases. However, this implementation method makes the final output image clearer and more coherent by correcting the stability of the image sequence, greatly improving the accuracy of image stitching and providing more accurate data support for subsequent diagnosis and treatment.

[0087] 5. Wide application range and strong popularity: Since the design of this embodiment does not rely on the specific structure of the capsule and external stabilization equipment, it can not only be applied to capsule gastroscopy, but also can be extended to other types of capsule endoscopes or medical equipment that requires image stabilization, and has strong versatility and application potential.

[0088] The following is a description of the beneficial effects of the image stabilization correction method for capsule gastroscopy image acquisition in combination with specific embodiments.

[0089] Example

[0090] See also Figure 2 , which is a flow chart of an embodiment of the present application; as shown in the figure, this embodiment calculates the posture of the capsule gastroscope camera through the image and corrects the posture and relative displacement of the image. The embodiment flow is as follows:

[0091] 1. Image acquisition and data set construction: Use a magnetic control device to pull the capsule to move in the stomach, record the control parameters of the device's motion path, and collect gastric wall image data obtained by capsule gastroscopy.

[0092] 2. Image preprocessing: Enhance the original image data set and use projection transformation to stretch the spherical image taken by capsule gastroscopy into a plane image, and crop the image in the center as the final image data.

[0093] 3. Generate a coordinate sequence of image matching feature point pairs. Use algorithms such as SIFT, SURF, or ORB to capture image feature points and match them. Move the coordinate center of the matching point pair from the upper left corner to the center of the image as direct data for subsequent processing.

[0094] 4. Capsule gastroscopy camera posture feedback and image posture correction. The axial displacement, rotation angle, and tilt angle of the image at adjacent moments are calculated based on the position of the feature points in the image, the axial displacement of the image is directly eliminated, and the rotation and tilt of the image are corrected using the accumulated rotation angle and tilt angle based on the moment.

[0095] 5. Correct the image. Convert the control parameters of the magnetic control device into relative position change, and then convert the relative position change into world coordinates and project it into the spherical coordinate system to obtain the preset image relative displacement path. Calculate the relative displacement of the image at adjacent moments and the correlation of the relative path of the magnetic control device, filter out the images that deviate from the predetermined path, and form a stable image data set.

[0096] In summary, the present application provides an image stabilization correction method for capsule gastroscopy image acquisition, which performs real-time correction for the rotation and displacement generated by the capsule when collecting gastric image data, and accurately estimates the position change of the capsule by matching and calculating the feature points in the image at adjacent moments and makes corresponding adjustments, so that the acquired image is more stable and the impact of rotation and shaking on the image is reduced. At the same time, the present application uses a plane projection transformation in the image preprocessing stage to stretch the spherical image into a plane image, highlight the feature information in the gastric wall environment, and ensure the accuracy of subsequent feature point matching. The present application eliminates unnecessary interference in the gastric cavity environment through data enhancement, noise reduction and other technologies, improves the extraction and matching efficiency of image feature points, and enables the algorithm to maintain good performance in complex environments.

[0097] This application uses Newton's method to solve the optimal rotation angle in the algorithm design, which can eliminate the errors generated in the feature point selection and matching process to the greatest extent and achieve high-precision image correction. At the same time, by aligning the centers of the feature points, the impact of displacement changes on the rotation calculation is minimized, making the correction process more efficient. In addition, this application abandons the reliance on external sensors and achieves stable control purely through image data, greatly reducing hardware cost and complexity. Compared with the prior art, this application completely transfers the lens stability control to the algorithm level, and constructs a complete stability control solution by matching feature points, displacement correction, and rotation correction between adjacent image frames. This algorithm-level control method does not require the addition of additional hardware structures, reduces the design difficulty and cost of the capsule, and can achieve high-precision correction of lens rotation and shaking in complex environments.

[0098] In addition, since the slippage and rotation of the capsule on the stomach wall will cause errors in the image stitching process, this application accurately calculates and corrects the displacement and rotation of the feature points, so that the final acquired image sequence can be kept in a plane parallel to the stomach wall, which is convenient for subsequent image stitching and processing, and improves the continuity and accuracy of the image data. Through the above improvements, this application not only solves the rotation and shaking problems in the process of capsule gastroscopy image acquisition to a large extent, but also provides a low-cost, high-efficiency, image stabilization output method suitable for complex stomach environments, which has broad application prospects and promotion value.

[0099] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0100] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. An image stabilization correction method for capsule gastroscopy image acquisition, characterized in that: The following steps are involved: Image acquisition: The original image of the inside of the stomach wall is captured by capsule endoscope; Preprocessing the image: performing data enhancement processing on the original image, and then performing plane projection processing to obtain a two-dimensional projection plane image; Generate coordinate sequence: extract the feature point sequence of the two-dimensional projection plane image and generate the coordinate sequence of matching feature point pairs; Correcting the lens posture: by calculating the lens posture, generating a posture-corrected image data sequence, and correcting the lens posture; Generate stable image sequence: By calculating the path of the device, a lens space posture control sequence is formed, and images that deviate from the predetermined path are eliminated to form stable and reliable image data. In the steps of extracting a sequence of feature points of a two-dimensional projection plane image: Take the image processed by the planar projection as input, and record these image sequences as: , a total of M elements, corresponding to M image data collected at M moments; Use SIFT, SURF or ORB algorithm to extract feature points, image sequence Generate M feature point sequences, recorded as: ， In the steps of generating a sequence of matching feature point coordinates: Take the extracted feature point sequence as input, and use FLANN to combine the matching feature points of two adjacent image data into an array match to store matching feature point pairs: , where k represents the time, corresponding to the sequence number in FP, and the feature points corresponding to M times constitute M-1 matching feature point pairs: MATCH=[ ], where each element The coordinates are expressed as: Where m represents the number of coordinate pairs of each set of matching points, that is There are m matching point pairs, and for each All coordinates in the center are transformed, that is, ,in The subtable is the width and height of the image, there are M-1 The resulting queue is called a matching feature point pair coordinate sequence, denoted as MATCH.

2. The image stabilization correction method for capsule gastroscopy image acquisition according to claim 1, characterized in that: In the step of performing data enhancement processing on the original image: The original image data collected in the image collection step is used as input and sequentially input into the image preprocessing stage, the original image data is subjected to noise reduction processing to reduce interference when retrieving feature points, and the image is subjected to color space transformation.

3. The image stabilization correction method for capsule gastroscopy image acquisition according to claim 2, characterized in that: In the step of performing plane projection processing on the original image: Calculate the sampling distance h of the capsule endoscope: ,in is the scaling factor, ranging from ; After obtaining the sampling distance, the original coordinates Transform to new coordinates: ； Taking the center of the image as the origin, crop the length and width of the image to 0.71 of the original ,in .

4. The image stabilization correction method for capsule gastroscopy image acquisition according to claim 3, characterized in that: Steps to calibrate the lens posture: Eliminate axial displacement: solve the axial displacement that minimizes the distance between the corresponding feature points of two images at adjacent moments, obtain the axial displacement of the lens at the current moment, then eliminate the axial displacement and align the rotation center; Eliminate the rotation angle: Find the rotation angle that minimizes the distance between the corresponding feature points of two images at adjacent moments, obtain the rotation angle of the lens at the current moment, and then eliminate the rotation angle; Correct the tilt posture: solve the tilt angle that minimizes the distance between the corresponding feature points of two images at adjacent moments, obtain the tilt angle of the lens at the current moment, and then correct the tilt posture.

5. The image stabilization correction method for capsule gastroscopy image acquisition according to claim 4, characterized in that: To eliminate axial displacement: Calculate the axial displacement: Take the obtained matching feature point pair coordinate sequence MATCH as input, calculate the axial displacement of two adjacent images along the X-axis and Y-axis, and calculate the displacement of each element in MATCH. The coordinates of the matching point pairs of the two images at adjacent moments in the image are used to calculate the displacement of the two images along the X-axis direction at adjacent moments: , the displacement difference along the Y axis is: ； Eliminate axial displacement: Take the X and Y axis displacements obtained in the step of calculating the axial displacement and the matching point pair coordinate spectrum sequence MATCH obtained in the step of generating the matching feature point pair coordinate sequence as input, and replace each element in MATCH The coordinates of the characteristic points at the k+1 moment eliminate the axial displacement , stored in MATCH_NORM, that is: ， Next, the displacement is eliminated. All coordinates in are normalized and stored in MATCH_NORM, which is used as the rotation center for the aligned image.

6. The image stabilization correction method for capsule gastroscopy image acquisition according to claim 5, characterized in that: To eliminate the rotation angle: Calculate the rotation angle: Take MATCH_NORM as input and assume that the image at time k+1 is rotated If the position of the regularized feature point after degrees coincides with the regularized feature point of the image at time k, it means that the image at time k has been rotated - Get the image at time k+1 and use Newton's method to solve it The minimum value of is the Gaussian coefficient: ， The value range is , yes The coordinates of the regularized feature points of the image at time k+1 Rotation After getting, through: Calculated; The rotation angle of the image at adjacent moments is obtained Then, rotate the image at time k+1 - To keep the rotation angle unchanged, each The calculated angle use Mark, considering the accumulation of rotation, the actual rotation correction angle is , save the Gaussian coefficient sequence: ; Eliminate rotation: Take the obtained rotation angle as input and replace each The image rotation at the corresponding k+1 moment , , record the new sequence as MATCH_ANTIROT; calculate the Gaussian coefficients saved in the step of rotating the angle and each in MATCH_ANTIROT The corresponding relationship is: , when the Gaussian coefficient of the corresponding feature point , delete the corresponding The coordinate pairs of the matching feature points in are obtained to obtain the MATCH_ANTIROT after the rows are deleted.

7. The image stabilization correction method for capsule gastroscopy image acquisition according to claim 6, characterized in that: To correct the tilt posture: Calculate the tilt angle: Take the obtained center-aligned and rotation-corrected coordinate sequence MATCH_ANTIROT as input, perform deflection correction on the image, and assume that the image at time k+1 is tilted by ,in Indicates the tilt around the X axis, Indicates the tilt around the Y axis, solved using Newton's method The minimum value of is the tilt angle of the image at time k+1, each Calculated tilt angle use Mark, the actual corrected tilt angle is , tilt angle sequence SL= ; Eliminate tilt: Take the obtained tilt angle sequence SL as input, and for each Coordinates within Multiply by the corresponding To correct the tilt, , update the coordinates of the corrected tilt angle in MATCH_ADJ.

8. The image stabilization correction method for capsule gastroscopy image acquisition according to claim 7, characterized in that: The steps to generate a stable image sequence are: Calculate the relative path: The scanning path of the capsule gastroscope on the inner wall of the stomach cavity is controlled by the device, and the posture of the device is controlled by a preset posture parameter sequence. The posture parameter sequence is expressed as: , each element represents the displacement value at time j, is the relative displacement at time j, relative to the initial position The absolute displacement of: , about the absolute pose sequence at time j: , transform the device coordinate system to relative to the initial position The world coordinate system of , the world coordinate corresponding to time j is: , with the initial position The center of the device is the origin, and the origin is the world coordinate at time j The line regularization of , we get the unit pose vector: , the world coordinate sequence can be expressed as: , the displacement difference between adjacent moments is: , and get n-1 shift difference sequences: , the reference plane normal vector of the displacement from time j to time j+1 is: , the reference plane is: ； Image sequence filtering: The relative path and the corresponding reference plane, as well as the tilt-corrected MATCH_ADJ are used as input. Each time the device generates a displacement , capsule endoscopy collection The corresponding relationship between the number of elements in the image sequence IMG and the number of elements in the device displacement difference sequence DP is: , the jth displacement Collected from The image data should be along the The direction of ,vertical And it is The negative direction is recorded as , the displacement vector The calculation formula is: , taking the displacement difference as output, calculate the previous The sum of the X and Y axis displacements of the image: ,in Indicates before MATCH_ADJ Coordinate displacement difference within the match element, correction angle The following formula can be used for calculation: , the relative displacement difference of the device at time j after correction is: , for time j The displacement difference of the adjacent moments of the image data, that is, the displacement difference of the adjacent moments of the image data in MATCH_ADJ The displacement difference recorded in: ,in , the deviation of each displacement The following formula can be used for calculation: Given a threshold greater than 0.5, , any Corresponding image Put it into IMG_LIST, the list IMG_LIST is the image sequence finally required.

Images