Artificial intelligence capsule endoscopy system alimentary canal image splicing method and device, storage medium and equipment

Abstract

The invention discloses an artificial intelligence capsule endoscopy system alimentary canal image splicing method and device, a storage medium and equipment. The method comprises the following steps: respectively cutting each original alimentary canal image shot by an artificial intelligence capsule endoscopy system to obtain a circular alimentary canal image, and forming a circular alimentary canal image set; performing rotation correction processing on images with depth-of-field areas in the circular alimentary canal image set to obtain a corrected alimentary canal image set, the images being in the same shooting view angle after the rotation correction processing; performing linear polar coordinate transformation on each image in the corrected alimentary canal image set and expanding the images into rectangular alimentary canal images to form a rectangular alimentary canal image set; and calculating the axial displacement between every two adjacent rectangular digestive tract images, and splicing the rectangular digestive tract images in sequence based on a weighted splicing fusion algorithm to form a panoramic image of the digestive tract. According to the method, image visual angle deviation caused by lens inclination is reduced, and smooth transition between images is realized.

Description

Technical Field

[0001] This invention belongs to the field of medical device imaging technology, specifically, it relates to a method, apparatus, computer-readable storage medium, and computer equipment for stitching images of the digestive tract in an artificial intelligence capsule endoscopy system. Background Technology

[0002] The AI-powered capsule endoscopy system is an intelligent capsule device integrating a miniature camera. Its size is comparable to a conventional pharmaceutical capsule, and it features a smooth outer shell for easy swallowing. After swallowing, the capsule moves naturally through the digestive tract with peristalsis, continuously capturing images of the digestive tract's interior. This image data is then wirelessly transmitted to a data recording device worn by the patient. The entire examination process has minimal impact on the patient's daily activities, providing a convenient, non-invasive diagnostic approach. The capsule is naturally excreted in the feces after the examination.

[0003] Artificial intelligence capsule endoscopy systems typically remain in the body for approximately eight hours, during which time tens of thousands of images can be captured. However, directly screening and diagnosing lesions from such a massive image dataset can easily lead to visual fatigue for doctors during prolonged, intensive observation, significantly increasing the risk of missed diagnoses and misdiagnoses. To address this issue, researchers in the field have proposed a scheme to stitch together continuously captured images, generating complete panoramic images to assist doctors in comprehensive and efficient lesion detection and analysis.

[0004] However, the movement of AI-powered capsule endoscopy systems within the digestive tract is accompanied by unpredictable directional changes, including tilting and axial rotation of the imaging direction, leading to deviations in viewpoint and displacement between consecutive images. Directly stitching these raw images together may result in a panoramic image that significantly deviates from the actual digestive tract anatomy, hindering accurate identification and diagnosis. Therefore, effectively correcting viewpoint and positional errors caused by capsule movement during image stitching is crucial for improving the accuracy and practicality of capsule endoscopy panoramic imaging technology. Furthermore, due to the spontaneous rotation and lens tilt of AI-powered capsule endoscopy systems during imaging, directly stitching images can result in significant deviations between the generated panoramic intestinal image and the actual anatomical structure, affecting image recognition and lesion observation. Summary of the Invention

[0005] (a) The technical problem to be solved by the present invention

[0006] The technical problem solved by this invention is: how to correct the deviations in the images of the digestive tract caused by the irregular movement of the artificial intelligence capsule endoscopy system, so that the stitched panoramic image can better match the actual contour of the digestive tract.

[0007] (II) The technical solution adopted in this invention

[0008] A method for stitching together digestive tract images from an artificial intelligence capsule endoscopy system, the method comprising: Each original digestive tract image captured by the artificial intelligence capsule endoscopy system is cropped to obtain a circular digestive tract image, which constitutes a circular digestive tract image set. The images with depth of field in the circular digestive tract image set are subjected to rotational correction processing to obtain a corrected digestive tract image set, wherein the images with depth of field are at the same shooting angle after rotational correction processing. Each image in the corrected digestive tract image set is transformed by linear polar coordinates and expanded into a rectangular digestive tract image to form a rectangular digestive tract image set. Calculate the axial displacement between any two adjacent rectangular digestive tract images in time sequence. Based on the axial displacement, the rectangular digestive tract images are sequentially stitched together using a weighted stitching and fusion algorithm to form a panoramic image of the digestive tract.

[0009] Preferably, each original digestive tract image captured by the artificial intelligence capsule endoscopy system is cropped to obtain a circular digestive tract image, forming a circular digestive tract image set, including: Masking was applied to each original digestive tract image to preserve the contents of the digestive tract in each original image; Iterate through each image after applying the mask to obtain the largest circle without black blank areas corresponding to each image; The radius of each largest circle is determined, and the smallest radius is used as the standard radius to crop each image after applying the mask, forming a set of circular digestive tract images, in which each circular digestive tract image has a common center.

[0010] Preferably, the method for performing rotation correction processing on images with depth regions in the circular digestive tract image set includes: Determine whether each image in the circular digestive tract image set has a depth direction; If so, then select circular digestive tract images that have depth of field, and rotate each selected circular digestive tract image to make each circular digestive tract image have a consistent shooting angle.

[0011] Preferably, the method for rotating the selected circular digestive tract images to ensure that the images have a consistent shooting angle includes: Calculate the rotation angle between the current circular digestive tract image and the preceding circular digestive tract image that is temporally adjacent: Obtain a first set of matching key points on the current circular digestive tract image and a second set of matching key points on the preceding circular digestive tract image, wherein each point in the first set of matching key points matches each point in the second set of matching key points; calculate the rotation angle based on the first set of matching key points and the second set of matching key points. An affine transformation is performed based on the rotation angle to rotate the current circular digestive tract image so that the current circular digestive tract image has the same shooting angle as the previous circular digestive tract image.

[0012] Preferably, the formula for calculating the rotation angle based on the first set of matching key points and the second set of matching key points is as follows:

[0013] , In the formula, For rotation angle, The coordinates of the keypoints in the first set of matching keypoints. The coordinates of the key points in the second set of matching key points. The common center of all circular images of the digestive tract is [the center of the circle]. This represents the number of keypoints in the first set of matching keypoints and the second set of matching keypoints.

[0014] Preferably, based on the axial displacement, the rectangular digestive tract images are sequentially stitched together using a weighted stitching and fusion algorithm to form a panoramic image of the digestive tract, including: The overlapping area between the first and second rectangular digestive tract images is determined based on the axial displacement between the first and second rectangular digestive tract images in the rectangular digestive tract image set. Based on the weighted stitching and fusion algorithm, the image portions of the first rectangular digestive tract image and the second rectangular digestive tract image in the overlapping area are fused and stitched together so that the first rectangular digestive tract image and the second rectangular digestive tract image are stitched together to form the first intermediate stitched image. The overlapping area between the first intermediate stitched image and the third rectangular digestive tract image is determined based on the axial displacement between the second and third rectangular digestive tract images in the rectangular digestive tract image set. Based on the weighted stitching and fusion algorithm, the first intermediate stitched image and the third rectangular digestive tract image are fused and stitched in the overlapping area of ​​the image portion, so that the first intermediate stitched image and the third rectangular digestive tract image are stitched together to form the second intermediate stitched image; The above process is repeated for the remaining images in the rectangular digestive tract image set, so that the last intermediate stitched image is stitched together with the last rectangular digestive tract image to form a panoramic image of the digestive tract.

[0015] Preferably, based on a weighted stitching and fusion algorithm, the image portions of the first rectangular digestive tract image and the second rectangular digestive tract image in the overlapping area are fused and stitched together, including: Define the energy function of the overlapping region of the image. :

[0016] , In the formula, This represents the brightness and color difference gradient of a pixel. This indicates the color or grayscale difference of pixels in the matching region of an image. It is a weighting factor; The minimum energy path is found using dynamic programming.

[0017] , In the formula, This indicates the distance from the left edge of the overlapping region to the current pixel. The minimum path cost; After the path search is completed, the actual seam path from the right to the left of the overlapping area is determined by a backtracking algorithm, and then weighted splicing and fusion are performed:

[0018] ,

[0019] , In the formula, These represent the overlapping areas of the first and second rectangular digestive tract images, respectively. and For the weight function, A fused image representing overlapping regions.

[0020] This application also discloses a stitching device for gastrointestinal images in an artificial intelligence capsule endoscopy system, the stitching device comprising: The preprocessing unit is used to crop each original digestive tract image captured by the artificial intelligence capsule endoscopy system to obtain circular digestive tract images, forming a circular digestive tract image set. A rotation correction unit is used to perform rotation correction processing on images with depth of field regions in the circular digestive tract image set to obtain a corrected digestive tract image set, wherein the images with depth of field regions are at the same shooting angle after rotation correction processing. An image unfolding unit is used to perform linear polar coordinate transformation on each image in the corrected digestive tract image set and unfold it into a rectangular digestive tract image, thus forming a rectangular digestive tract image set. The displacement calculation unit is used to calculate the axial displacement between every two adjacent rectangular digestive tract images in the time sequence. A panoramic image stitching unit is used to stitch together each rectangular digestive tract image sequentially based on a weighted stitching and fusion algorithm according to the axial displacement, so as to form a panoramic image of the digestive tract.

[0021] This application also discloses a computer-readable storage medium storing a program for stitching images of the digestive tract of an artificial intelligence capsule endoscopy system. When the program is executed by a processor, it implements the above-described method for stitching images of the digestive tract of an artificial intelligence capsule endoscopy system.

[0022] This application also discloses a computer device, which includes a computer-readable storage medium, a processor, and a program for stitching images of the digestive tract of an artificial intelligence capsule endoscopy system stored in the computer-readable storage medium. When the program for stitching images of the digestive tract of an artificial intelligence capsule endoscopy system is executed by the processor, it implements the above-described method for stitching images of the digestive tract of an artificial intelligence capsule endoscopy system.

[0023] (III) Beneficial Effects

[0024] The present invention discloses a method and device for stitching images of the digestive tract in an artificial intelligence capsule endoscopy system, which has the following technical advantages compared to other stitching methods: First, the images are corrected to reduce image perspective deviations caused by lens tilt, making the stitched panoramic intestinal image more closely match the actual contours of the intestine. Second, during actual stitching, a weighted stitching and fusion algorithm is used to select and apply the optimal suture path, ensuring smooth transitions between images, reducing artifacts and misalignments in the suture areas, ultimately obtaining high-quality panoramic intestinal images and improving the basis for more accurate lesion detection. Attached Figure Description

[0025] Figure 1 This is a flowchart of a method for stitching digestive tract images using an artificial intelligence capsule endoscopy system according to Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the rotation correction of a circular digestive tract image according to Embodiment 1 of the present invention; Figure 3 This is a schematic diagram of two adjacent rectangular digestive tract images stitched together according to Embodiment 1 of the present invention; Figure 4This is a schematic diagram of the stitching of an intermediate image and a rectangular digestive tract image according to Embodiment 1 of the present invention; Figure 5 This is a schematic diagram of the digestive tract image stitching device of the artificial intelligence capsule endoscopy system according to Embodiment 2 of the present invention; Figure 6 This is a schematic diagram of a computer device according to Embodiment 4 of the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0027] Before describing the various embodiments of this application in detail, the technical concept of this application is briefly described first: In the prior art, due to the irregular movements such as self-rotation and lens tilt during the imaging process of the artificial intelligence capsule endoscopy system, if the captured images are directly stitched together, the resulting panoramic intestinal image will differ significantly from the actual contour of the intestine, affecting subsequent identification and observation. To address this, the image stitching method for the digestive tract of the artificial intelligence capsule endoscopy system provided in this application first performs rotation correction processing on images with depth of field regions to reduce image perspective deviation caused by lens tilt. Then, after image unfolding in polar coordinates, the axial displacement between adjacent images is calculated. Based on a weighted stitching fusion algorithm, the images are stitched together. Through image correction, the image deviation caused by the irregular movements of the artificial intelligence capsule endoscopy system is greatly reduced. The weighted stitching fusion algorithm ensures smooth transitions between images, reduces artifacts and misalignments in the seam areas, and ultimately obtains high-quality panoramic intestinal images, improving the basis for more accurate lesion detection.

[0028] Specifically, such as Figure 1 As shown, the method for stitching digestive tract images using the artificial intelligence capsule endoscopy system in this embodiment includes the following steps: Step S10: Crop each original digestive tract image captured by the artificial intelligence capsule endoscopy system to obtain circular digestive tract images, forming a circular digestive tract image set; Step S20: Perform rotational correction processing on the images with depth of field in the circular digestive tract image set to obtain a corrected digestive tract image set, wherein the images with depth of field are at the same shooting angle after rotational correction processing; Step S30: Perform linear polar coordinate transformation on each image in the corrected digestive tract image set and expand it into a rectangular digestive tract image to form a rectangular digestive tract image set; Step S40: Calculate the axial displacement between any two adjacent rectangular digestive tract images in time sequence. Step S50: Based on the axial displacement, the rectangular digestive tract images are stitched together sequentially using a weighted stitching and fusion algorithm to form a panoramic image of the digestive tract.

[0029] Since the effective captured portion of the original digestive tract images taken by the artificial intelligence capsule endoscopy system is not necessarily a standard circular area, when it is not a circular area, it is necessary to cut out the largest circular part. Otherwise, black blank parts will be mixed in when stitching. Therefore, in step S10, after cropping each original digestive tract image, a circular digestive tract image is obtained, forming a circular digestive tract image set to ensure that the images have uniform boundary characteristics in subsequent processing.

[0030] Specifically, each raw digestive tract image captured by the AI ​​capsule endoscopy system is cropped to obtain circular digestive tract images, forming a circular digestive tract image set. This process includes: First, applying a mask to each raw digestive tract image to retain its contents while filtering out invalid information such as patient information and timestamps. Next, iterating through the masked images to find the largest circle without any black blank areas. Finally, determining the radius of each largest circle, and using the smallest radius as the standard radius to crop the masked images, forms the circular digestive tract image set, where all circular digestive tract images share a common center.

[0031] Furthermore, the digestive tract structure is abstracted into an ideal cylindrical region. When taking a picture from the center of the slice circle forward, the depth-of-field region of the image coincides with the extreme point region. However, during actual imaging within the digestive tract, due to tilting and shaking of the AI ​​capsule endoscopy system, the shooting direction often deviates from the central axis forward, resulting in different shooting angles for each image. Therefore, in step S20, it is determined whether each image in the circular digestive tract image set has a depth-of-field direction. If so, images with a depth-of-field direction are selected, and each selected circular digestive tract image is rotated to ensure a consistent shooting angle. When an image does not have a depth-of-field direction, it indicates that the image was taken by the AI ​​capsule endoscopy system at a turning point; therefore, no rotation is performed, and it is directly included in the corrected digestive tract image set.

[0032] Specifically, the method of rotating each selected circular digestive tract image to make the circular digestive tract images have a consistent shooting angle includes: calculating the rotation angle between the current circular digestive tract image and the previous circular digestive tract image that is temporally adjacent; performing an affine transformation based on the rotation angle to rotate the current circular digestive tract image so that the shooting angle of the current circular digestive tract image is consistent with that of the previous circular digestive tract image.

[0033] The method for calculating the rotation angle includes: obtaining the first set of matching key points on the current circular digestive tract image. The second set of matching key points on the previous circular digestive tract image In this process, each point in the first set of matching keypoints is matched one-to-one with each point in the second set of matching keypoints. Next, the rotation angle is calculated based on the first and second sets of matching keypoints, using the following formula:

[0034] , In the formula, For rotation angle, The coordinates of the keypoints in the first set of matching keypoints. The coordinates of the key points in the second set of matching key points. The common center of all circular images of the digestive tract is [the center of the circle]. This represents the number of keypoints in the first and second matching keypoint sets. The rotation angle between the current circular digestive tract image and the previous circular digestive tract image is as follows: Figure 2 As shown.

[0035] Furthermore, a linear polar coordinate transformation is performed on the corrected digestive tract image set to convert it from a circular image to a rectangular image. Specifically, this process first crops the corrected image along a predetermined angle to form the edges of a rectangular digestive tract image. After the transformation, the cropped edges are mapped to the top and bottom edges of the rectangular digestive tract image, the center point of the corrected image corresponds to the midpoint of the left edge of the rectangular digestive tract image, and the circumference of the image is mapped to the right edge of the rectangular digestive tract image. During the processing, the cropping at the predetermined angle helps maintain the consistency and integrity of the image, ensuring that the edge correspondences are correct after the image is unfolded. By performing this transformation step on each corrected digestive tract image, a set of rectangular digestive tract images is generated for subsequent analysis and stitching.

[0036] For example, the calculation process for the axial displacement between any two adjacent rectangular digestive tract images in time sequence is as follows: After affine transformation and linear polar coordinate transformation, the first set of matching key points on the current circular digestive tract image is obtained. The second set of matching key points on the previous circular digestive tract image Mapped to the current rectangular digestive tract image and the previous rectangular digestive tract image respectively, along the length of the rectangular digestive tract image, the first set of matching key points and its corresponding second set of matching key points are traversed pairwise, and their abscissa differences are calculated. Specifically, for each pair of matching key points, the difference in abscissa between the corresponding key points in the current circular digestive tract image and the previous image is calculated, and the maximum value of all differences is taken as the axial displacement. This length direction is perpendicular to the width direction of the image, ensuring that the width direction is parallel to the motion direction of the AI ​​capsule endoscopy system, so that the calculation of the axial displacement accurately reflects the relative displacement of the image in the motion direction.

[0037] In one or more embodiments, based on the axial displacement, the rectangular digestive tract images are sequentially stitched together using a weighted stitching and fusion algorithm to form a panoramic image of the digestive tract, including: The overlapping area between the first and second rectangular digestive tract images is determined based on the axial displacement between them in the rectangular digestive tract image set. Figure 3 As shown.

[0038] Based on a weighted stitching and fusion algorithm, the overlapping areas of the first rectangular digestive tract image P1 and the second rectangular digestive tract image P2 are fused and stitched together to form the first intermediate stitched image M1. Figure 3 As shown.

[0039] The overlapping area between the first intermediate stitched image M1 and the third rectangular digestive tract image P3 is determined based on the axial displacement between the second rectangular digestive tract image P2 and the third rectangular digestive tract image P3 in the rectangular digestive tract image set. Figure 4 As shown.

[0040] Based on a weighted stitching and fusion algorithm, the first intermediate stitched image M1 and the third rectangular digestive tract image P3 are fused and stitched together in the overlapping area to form the second intermediate stitched image M2. Figure 4 As shown.

[0041] The above process is repeated for the remaining images in the rectangular digestive tract image set, so that the last intermediate stitched image is stitched together with the last rectangular digestive tract image to form a panoramic image of the digestive tract.

[0042] For example, during the stitching process, a weighted fusion stitching algorithm is employed, rotating the optimal seam path using the minimum energy path. This algorithm analyzes the gradient changes and pixel similarity of the images to find the path with the least visual difference in the overlapping area, thereby ensuring a smooth transition in the seam area and an artifact-free stitching effect. Specifically, based on the weighted stitching fusion algorithm, the image portions of the first rectangular digestive tract image and the second rectangular digestive tract image in the overlapping area are fused and stitched, including: First, define the energy function for the overlapping image region. :

[0043] , In the formula, the energy function Used to measure the energy of the seam path at each pixel location. This represents the brightness and color difference gradient of a pixel, indicating the pixel's representation along the gradient direction. This indicates the color or grayscale difference of pixels in the matching region of an image. It is a weighting factor used to adjust the contribution of gradient and similarity to the cost function.

[0044] The minimum energy path is found using dynamic programming.

[0045] , In the formula, This is the path cost accumulation function, representing the distance from the left edge of the overlapping region to the current pixel. The path with the minimum cost can be found on the right side of the overlapping region by traversing the entire overlapping region matrix from left to right. After the path search is completed, the actual seam path from the right to the left of the overlapping area is determined by a backtracking algorithm, and then weighted splicing and fusion are performed:

[0046] ,

[0047] , In the formula, These represent the overlapping areas of the first and second rectangular digestive tract images, respectively. and For the weight function, A fused image representing overlapping regions.

[0048] Compared to traditional forward-distance stitching methods, the weighted stitching fusion algorithm proposed in this application achieves better stitching results. Traditional forward-distance stitching methods typically rely on simple matching of adjacent regions in the image, stitching based on a set step size. While this method can achieve basic image stitching, noticeable seam marks or visual discontinuities often appear at the seams, especially at the edges and details, where distortion, overlap, or color differences can easily occur, leading to unsatisfactory stitching results. This is particularly true in stitching small intestine images, where the varying angles and depth of field amplify the shortcomings of traditional stitching methods. In contrast, the weighted stitching fusion algorithm optimizes the image stitching process with a more refined algorithm, dynamically adjusting the position, angle, and overlapping areas of the seams to minimize visual differences at the stitching points. This method not only handles complex textures, details, and lighting variations in images but also intelligently identifies key feature regions, ensuring smooth transitions at the seams while maintaining overall image consistency and realism. By precisely adjusting the seams, distortion and artifacts commonly found in traditional methods can be effectively avoided, greatly improving the splicing effect.

[0049] The image stitching method for the digestive tract using an AI capsule endoscopy system disclosed in Embodiment 1 significantly reduces image deviations caused by irregular movements (such as rotation and tilt) of the AI ​​capsule endoscopy system during imaging, resulting in a more accurate and comprehensive panoramic image that better reflects the actual structural features of the intestine. This stitching method greatly improves image accuracy and diagnostic effectiveness, providing reliable technical support for subsequent analysis and lesion detection by physicians.

[0050] like Figure 5As shown in Embodiment 2 of this application, a stitching device for digestive tract images of an artificial intelligence capsule endoscopy system is also disclosed. The stitching device includes a preprocessing unit 100, a rotation correction unit 200, an image unfolding unit 300, a displacement calculation unit 400, and a panoramic image stitching unit 500. The preprocessing unit 100 is used to crop each original digestive tract image captured by the artificial intelligence capsule endoscopy system to obtain circular digestive tract images, forming a circular digestive tract image set; the rotation correction unit 200 is used to perform rotation correction processing on the images with depth of field in the circular digestive tract image set to obtain a corrected digestive tract image set, wherein the images with depth of field are at the same shooting angle after rotation correction processing; the image unfolding unit 300 is used to perform linear polar coordinate transformation on each image in the corrected digestive tract image set and unfold it into a rectangular digestive tract image, forming a rectangular digestive tract image set; the displacement calculation unit 400 is used to calculate the axial displacement between every two adjacent rectangular digestive tract images in the rectangular digestive tract image set in time; the panoramic image stitching unit 500 is used to stitch the rectangular digestive tract images sequentially according to the axial displacement based on a weighted stitching fusion algorithm to form a panoramic digestive tract image. The detailed processing procedures of the preprocessing unit 100, rotation correction unit 200, image unfolding unit 300, displacement calculation unit 400 and panoramic image stitching unit 500 are described in accordance with the description in Embodiment 1, and will not be repeated here.

[0051] Embodiment 3 of this application also discloses a computer-readable storage medium storing a stitching program for digestive tract images of an artificial intelligence capsule endoscopy system. When the stitching program for digestive tract images of an artificial intelligence capsule endoscopy system is executed by a processor, the above-described stitching method for digestive tract images of an artificial intelligence capsule endoscopy system is implemented.

[0052] This third embodiment also discloses a computer device, at the hardware level, such as... Figure 6 As shown, the computer device includes a processor 12, an internal bus 13, a network interface 14, and a computer-readable storage medium 11. The processor 12 reads the corresponding computer program from the computer-readable storage medium and runs it, forming a request processing device at the logical level. Of course, in addition to the software implementation, one or more embodiments of this specification do not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices. The computer-readable storage medium 11 stores a stitching program for the digestive tract images of the artificial intelligence capsule endoscopy system. When the processor executes the stitching program for the digestive tract images of the artificial intelligence capsule endoscopy system, it implements the above-described stitching method for the digestive tract images of the artificial intelligence capsule endoscopy system.

[0053] Computer-readable storage media include both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer-readable storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0054] The specific embodiments of the present invention have been described in detail above. Although some embodiments have been shown and described, those skilled in the art should understand that modifications and improvements can be made to these embodiments without departing from the principles and spirit of the present invention as defined by the claims and their equivalents, and such modifications and improvements should also be within the protection scope of the present invention.

Claims

1. A method for stitching together digestive tract images in an artificial intelligence capsule endoscopy system, characterized in that, The splicing method includes: Each original digestive tract image captured by the artificial intelligence capsule endoscopy system is cropped to obtain a circular digestive tract image, which constitutes a circular digestive tract image set. The images with depth of field in the circular digestive tract image set are subjected to rotational correction processing to obtain a corrected digestive tract image set, wherein the images with depth of field are at the same shooting angle after rotational correction processing. Each image in the corrected digestive tract image set is transformed by linear polar coordinates and expanded into a rectangular digestive tract image to form a rectangular digestive tract image set. Calculate the axial displacement between any two adjacent rectangular digestive tract images in time sequence. Based on the axial displacement, the rectangular digestive tract images are sequentially stitched together using a weighted stitching and fusion algorithm to form a panoramic image of the digestive tract.

2. The method for stitching digestive tract images in an artificial intelligence capsule endoscopy system according to claim 1, characterized in that, Each raw digestive tract image captured by the artificial intelligence capsule endoscopy system is cropped to obtain circular digestive tract images, forming a circular digestive tract image set, including: Masking was applied to each original digestive tract image to preserve the contents of the digestive tract in each original image; Iterate through each image after applying the mask to obtain the largest circle without black blank areas corresponding to each image; The radius of each largest circle is determined, and the smallest radius is used as the standard radius to crop each image after applying the mask, forming a set of circular digestive tract images, in which each circular digestive tract image has a common center.

3. The method for stitching digestive tract images in an artificial intelligence capsule endoscopy system according to claim 1, characterized in that, The method for performing rotation correction processing on images with depth-of-field regions in the circular digestive tract image set includes: Determine whether each image in the circular digestive tract image set has a depth direction; If so, then select circular digestive tract images that have depth of field, and rotate each selected circular digestive tract image to make each circular digestive tract image have a consistent shooting angle.

4. The method for stitching digestive tract images in an artificial intelligence capsule endoscopy system according to claim 3, characterized in that, The method of rotating the selected circular images of the digestive tract to ensure that the images have a consistent shooting perspective includes: Calculate the rotation angle between the current circular digestive tract image and the preceding circular digestive tract image that is temporally adjacent: Obtain a first set of matching key points on the current circular digestive tract image and a second set of matching key points on the preceding circular digestive tract image, wherein each point in the first set of matching key points matches each point in the second set of matching key points; calculate the rotation angle based on the first set of matching key points and the second set of matching key points. An affine transformation is performed based on the rotation angle to rotate the current circular digestive tract image so that the current circular digestive tract image has the same shooting angle as the previous circular digestive tract image.

5. The method for stitching digestive tract images in an artificial intelligence capsule endoscopy system according to claim 4, characterized in that, The formula for calculating the rotation angle based on the first set of matching key points and the second set of matching key points is as follows: ， In the formula, α is the rotation angle. The coordinates of the key points in the first set of matching key points. The coordinates of the key points in the second set of matching key points. Let m be the common center of all circular digestive tract images, and m be the number of keypoints in the first and second matching keypoint sets.

6. The method for stitching digestive tract images in an artificial intelligence capsule endoscopy system according to claim 1, characterized in that, Based on the axial displacement, the rectangular digestive tract images are sequentially stitched together using a weighted stitching and fusion algorithm to form a panoramic image of the digestive tract, including: The overlapping area between the first and second rectangular digestive tract images is determined based on the axial displacement between the first and second rectangular digestive tract images in the rectangular digestive tract image set. Based on the weighted stitching and fusion algorithm, the image portions of the first rectangular digestive tract image and the second rectangular digestive tract image in the overlapping area are fused and stitched together so that the first rectangular digestive tract image and the second rectangular digestive tract image are stitched together to form the first intermediate stitched image. The overlapping area between the first intermediate stitched image and the third rectangular digestive tract image is determined based on the axial displacement between the second and third rectangular digestive tract images in the rectangular digestive tract image set. Based on the weighted stitching and fusion algorithm, the first intermediate stitched image and the third rectangular digestive tract image are fused and stitched in the overlapping area of ​​the image portion, so that the first intermediate stitched image and the third rectangular digestive tract image are stitched together to form the second intermediate stitched image; The above process is repeated for the remaining images in the rectangular digestive tract image set, so that the last intermediate stitched image is stitched together with the last rectangular digestive tract image to form a panoramic image of the digestive tract.

7. The method for stitching digestive tract images in an artificial intelligence capsule endoscopy system according to claim 6, characterized in that, Based on a weighted stitching and fusion algorithm, the overlapping areas of the first and second rectangular digestive tract images are fused and stitched together, including: Define the energy function of the overlapping region of the image. : ， In the formula, This represents the brightness and color difference gradient of a pixel. This indicates the color or grayscale difference of pixels in the matching region of an image. It is a weighting factor; The minimum energy path is found using dynamic programming. ， In the formula, This indicates the distance from the left edge of the overlapping region to the current pixel. The minimum path cost; After the path search is completed, the actual seam path from the right to the left of the overlapping area is determined by a backtracking algorithm, and then weighted splicing and fusion are performed: ， ， In the formula, These represent the overlapping areas of the first and second rectangular digestive tract images, respectively. and For the weight function, A fused image representing overlapping regions.

8. A device for stitching together digestive tract images in an artificial intelligence capsule endoscopy system, characterized in that, The splicing device includes: The preprocessing unit is used to crop each original digestive tract image captured by the artificial intelligence capsule endoscopy system to obtain circular digestive tract images, forming a circular digestive tract image set. A rotation correction unit is used to perform rotation correction processing on images with depth of field regions in the circular digestive tract image set to obtain a corrected digestive tract image set, wherein the images with depth of field regions are at the same shooting angle after rotation correction processing. An image unfolding unit is used to perform linear polar coordinate transformation on each image in the corrected digestive tract image set and unfold it into a rectangular digestive tract image, thus forming a rectangular digestive tract image set. The displacement calculation unit is used to calculate the axial displacement between every two adjacent rectangular digestive tract images in the time sequence. A panoramic image stitching unit is used to stitch together each rectangular digestive tract image sequentially based on a weighted stitching and fusion algorithm according to the axial displacement, so as to form a panoramic image of the digestive tract.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a stitching program for digestive tract images of an artificial intelligence capsule endoscopy system. When the stitching program for digestive tract images of an artificial intelligence capsule endoscopy system is executed by a processor, it implements the stitching method for digestive tract images of an artificial intelligence capsule endoscopy system according to any one of claims 1 to 7.

10. A computer device, characterized in that, The computer device includes a computer-readable storage medium, a processor, and a program for stitching images of the digestive tract of an artificial intelligence capsule endoscopy system stored in the computer-readable storage medium. When the program for stitching images of the digestive tract of an artificial intelligence capsule endoscopy system is executed by the processor, it implements the method for stitching images of the digestive tract of an artificial intelligence capsule endoscopy system according to any one of claims 1 to 7.

Images