Digestive cavity inner wall three-dimensional model reconstruction method and apparatus, device, and medium

Abstract

The present disclosure provides a digestive cavity inner wall three-dimensional model reconstruction method and apparatus, a device, and a medium. The method comprises: acquiring an image set collected by a capsule endoscope inside a digestive cavity; determining, in the image set, a plurality of first image frames formed under the influence of digestive cavity wall peristalsis, and determining a folded area in each first image frame; restoring the folded area in each first image frame into a smooth area to obtain a plurality of processed first image frames; and obtaining a digestive cavity three-dimensional model on the basis of the plurality of processed first image frames. The folded areas in the plurality of first image frames, influenced by digestive cavity wall peristalsis, in the image set are restored into the smooth areas, so that more stable images are obtained; and three-dimensional model construction is performed on the basis of these stable images, facilitating the construction of more accurate digestive cavity models, helping doctors to determine the lesions of patients more accurately, and improving the medical treatment efficiency and satisfaction of the patients.

Description

Method, device, equipment and medium for reconstructing three-dimensional model of inner wall of digestive cavity

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] The present disclosure claims priority to the Chinese patent application No. 2024104197099, filed on April 9, 2024, and entitled "Method, device, equipment and medium for reconstructing three-dimensional model of inner wall of digestive cavity", the entire content of which is incorporated herein by reference. TECHNICAL FIELD

[0003] The present disclosure relates to the technical field of three-dimensional model construction, in particular to a method, device, computer equipment, storage medium and computer program product for reconstructing three-dimensional model of inner wall of digestive cavity BACKGROUND

[0004] With the gradual maturity of sensor technology, sensors are gradually applied to more aspects. The role of sensor technology in the construction of three-dimensional models cannot be underestimated. Currently, common three-dimensional reconstruction technologies mainly include three-dimensional reconstruction based on binocular cameras and three-dimensional reconstruction with depth sensors. The capsule endoscope is equipped with a three-dimensional model construction configuration for the digestive cavity. Due to the volume of the capsule endoscope itself, it is difficult to install a camera on it. In addition, due to the volume limitation of the depth sensor, common TOF (time of flight), structured light, radar and other depth sensors cannot be applied to the capsule endoscope. In addition, due to the peristaltic state of the digestive cavity at all times, the traditional SLAM (Simultaneous Localization and Mapping) technology cannot obtain enough stable areas when constructing a map, which leads to the failure of SLAM technology to construct a map, resulting in an inaccurate three-dimensional image. SUMMARY

[0005] Therefore, the present disclosure provides a method, device, equipment and medium for reconstructing three-dimensional model of inner wall of digestive cavity, which can provide a stable image and construct a three-dimensional image.

[0006] The present disclosure provides a method for reconstructing three-dimensional model of inner wall of digestive cavity, comprising:

[0007] obtaining an image set collected by a capsule endoscope inside a digestive cavity;

[0008] determining a plurality of first images formed under the influence of peristalsis of the wall of the digestive cavity in the image set, and determining a wrinkled area in each first image;

[0009] restoring the wrinkled area in each first image to a flat area to obtain a plurality of processed first images;

[0010] obtaining a three-dimensional model of the inner wall of the digestive cavity based on the plurality of processed first images.

[0011] In one embodiment, the plurality of images to be processed formed under the influence of peristalsis of the wall of the digestive cavity are determined from the image set, and the wrinkled regions in each of the images to be processed are determined, comprising:

[0012] inputting the images in the image set into a pre-constructed wrinkled region identification model;

[0013] obtaining the plurality of first images formed under the influence of peristalsis of the wall of the digestive cavity and the wrinkled regions in each of the first images according to the prediction results output by the wrinkled region identification model.

[0014] Optionally, the wrinkled region identification model comprises a neural network trained and configured to extract features from input images and output feature maps, a region proposal network configured to generate regions of interest of the images, and a convolutional neural network based on mask regions.

[0015] Optionally, inputting the images in the image set into the pre-constructed wrinkled region identification model comprises:

[0016] obtaining the acceleration of the capsule endoscope when each image is collected;

[0017] filtering a candidate set from the image set according to whether the acceleration is greater than a threshold value;

[0018] performing motion blur detection on each image in the candidate set to obtain the motion blur conditions of each image in the candidate set;

[0019] filtering a target set from the candidate set according to whether the motion blur conditions meet a preset condition;

[0020] inputting each image in the target set into the pre-constructed wrinkled region identification model.

[0021] Optionally, each image in the target set is an image blurred due to motion.

[0022] Optionally, restoring the wrinkled regions in each of the first images to flat regions to obtain the plurality of processed first images comprises:

[0023] obtaining the motion conditions between the plurality of first images;

[0024] restoring the wrinkled regions in each of the first images to flat regions based on the motion conditions to obtain the plurality of processed first images.

[0025] Optionally, the motion conditions comprise optical flow between adjacent first images, and obtaining the motion conditions between the plurality of first images comprises:

[0026] performing corner point detection on each first image to obtain corner points of each first image;

[0027] calculating optical flow between adjacent first images according to the corner points of each first image.

[0028] Optionally, based on the motion condition, the wrinkled region in each first image is restored to a flat region to obtain a plurality of processed first images, including:

[0029] performing motion smoothing on the optical flow between adjacent first images to obtain smoothed optical flow;

[0030] performing image inpainting on the plurality of first images according to the smoothed optical flow between adjacent first images to obtain a plurality of processed first images.

[0031] Optionally, based on the plurality of processed first images, a three-dimensional model of the inner wall of the digestive cavity is obtained, including:

[0032] inputting the plurality of processed first images into a pre-constructed three-dimensional reconstruction algorithm;

[0033] obtaining the three-dimensional model of the inner wall of the digestive cavity according to an output result of the three-dimensional reconstruction algorithm.

[0034] Optionally,

[0035] inputting the plurality of processed first images into a pre-constructed three-dimensional reconstruction algorithm, including:

[0036] inputting the plurality of processed first images and a second image other than the first image in the image set into the three-dimensional reconstruction algorithm;

[0037] the output result of the three-dimensional reconstruction algorithm includes an initial three-dimensional model of the inner wall of the digestive cavity generated according to the plurality of processed first images and the second image other than the first image in the image set, and the three-dimensional model of the inner wall of the digestive cavity is obtained according to the output result of the three-dimensional reconstruction algorithm, including:

[0038] if the accuracy of the initial three-dimensional model of the inner wall of the digestive cavity does not reach an accuracy threshold, obtaining an acceleration corresponding to the capsule endoscope when each second image is collected;

[0039] determining a to-be-restored second image in each second image according to the relative size of the acceleration;

[0040] performing flat restoration processing on the to-be-restored second image to obtain a restored second image;

[0041] obtaining the three-dimensional model of the inner wall of the digestive cavity based on the restored second image, the plurality of processed first images, and other second images other than the restored second image. Optionally, the flat restoration processing on the to-be-restored second image to obtain the restored second image includes:

[0042] The second image to be recovered is processed by a filtering algorithm and a smoothing algorithm respectively to obtain a corresponding recovered second image.

[0043] The disclosure also provides a device for reconstructing a three-dimensional model of a digestive cavity wall, comprising:

[0044] An image set acquisition module configured to acquire an image set captured by a capsule endoscope inside a digestive cavity;

[0045] A wrinkled region determination module configured to determine a plurality of first images formed under the influence of peristalsis of the digestive cavity wall in the image set, and determine a wrinkled region in each first image;

[0046] An image processing module configured to recover the wrinkled region in each first image into a flat region to obtain a plurality of processed first images;

[0047] A three-dimensional model determination module configured to recover the wrinkled region in each first image into a flat region to obtain a plurality of processed first images.

[0048] The disclosure also provides a computer device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the following steps when executing the computer program:

[0049] Acquiring an image set captured by a capsule endoscope inside a digestive cavity;

[0050] Determining a plurality of first images formed under the influence of peristalsis of the digestive cavity wall in the image set, and determining a wrinkled region in each first image;

[0051] Recovering the wrinkled region in each first image into a flat region to obtain a plurality of processed first images;

[0052] Obtaining a three-dimensional model of the inner wall of the digestive cavity based on the plurality of processed first images.

[0053] The disclosure also provides a computer-readable storage medium having a computer program stored thereon, and the computer program is executed by a processor to implement the following steps:

[0054] Acquiring an image set captured by a capsule endoscope inside a digestive cavity;

[0055] Determining a plurality of first images formed under the influence of peristalsis of the digestive cavity wall in the image set, and determining a wrinkled region in each first image;

[0056] Recovering the wrinkled region in each first image into a flat region to obtain a plurality of processed first images;

[0057] Based on the plurality of processed first images, a three-dimensional model of the inner wall of the digestive cavity is obtained.

[0058] In a fifth aspect, the present disclosure also provides a computer program product comprising a computer program which, when executed by a processor, implements the following steps:

[0059] An image set collected by the capsule endoscope inside the digestive cavity is obtained.

[0060] A plurality of first images affected by the peristalsis of the digestive cavity wall are determined in the image set, and a wrinkled region in each first image is determined.

[0061] The wrinkled region in each first image is restored to a flat region to obtain a plurality of processed first images.

[0062] Based on the plurality of processed first images, a three-dimensional model of the inner wall of the digestive cavity is obtained.

[0063] The reconstruction method, device, equipment and medium of the three-dimensional model of the inner wall of the digestive cavity provided by the present disclosure, by obtaining an image set collected by the capsule endoscope inside the digestive cavity, determining a plurality of first images affected by the peristalsis of the digestive cavity wall in the image set, and determining a wrinkled region in each first image, restoring the wrinkled region in each first image to a flat region to obtain a plurality of processed first images, and obtaining a three-dimensional model of the digestive cavity based on the plurality of processed first images. By restoring the wrinkled region in the plurality of first images affected by the peristalsis of the digestive cavity wall in the image set to a flat region, more stable images are obtained, and the three-dimensional model is built according to these stable images, which is conducive to constructing a more accurate digestive cavity model, helping doctors to more accurately judge the patient's lesion, and improving the efficiency and satisfaction of the patient. BRIEF DESCRIPTION OF DRAWINGS

[0064] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the related art, the drawings needed to be used in the embodiments or related art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative labor.

[0065] FIG. 1 is an application environment diagram of the reconstruction method of the three-dimensional model of the inner wall of the digestive cavity in an embodiment;

[0066] FIG. 2 is a flow diagram of the reconstruction method of the three-dimensional model of the inner wall of the digestive cavity in an embodiment;

[0067] FIG. 3 is a model diagram for determining the wrinkled region in each first image in an embodiment;

[0068] Fig. 4 is a flow diagram of a method of constructing a three-dimensional model of the inner wall of a digestive cavity according to another embodiment;

[0069] Fig. 5 is a diagram of the process of constructing a three-dimensional model of a digestive cavity according to an embodiment;

[0070] Fig. 6 is a block diagram of a reconstruction device for a three-dimensional model of the inner wall of a digestive cavity according to an embodiment;

[0071] Fig. 7 is a diagram of the internal structure of a computer device according to an embodiment. DETAILED DESCRIPTION

[0072] To make the purposes, technical solutions and advantages of the present disclosure clearer, the present disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present disclosure and not to limit the present disclosure.

[0073] The reconstruction method of a three-dimensional model of the inner wall of a digestive cavity provided by the embodiments of the present disclosure can be applied in the application environment shown in Fig. 1. In the application environment, a capsule endoscope 102 communicates with a server 104 through a network, and there can be a third-party data processing device between the capsule endoscope 102 and the server 104 for data transfer or processing. A data storage system can store data required to be processed by the server 104. The data storage system can be integrated on the server 104, or placed on a cloud or other network server. The server 104 acquires an image set collected by the capsule endoscope 102 inside a digestive cavity of a human body, determines a plurality of first images formed under the influence of peristalsis of the wall of the digestive cavity from the image set, determines a wrinkled area in each of the first images, restores the wrinkled area in each of the first images to a flat area, obtains a plurality of processed first images, and obtains a corresponding three-dimensional model of the inner wall of the digestive cavity according to the plurality of processed first images. The server 104 can be implemented by an independent server or a server cluster composed of multiple servers.

[0074] In an exemplary embodiment, as shown in Fig. 2, a reconstruction method of a three-dimensional model of the inner wall of a digestive cavity is provided. The method is described below by taking the server 104 in Fig. 1 as an example, and includes the following steps S201 to S204.

[0075] In the method, the steps S201 to S204 are as follows.

[0076] In step S201, an image set collected by a capsule endoscope inside a digestive cavity is acquired.

[0077] In the embodiment, the capsule endoscope is a medical device configured to examine internal organs and tissues of a human body. The capsule endoscope is ingested into the human body and captures the inside of the digestive cavity through the camera on the endoscope. The captured photos clearly record the operation of the digestive cavity. The digestive cavity can be understood as the part of the human body that constitutes the digestive system, including the esophagus, stomach, small intestine, large intestine, etc.

[0078] Optionally, the server 104 obtains the photos of the inside of the digestive cavity captured by the capsule endoscope through the network.

[0079] Step S202, determine a plurality of first images formed under the influence of the peristalsis of the wall of the digestive cavity in the image set, and determine the wrinkled area in each first image.

[0080] It can be understood that the first image is an image affected by peristalsis.

[0081] Optionally, since the human body is a dynamic process of digestion, that is, the digestive organs are in a state of continuous peristalsis under normal physiological conditions, the endoscope may be affected by the peristalsis of the wall of the digestive cavity when capturing images. Therefore, it is necessary to determine a plurality of first images affected by peristalsis in the image set that has been captured, and to determine the unstable wrinkled area in each first image affected by peristalsis waves. At the same time, the obtained image set also contains a plurality of images not affected by the peristalsis of the wall of the digestive cavity. These unaffected images need to be retained for subsequent corresponding model building.

[0082] Step S203, restore the wrinkled area in each first image to a flat area to obtain a plurality of processed first images.

[0083] It can be understood that the processed first image refers to the image after restoration to flatness.

[0084] Optionally, if a corresponding three-dimensional model is directly constructed according to the first image seriously affected by the peristalsis of the wall of the digestive cavity, the three-dimensional model constructed will be inaccurate, thereby affecting the accuracy of lesion diagnosis. Therefore, it is necessary to restore the wrinkled area in each first image to a flat area, and then obtain a plurality of processed first images.

[0085] Step S204, obtain a three-dimensional model of the inner wall of the digestive cavity based on the plurality of processed first images.

[0086] Optionally, the three-dimensional model of the inner wall of the digestive cavity of the current user can be constructed based on the plurality of processed first images and the original images in the previous image set that are not affected by the peristalsis waves of the wall of the digestive cavity.

[0087] In the reconstruction method of the three-dimensional model of the inner wall of the digestive cavity, the image set collected inside the digestive cavity by the endoscope is obtained, a plurality of first images formed under the influence of peristalsis of the wall of the digestive cavity are determined in the image set, the corrugated area in each first image is determined, the corrugated area in each first image is restored to a flat area, a plurality of processed first images are obtained, and a three-dimensional model of the digestive cavity is obtained based on the plurality of processed first images. By restoring the corrugated area in the plurality of first images in the image set affected by the peristalsis of the wall of the digestive cavity to a flat area, more stable images are obtained, and the three-dimensional model is built according to these stable images, which is beneficial to construct a more intuitive digestive cavity model, helps doctors to more accurately judge the patient's lesion, and improves the efficiency and satisfaction of the patient.

[0088] Optionally, the step S202, that is, determining a plurality of first images formed under the influence of peristalsis of the wall of the digestive cavity in the image set and determining the corrugated area in each first image, comprises: inputting the images in the image set into a pre-constructed corrugated area identification model; and obtaining the plurality of first images formed under the influence of peristalsis of the wall of the digestive cavity and the corrugated area in each first image according to the prediction result output by the corrugated area identification model.

[0089] Optionally, the pre-constructed corrugated area identification model can include a neural network trained to extract features from input images and output feature maps, a region proposal network (RPN) configured to generate regions of interest of images, and a mask region-based convolutional neural network (Mask R-CNN).

[0090] Optionally, the neural network outputting the feature map can be a recursive feature pyramid network (RFP), but the neural network can also be other networks, and the present disclosure does not make too many limitations on this.

[0091] In one example, the specific determination of the corrugated area in each first image by the corrugated area identification model is shown in FIG. 3:

[0092] First, the image set is preprocessed, and the preprocessing operation includes image normalization, image smoothing, image enhancement and the like. The image set is a color image collected by a monocular camera of an endoscope.

[0093] Then, the image is input into a pre-trained neural network (a recursive feature pyramid (RFP) network) to obtain a feature map corresponding to the image; the recursive feature pyramid network RFP is mainly configured to extract features for the input image and output a feature map. As shown in FIG. 3, the recursive feature pyramid network RFP used in the present disclosure includes bottom-up backbone layers (C2, C3, C4, and C5, respectively) and top-down FPN (Feature Pyramid Network) layers (P5, P4, P3, and P2, respectively, in FIG. 3); it is worth mentioning that the recursive feature pyramid network RFP combines additional feedback connections (indicated by dashed arrows in FIG. 3) from the FPN layers to the backbone layers, so that the features of the input image are repeatedly purified, the expression capability of the FPN layers is enriched, and the extracted features are more suitable for subsequent detection, especially for accurately identifying small targets in a complex background. The recursive function is to enable the error feedback information of target detection to more directly adjust the parameters of the backbone layer.

[0094] Next, the feature map output by the recursive feature pyramid network is input into an RPN (Region Proposal Network) network for binary classification (configured to distinguish between foreground and background) and BB (bounding box) regression) to output two types of feature maps, namely a target shape feature map and a target position feature map, and each target position feature map includes a predetermined number of candidate boxes, as shown by the “region of interest” (ROI, Region Of Interest) in FIG. 3; a portion of the ROIs are filtered out by a non-maximum suppression method.

[0095] Next, the remaining ROIs are subjected to an ROI Align operation. Since the feature map output by the region proposal network is smaller than the feature map input into the region proposal network, the boundary boxes on the output feature map are shifted in position when mapped onto the input feature map, which will result in inaccurate subsequent target detection, so the ROI Align operation is needed, that is, the pixels of the feature map of the input region proposal network (RPN) are first matched with the pixels of the feature map output by the recursive feature pyramid network (RFP), and then the feature map of the input region proposal network is matched with fixed features to obtain an input feature map containing ROIs.

[0096] Finally, the ROI in the matched feature map is classified. Specifically, the input feature map containing the ROI is input into a Mask R-CNN (Mask Region-based Convolutional Neural Network) for classification. The Mask R-CNN is a deep learning model configured for object detection and instance segmentation, which can generate a bounding box of the object, an instance segmentation image of the object, and a binary mask of the object at the same time. After the matched input feature map with the ROI is recognized and segmented by the Mask R-CNN, the input feature map is segmented into a peristaltic wave region and a non-peristaltic wave region, and a bounding box of the peristaltic wave region and a binary mask of the peristaltic wave region are generated at the same time. In this regard, a FCN (Fully Convolutional Networks) operation is performed on each ROI, that is, a fully connected layer in the Mask R-CNN is replaced by a convolutional layer.

[0097] In this embodiment, the image set is input into the pre-trained wrinkle region recognition model, and the corresponding wrinkle region of each first image is determined according to the output result of the model. The pre-trained wrinkle region recognition model is used to quickly identify the wrinkle region of each first image, which accelerates the construction speed of the three-dimensional model and is beneficial to improve the medical experience of the patient.

[0098] In one of the embodiments, the images in the image set are input into the pre-constructed wrinkle region recognition model, including: obtaining the acceleration of the capsule endoscope when collecting each image; according to whether the acceleration is greater than a threshold, screening a candidate set from the image set; performing motion blur detection on each image in the candidate set to obtain the motion blur condition of each image in the candidate set; according to whether the motion blur condition meets a preset condition, screening a target set from the candidate set; and inputting each image in the target set into the pre-constructed wrinkle region recognition model.

[0099] As can be understood, the endoscope sent into the human body is affected by a force due to the influence of gravity, and the force is not fixed. The force value corresponding to each moment can be different. Therefore, the capsule endoscope can be subjected to different forces when collecting each image, so that the current acceleration is also different. Since the acceleration of the image is increased due to the influence of the peristaltic wave, the images with an acceleration greater than a threshold can be screened from the image set as a candidate set for further judging the motion blur condition.

[0100] Optionally, the motion blur condition refers to blur caused by motion affecting the captured image, the motion blur condition of each image in the candidate set is detected, a target set in which the motion blur condition satisfies a determination condition that the target set is possibly affected by the peristalsis of the digestive cavity wall is selected, and the target set is input into the wrinkle recognition region recognition model.

[0101] Optionally, the candidate set in which the motion blur condition needs to be further detected is selected from the image set by acceleration, and the target set input into the wrinkle region recognition model is determined from the candidate set according to the motion blur condition, so that the detection cost is reduced by narrowing the range of motion blur condition detection, and the cost of model output is also reduced by inputting only the target set in which the motion blur condition satisfies the preset condition into the wrinkle region recognition model for recognition, thereby accelerating the construction speed and construction quality of the three-dimensional model.

[0102] In one embodiment, the step S203 of restoring the wrinkle region in each first image to a flat region to obtain a plurality of processed first images includes: obtaining a motion condition between the plurality of first images; and based on the motion condition, restoring the wrinkle region in each first image to a flat region to obtain a plurality of processed first images.

[0103] Optionally, since the wrinkle region image can be considered as an image with small, random direction, and high frequency motion, the motion condition can include optical flow between adjacent first images.

[0104] In one embodiment of the present disclosure, the specific steps of how to restore the wrinkle region in each first image to a flat region are as follows:

[0105] ① Corner point detection is performed on each first image to obtain corner points of each first image.

[0106] Optionally, any object in an image usually contains unique features, which are often composed of a large number of pixel points, and a corner point is a small set of points that can accurately describe the object. The corner point detection algorithm can analyze the most obvious feature points of the image in order to perform object recognition and tracking.

[0107] Optionally, in a stomach image, the corner point can be a wrinkle region in the image, and compared with other regions in the image, the wrinkle regions have obvious changes in brightness.

[0108] ② The optical flow between adjacent first images is calculated according to the corner points of each first image.

[0109] Optionally, the moving track of the target object in the image caused by the movement of the target object or the capsule endoscope in the two continuous frames of images is referred to as optical flow. It is a 2D vector field, which can be used to show the moving track of a point from the first frame of image to the second frame of image.

[0110] It can be understood that the moving track can include the motion direction, displacement, speed, acceleration and the like of the point.

[0111] Optionally, since there can be some points of wrong tracking (outliers) when directly using the optical flow method for tracking, in order to improve the image processing accuracy, the outliers can be corrected / removed by using the RANSAC (RANdom SAmple Consensus) algorithm.

[0112] Specifically, the corner points in the adjacent first images can be matched by using the RANSAC algorithm, and then the optical flow between the adjacent first images can be calculated according to the mutually matched corner points in the adjacent first images.

[0113] It can be understood that the RANSAC algorithm can find the mutually matched point set from the data containing noise in the adjacent first images, and then calculate the transformation matrix between the adjacent images. On this basis, the optical flow between the two frames of images can be calculated according to the transformation matrix.

[0114] ③Motion smoothing is performed on the optical flow between the adjacent first images to obtain the smoothed optical flow.

[0115] Optionally, the median filtering, mean filtering, Kalman filtering and the like filtering algorithms can be used to filter the motion parameters in the image sequence, so as to obtain the smoothed motion track.

[0116] It can be understood that the motion parameters refer to the parameters contained in the moving track between the adjacent first images, i.e. the displacement, speed, acceleration and the like.

[0117] ④According to the smoothed optical flow between the adjacent first images, image inpainting is performed on the multiple frames of first images to obtain the multiple frames of processed first images.

[0118] Optionally, the smoothed motion track can be subtracted from the original track to obtain the motion inpainting parameters of each frame of first image.

[0119] Optionally, according to the motion inpainting parameters of each frame of first image, the first image containing the wrinkle region output by the above wrinkle region identification model can be transformed to obtain a stabilized image, so as to restore the region affected by the peristaltic wave to the region before being affected by the peristaltic wave.

[0120] In this embodiment, by obtaining the motion between multiple frames of first images and restoring the wrinkled regions in each frame of first images to flat regions according to the motion, more stable data foundation is laid for subsequent construction of the three-dimensional model of the inner wall of the digestive cavity, and the integrity and accuracy of the model construction are ensured.

[0121] In one exemplary embodiment, the step S204, i.e., obtaining the three-dimensional model of the inner wall of the digestive cavity based on the multiple frames of processed first images, comprises: inputting the multiple frames of processed first images into a pre-constructed three-dimensional reconstruction algorithm; and obtaining the three-dimensional model of the inner wall of the digestive cavity according to the output result of the three-dimensional reconstruction algorithm.

[0122] The processed first image is a flat image, and the three-dimensional reconstruction algorithm can be understood as a process of reconstructing a three-dimensional scene or object by computer algorithm by fusing a series of two-dimensional images or point cloud data obtained from different angles or sensors. The three-dimensional reconstruction algorithm disclosed in the present disclosure but not limited to DIM-SLAM (Dense RGB Simultaneous Localization and Mapping with Neural Implicit Maps, Dense RGB Simultaneous Localization and Mapping with Neural Implicit Maps).

[0123] In this embodiment, the three-dimensional reconstruction algorithm DIM-SLAM is taken as an example to exemplarily introduce the process of constructing the three-dimensional model of the inner wall of the digestive cavity, please refer to the following steps:

[0124] 1. Data acquisition:

[0125] The above-mentioned images not affected by peristaltic waves (these images are two-dimensional images, including the original images not affected by peristaltic waves collected by the capsule endoscope and the processed first images) are collected. These two-dimensional images cover different regions of the stomach cavity to facilitate comprehensive three-dimensional reconstruction.

[0126] 2. Image preprocessing:

[0127] Before SLAM (Simultaneous Localization and Mapping), the image needs to be optimized through denoising, contrast enhancement and other steps to improve the detectability of feature points.

[0128] 3. Feature extraction and matching:

[0129] The feature extraction algorithm is used to extract the feature points in the image.

[0130] Based on the feature points between adjacent frames, correlation matching is performed to track the motion of these points in the image sequence.

[0131] 4. Capsule endoscope motion estimation:

[0132] Using the matching information of feature points, the motion posture of the camera (the camera on the capsule endoscope, and then the posture of the capsule endoscope) is estimated by PnP (Perspective-n-Point) algorithm and other methods. This step is the key to estimate the posture p (including position and direction) of the capsule endoscope.

[0133] 5. Map construction:

[0134] First, the tracked feature points are projected into three-dimensional space to construct a sparse 3D map of the stomach cavity.

[0135] 5.1, Selecting convolutional neural network and fully connected network structure to process data: Considering that the input data is mainly two-dimensional image, the two-dimensional image features are used to infer the three-dimensional structure, so CNN (Convolutional Neural Network) should be used to process image data to extract features (convolutional layer can effectively extract and process image features, and pooling layer can help the network understand features of different scales, which meets the needs of three-dimensional reconstruction), and fully connected layer is used to process and predict points in three-dimensional space.

[0136] 5.2, Network training: Using known sparse three-dimensional point cloud and corresponding RGB (red green blue) image data to train the neural network, learning to predict the occupancy state or depth value of any point in space.

[0137] 5.3, Sparse to dense mapping: Using the trained model, the data points in the sparse map are input, and the network outputs a continuous geometric field representation, which can be used to infer the dense structure of the entire scene.

[0138] 5.4, Scene reconstruction: Through the neural implicit representation network, the depth of each pixel point is inferred to generate a depth map, so as to obtain more continuous and detailed 3D information.

[0139] In DIM-SLAM, neural implicit map representation is used to process these data, and deep learning is used to optimize the accuracy and details of the map.

[0140] It should be noted that the representation of three-dimensional space can be divided into explicit and implicit. Common explicit representations include voxels, point clouds, and triangular meshes. Common implicit representations include signed distance functions, occupancy fields, and neural radiance fields (NeRF). Neural implicit maps actually use neural networks to fit specific mapping relationships to represent three-dimensional space. In this paper, neural networks are used to implicitly represent three-dimensional maps.

[0141] Specifically, as shown in FIG. 4, for each camera pose p obtained in step 4 above, the multi-scale feature volume is sampled along the view ray corresponding to the pose, and the multi-scale feature volume obtained by sampling is spliced to obtain a spliced 3D feature code. The spliced features are input into a multilayer perceptron (MLP) to calculate the depth and color of each pixel, and a depth image and a color image are obtained. The depth image and the color image are matched, and a 3D scene map is solved using the multilayer perceptron.

[0142] In detail, the camera pose p is input into the multilayer perceptron to obtain the predicted depth D p and color I p corresponding to the feature volume corresponding to the view ray corresponding to the pose as follows:

[0143] The multilayer perceptron is adjusted using a photometric distortion loss function (L warping ) and a color loss function (L render ) to enhance the consistency between the predicted results and the specific observation image (selected from a group of frame sequences as a key frame as a specific observation image).

[0144] 5.5, refinement and optimization: post-processing of the generated dense map, such as noise reduction and smoothing, to further improve the reconstruction quality.

[0145] 5.6, rendering and visualization: rendering the refined three-dimensional map to generate a visual 3D map. This map not only has high-resolution details in areas with rich feature points, but also can depict areas that are not covered in the original sparse map. This dense map provides a comprehensive and detailed three-dimensional model of the digestive tract for doctors, greatly enhancing the accuracy and efficiency of digestive tract examination and diagnosis.

[0146] 6. Photometric consistency check:

[0147] The photometric consistency check, i.e., using photometric warping loss, is applied to verify the correctness of the map points, ensuring the accuracy of the reconstruction results.

[0148] The photometric warping loss is the MSE loss between the rendered image and the real image, and the exposure variable is added to the loss function because different images are taken at different positions and thus receive different lighting, which affects image rendering, so the exposure variable is added.

[0149] Specifically:

[0150] 6.1. Projection mapping:

[0151] ① Determine the reference view: select an image at a time point as the reference view. Usually the first image in the image sequence or the most stable image in the motion trajectory is selected.

[0152] ② Estimate camera pose: use SLAM (Simultaneous Localization and Mapping) algorithm to estimate the position and orientation of the capsule endoscope at each time point. This usually involves feature point matching and motion estimation.

[0153] ③ Establish a three-dimensional reference framework: according to the estimated camera pose, project the images taken by the capsule endoscope into a common three-dimensional reference framework for comparison.

[0154] ④ Image alignment: use image registration techniques to align images at different time points according to the three-dimensional reference framework, ensuring that the same physical points in the images match in position in different images.

[0155] ⑤ Extraction and comparison of luminance values: extract the luminance values of the same scene points in the aligned images and compare them. In an ideal case, even if taken from different angles, the luminance values of the same physical points in different images should be consistent.

[0156] ⑥ Difference analysis and threshold check: calculate the luminance difference and compare it with the preset threshold. If the luminance difference exceeds the threshold, it indicates that there may be errors in the three-dimensional position estimation or image registration at that point.

[0157] 6.2. Photometric comparison:

[0158] Compare the luminance values (brightness and color) of the projected pixel points in the reference frame and the current frame. Ideally, if the camera pose and three-dimensional map are accurate, these values should be consistent.

[0159] 6.3. Error calculation:

[0160] The photometric error is calculated, typically by computing the sum of squares of the photometric differences between the reference frame and the other frames.

[0161] 6.4. Threshold decision:

[0162] A threshold is set to determine whether the photometric error is within an acceptable range. If the error is greater than the threshold, it may indicate that the camera pose estimation or the three-dimensional map is incorrect.

[0163] 7. Precision optimization and refinement:

[0164] Optimization techniques in SLAM, such as Bundle Adjustment, are used to further optimize the precision of the camera trajectory and map points.

[0165] Deep neural networks are applied to refine the sparse map, generating a more detailed and continuous three-dimensional model of the stomach cavity.

[0166] 8. Result verification:

[0167] Finally, the reconstructed three-dimensional model needs to be verified to ensure that it accurately reflects the actual structure of the stomach cavity.

[0168] In this embodiment, by inputting multiple frames of processed first images into a pre-constructed three-dimensional reconstruction algorithm, a corresponding digestive cavity inner wall three-dimensional model is obtained according to the output results of the three-dimensional reconstruction algorithm. Using the pre-constructed three-dimensional reconstruction algorithm to construct the digestive cavity inner wall three-dimensional model is beneficial to speeding up the construction of the digestive cavity inner wall three-dimensional model. At the same time, after construction, the model is further judged for accuracy and processed if the accuracy is not high, thereby improving the accuracy of the constructed digestive cavity inner wall three-dimensional model.

[0169] In one embodiment, the above-mentioned inputting of multiple frames of processed first images into a pre-constructed three-dimensional reconstruction algorithm can include inputting the multiple frames of processed first images and second images other than the first images in the image set into the three-dimensional reconstruction algorithm.

[0170] On this basis, the output results of the three-dimensional reconstruction algorithm can include an initial digestive cavity inner wall three-dimensional model generated according to the multiple frames of processed first images and the second images other than the first images in the image set. Considering that the initial digestive cavity inner wall three-dimensional model may have a low accuracy, the accuracy of the initial digestive cavity inner wall three-dimensional model can be calculated, and the images can be further processed to generate a more accurate digestive cavity inner wall three-dimensional model if the accuracy of the initial digestive cavity inner wall three-dimensional model is low.

[0171] Specifically, the output result of the three-dimensional reconstruction algorithm is used to obtain the three-dimensional model of the inner wall of the digestive cavity, which can comprise: if the accuracy of the initial three-dimensional model of the inner wall of the digestive cavity does not reach the accuracy threshold, the acceleration corresponding to the collection of each second image by the capsule endoscope is obtained; the relative size of the acceleration is used to determine the second image to be recovered in each second image; the second image to be recovered is subjected to flat recovery processing to obtain a recovered second image; and the three-dimensional model of the inner wall of the digestive cavity is obtained based on the recovered second image, the plurality of processed first images, and other second images except the recovered second image.

[0172] Optionally, when the three-dimensional model of the inner wall of the digestive cavity is first constructed, the plurality of processed first images and the second images in the image set except the first images can be input into the pre-constructed three-dimensional reconstruction algorithm to obtain an initial three-dimensional model of the inner wall of the digestive cavity. If the accuracy of the initial three-dimensional model of the inner wall of the digestive cavity does not reach the preset accuracy threshold, the acceleration corresponding to the collection of each second image by the capsule endoscope is obtained, and the relative size of the acceleration is used to select the second image to be recovered. The second image to be recovered is also affected by the peristaltic wave of the digestive cavity wall, but the degree of influence is not as serious as the first image. In order to obtain enough stable images, the second image to be recovered is subjected to flat recovery processing to obtain a recovered second image. Finally, the three-dimensional model of the inner wall of the digestive cavity is obtained based on the recovered second image, the plurality of processed first images, and other images in the image set.

[0173] Optionally, if the accuracy of the initial three-dimensional model of the inner wall of the digestive cavity reaches the preset accuracy threshold, the initial three-dimensional model of the inner wall of the digestive cavity can be directly used as the final three-dimensional model of the inner wall of the digestive cavity.

[0174] Through the above method, the second image to be recovered which is not greatly affected by the peristaltic wave of the digestive cavity wall is subjected to flat recovery processing, so that more stable images are obtained, and the accuracy of the three-dimensional model of the inner wall of the digestive cavity is improved.

[0175] In one embodiment, the second image to be recovered is subjected to flat recovery processing to obtain a recovered second image, which comprises: each second image to be recovered is subjected to flat recovery processing by a filtering algorithm and a smoothing algorithm to obtain a corresponding recovered second image.

[0176] Optionally, the filtering algorithm is used to remove noise in the second image to be recovered, and the smoothing algorithm is used to remove fluctuations or discontinuities in the second image to be recovered to make it smoother, so as to obtain a corresponding recovered second image. Since the second image to be recovered is not greatly affected by the peristaltic wave of the digestive cavity wall, the filtering algorithm and the smoothing algorithm can be directly used to reduce the processing cost and the steps are simple, which is conducive to speeding up the construction speed of the three-dimensional model of the inner wall of the digestive cavity.

[0177] In an exemplary embodiment, as shown in FIG. 5, a detailed method for reconstructing a three-dimensional model of the inner wall of the digestive cavity is provided, and the specific steps include S501 to S512, wherein:

[0178] Step S501, obtaining an image set collected by the capsule endoscope inside the digestive cavity.

[0179] Step S502, obtaining the acceleration of the capsule endoscope when collecting each image.

[0180] Step S503, selecting images with acceleration greater than a threshold value as a candidate set, and detecting the motion blur condition of each image in the candidate set.

[0181] Step S504, selecting images with motion blur conditions meeting a preset condition from the candidate set as a target set, and inputting each image in the target set into a shrinkage region identification model.

[0182] Step S505, obtaining a plurality of first images formed under the influence of the peristalsis of the digestive cavity wall and the shrinkage region in each first image according to the prediction result output by the shrinkage region identification model.

[0183] Step S506, obtaining the motion condition between the plurality of first images, and restoring the shrinkage region in each first image to a flat region according to the motion condition to obtain a plurality of processed first images. Step S507, inputting the plurality of processed first images into a pre-constructed three-dimensional reconstruction algorithm to obtain a three-dimensional model of the inner wall of the digestive cavity.

[0184] Specifically, the three-dimensional model of the inner wall of the digestive cavity can be obtained through steps S508 to S513.

[0185] Step S508, inputting the plurality of processed first images and the second images other than the first images in the image set into a three-dimensional reconstruction algorithm to obtain an initial three-dimensional model of the inner wall of the digestive cavity.

[0186] Step S509, if the accuracy of the initial three-dimensional model of the inner wall of the digestive cavity does not reach an accuracy threshold, obtaining the acceleration of the capsule endoscope when collecting each second image.

[0187] Step S510, determining a to-be-restored second image in each second image according to the relative size of the acceleration.

[0188] Step S511, performing flat restoration processing on each to-be-restored second image through a filtering algorithm and a smoothing algorithm to obtain a corresponding restored second image.

[0189] Step S512, performing flat restoration processing on the to-be-restored second image to obtain a restored second image.

[0190] Step S513, based on the recovered second image, the multiple frames of processed first images and other second images except the recovered second image, a three-dimensional model of the inner wall of the digestive cavity is obtained.

[0191] Compared with the prior art, the present disclosure has the following advantages:

[0192] The image set of the inside of the digestive cavity obtained by the capsule endoscope is input into the shrinkage area identification model, and the shrinkage area of the first image affected by the peristaltic wave of the digestive cavity wall is determined according to the output result, thereby laying a data foundation for subsequent flattening and recovery of the shrinkage area of each frame of first image.

[0193] The shrinkage area of each frame of first image is flattened and recovered according to the motion between multiple frames of first images, and multiple frames of processed first images are obtained. More stable first images are obtained by the above method, which is conducive to improving the accuracy of the three-dimensional model of the inner wall of the digestive cavity.

[0194] The multiple frames of processed first images are input into the pre-constructed three-dimensional reconstruction algorithm, and a three-dimensional model of the inner wall of the digestive cavity is output, thereby accelerating the construction speed of the three-dimensional model of the inner wall of the digestive cavity.

[0195] It should be understood that, although each step in the flowchart involved in each of the above embodiments is displayed in sequence according to the arrow, these steps are not necessarily executed in the order indicated by the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other orders. Moreover, at least part of the steps in the flowchart involved in each of the above embodiments can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or steps or stages in other steps.

[0196] Based on the same inventive concept, the present disclosure also provides a three-dimensional model of the inner wall of the digestive cavity reconstruction device configured to implement the above-mentioned three-dimensional model of the inner wall of the digestive cavity reconstruction method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more three-dimensional model of the inner wall of the digestive cavity reconstruction device embodiments provided below can refer to the limitations of the three-dimensional model of the inner wall of the digestive cavity reconstruction method described above, which will not be repeated here.

[0197] In one exemplary embodiment, as shown in FIG. 6, a three-dimensional model of the inner wall of the digestive cavity reconstruction device is provided, which includes an image set acquisition module 601, a shrinkage area determination module 602, an image processing module 603 and a three-dimensional model determination module 604, wherein:

[0198] The image set obtaining module 601 is configured to obtain an image set collected by the capsule endoscope inside the digestive cavity.

[0199] The wrinkled region determining module 602 is configured to determine, in the image set, a plurality of first images formed under the influence of the peristalsis of the wall of the digestive cavity, and determine a wrinkled region in each of the first images.

[0200] The image processing module 603 is configured to restore the wrinkled region in each of the first images to a flat region to obtain a plurality of processed first images.

[0201] The three-dimensional model determining module 604 is configured to obtain a three-dimensional model of the inner wall of the digestive cavity based on the plurality of processed first images.

[0202] In one embodiment, the wrinkled region determining module 602 comprises an image input sub-module and a wrinkled region determining sub-module, wherein:

[0203] The image input sub-module is configured to input the images in the image set into a pre-constructed wrinkled region identification model.

[0204] The wrinkled region determining sub-module is configured to obtain, according to a prediction result output by the wrinkled region identification model, the plurality of first images formed under the influence of the peristalsis of the wall of the digestive cavity and the wrinkled region in each of the first images.

[0205] In one exemplary embodiment, the image input sub-module is specifically configured to obtain an acceleration corresponding to the collection of each image by the capsule endoscope; filter out a candidate set from the image set according to whether the acceleration is greater than a threshold value; perform motion blur detection on each image in the candidate set to obtain a motion blur condition of each image in the candidate set; filter out a target set from the candidate set according to whether the motion blur condition meets a preset condition; and input each image in the target set into the pre-constructed wrinkled region identification model.

[0206] In one embodiment, the image processing module 603 is specifically configured to obtain a motion condition between the plurality of first images; and restore the wrinkled region in each of the first images to a flat region based on the motion condition to obtain the plurality of processed first images.

[0207] In one exemplary embodiment, the image processing module 603 is specifically configured to perform corner point detection on each first image to obtain a corner point of each first image; and calculate an optical flow between adjacent first images according to the corner point of each first image.

[0208] In one exemplary embodiment, the image processing module 603 is specifically configured to perform motion smoothing on the optical flow between adjacent first images to obtain a smoothed optical flow.

[0209] According to the smoothed optical flow between adjacent first images, the multi-frame first images are inpainted to obtain multi-frame processed first images.

[0210] In one embodiment, the three-dimensional model determining module 604 is specifically configured to input the multi-frame processed first images into a pre-constructed three-dimensional reconstruction algorithm; and according to an output result of the three-dimensional reconstruction algorithm, obtain the three-dimensional model of the inner wall of the digestive cavity.

[0211] In one exemplary embodiment, the three-dimensional model determining module 604 is further specifically configured to input the multi-frame processed first images and second images other than the first images in the image set into the three-dimensional reconstruction algorithm; if the accuracy of the initial three-dimensional model of the inner wall of the digestive cavity does not reach the accuracy threshold, obtain the acceleration of the capsule endoscope when collecting each second image; according to the relative size of the acceleration, determine a to-be-recovered second image in each second image; perform flat recovery processing on the to-be-recovered second image to obtain a recovered second image; and based on the recovered second image, the multi-frame processed first images and the other second images other than the recovered second image, obtain the three-dimensional model of the inner wall of the digestive cavity.

[0212] In one of the embodiments, the three-dimensional model determining module 604 is further configured to perform flat recovery processing on each to-be-recovered second image by a filtering algorithm and a smoothing algorithm respectively to obtain a corresponding recovered second image.

[0213] Each module in the above-described reconstruction device of the three-dimensional model of the inner wall of the digestive cavity can be realized by software, hardware and a combination thereof in whole or in part. Each module described above can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a memory in a computer device in software form so as to be called and executed by a processor to perform the operations corresponding to each module.

[0214] In an example embodiment, a computer device is provided, which can be a server, and an internal structure diagram of the computer device can be as shown in FIG. 7. The computer device includes a processor, a memory, an input / output interface (I / O), and a communication interface. The processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The database of the computer device is configured to store image set data collected by a capsule endoscope in a digestive cavity. The input / output interface of the computer device is configured to exchange information between the processor and external devices. The communication interface of the computer device is configured to communicate with a terminal outside through a network connection. The computer program is executed by the processor to implement a method for reconstructing a three-dimensional model of an inner wall of a digestive cavity.

[0215] Those skilled in the art can understand that the structure shown in FIG. 7 is only a block diagram of part of the structure related to the present disclosure, and does not constitute a limitation on the computer device to which the present disclosure scheme should be configured. A specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0216] In an example embodiment, a computer device is provided, which includes a memory and a processor, and the memory stores a computer program. The processor implements the method for reconstructing a three-dimensional model of an inner wall of a digestive cavity according to the above-mentioned embodiments when executing the computer program.

[0217] In an embodiment, a computer readable storage medium is provided, which stores a computer program. The computer program is executed by a processor to implement the method for reconstructing a three-dimensional model of an inner wall of a digestive cavity according to the above-mentioned embodiments.

[0218] In an embodiment, a computer program product is provided, which includes a computer program. The computer program is executed by a processor to implement the method for reconstructing a three-dimensional model of an inner wall of a digestive cavity according to the above-mentioned embodiments.

[0219] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data configured to be analyzed, stored data, displayed data, etc.) involved in the present disclosure are all information and data authorized by the user or authorized by all parties, and the collection, use, and processing of related data need to comply with relevant regulations.

[0220] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of each method can be included. Any reference to memory, database or other medium used in each embodiment provided by the present disclosure can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in each embodiment provided by the present disclosure can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in each embodiment provided by the present disclosure can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0221] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of each technical feature in the above embodiments are not described, however, as long as the combination of these technical features does not exist contradictory, it should be considered as the scope of the present disclosure.

[0222] The above-described embodiments only express several implementation manners of the present disclosure, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent of the present disclosure. It should be noted that for those skilled in the art, without departing from the concept of the present disclosure, a number of modifications and improvements can be made, which are within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the appended claims. Industrial applicability:

[0223] The present disclosure provides a reconstruction method, device and equipment of a three-dimensional model of a digestive cavity inner wall and a medium, which can construct a more accurate digestive cavity model, help doctors more accurately judge the patient's lesion, and improve the efficiency and satisfaction of the patient's medical treatment.

Claims

1. A method for reconstructing a three-dimensional model of the inner wall of a digestive cavity, characterized in that: The method comprises: Acquire an image set collected by a capsule endoscope inside the digestive cavity; Determining, in the image set, a plurality of first image frames formed under the influence of peristalsis of the digestive cavity wall, and determining a shrinkage region in each first image frame; Restoring the wrinkled area in each frame of the first image into a flat area to obtain multiple frames of processed first images; A three-dimensional model of the inner wall of the digestive cavity is obtained based on the multiple frames of processed first images.

2. The method according to claim 1, characterized in that The step of determining a plurality of first images formed under the influence of the peristalsis of the digestive cavity wall in the image set, and determining a shrinkage region in each first image frame, comprises: Inputting the images in the image set into a pre-built wrinkle area recognition model; According to the prediction results output by the shrinkage area recognition model, multiple frames of first images formed under the influence of the peristalsis of the digestive cavity wall and the shrinkage area in each frame of the first image are obtained.

3. The method according to claim 2, characterized in that The shrinkage area recognition model includes a trained neural network configured to extract features from an input image and output a feature map, a region proposal network configured to generate a region of interest of an image, and a convolutional neural network based on a mask area.

4. The method according to any one of claims 2 to 3, characterized in that: The step of inputting the images in the image set into a pre-built shrinkage area recognition model comprises: Obtaining the acceleration corresponding to each image captured by the capsule endoscope; Filtering a candidate set from the image set according to whether the acceleration is greater than a threshold; Performing motion blur detection on each image in the candidate set to obtain a motion blur condition of each image in the candidate set; Filtering a target set from the candidate set according to whether the motion blur condition meets a preset condition; Each image in the target set is input into a pre-built wrinkle region recognition model.

5. The method according to claim 4, characterized in that Each image in the target set is blurred due to motion.

6. The method according to any one of claims 1 to 5, characterized in that Restoring the wrinkled area in each frame of the first image into a flat area to obtain multiple frames of processed first images includes: Acquiring motion conditions between multiple frames of first images; Based on the motion situation, the wrinkled area in each frame of the first image is restored to a flat area to obtain multiple frames of processed first images.

7. The method according to claim 6, characterized in that The motion condition includes optical flow between adjacent first images; The acquiring of motion conditions between multiple frames of first images includes: Performing corner point detection on each of the first images to obtain a corner point of each of the first images; The optical flow between adjacent first images is calculated according to the corner points of each of the first images.

8. The method according to claim 7, characterized in that The method of restoring the wrinkled area in each frame of the first image to a flat area based on the motion condition to obtain multiple frames of processed first images includes: Performing motion smoothing on the optical flow between adjacent first images to obtain a smoothed optical flow; Image restoration is performed on multiple frames of first images according to smoothed optical flows between adjacent first images to obtain multiple frames of processed first images.

9. The method according to any one of claims 1 to 8, characterized in that The step of obtaining a three-dimensional model of the inner wall of the digestive cavity based on the multiple frames of processed first images includes: Inputting the plurality of frames of processed first images into a pre-built three-dimensional reconstruction algorithm; According to the output results of the three-dimensional reconstruction algorithm, a three-dimensional model of the inner wall of the digestive cavity is obtained.

10. The method according to claim 9, characterized in that The step of inputting the plurality of frames of processed first images into a pre-constructed three-dimensional reconstruction algorithm comprises: Inputting the plurality of frames of processed first images and second images in the image set excluding the first images into the three-dimensional reconstruction algorithm; The output result of the three-dimensional reconstruction algorithm includes an initial three-dimensional model of the inner wall of the digestive cavity generated based on the multiple frames of processed first images and a second image in the image set other than the first image. The three-dimensional model of the inner wall of the digestive cavity obtained based on the output result of the three-dimensional reconstruction algorithm includes: If the accuracy of the initial digestive cavity inner wall three-dimensional model does not reach the accuracy threshold, obtaining the acceleration corresponding to the capsule endoscope when acquiring each second image; determining a second image to be restored in each second image according to the relative magnitude of the acceleration; Performing a smoothing restoration process on the second image to be restored to obtain a restored second image; A three-dimensional model of the inner wall of the digestive cavity is obtained based on the restored second image, multiple frames of processed first images, and other second images except the restored second image.

11. The method according to claim 10, characterized in that The performing a smoothing and restoring process on the second image to be restored to obtain a restored second image includes: Each of the second images to be restored is smoothed and restored by using a filtering algorithm and a smoothing algorithm to obtain a corresponding restored second image.

12. A device for reconstructing a three-dimensional model of the inner wall of a digestive cavity, characterized in that: The device comprises: an image set acquisition module configured to acquire an image set collected by the capsule endoscope inside the digestive cavity; a shrinkage region determining module configured to determine, in the image set, a plurality of first image frames formed under the influence of peristalsis of the digestive cavity wall, and determine a shrinkage region in each first image frame; an image processing module configured to restore the wrinkled area in each frame of the first image to a flat area to obtain multiple frames of processed first images; The three-dimensional model determination module is configured to obtain a three-dimensional model of the inner wall of the digestive cavity based on the multiple frames of processed first images.

13. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 11 are implemented.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

Images