Capsule endoscopy lesion automatic identification and positioning system

Abstract

The invention relates to the technical field of medical instruments, in particular to a capsule endoscopy focus automatic identification and positioning system. The system comprises an image acquisition module, an intelligent analysis module and a space positioning module. The image acquisition module receives an alimentary canal image sequence captured by the capsule endoscopy; the intelligent analysis module is integrated with a pre-trained deep convolutional neural network, analyzes the image frame by frame, and outputs a pixel-level segmentation mask identifying the contour and type of the focus; and the spatial positioning module estimates the pose of the capsule endoscopy and constructs a topological map of the inner wall of the digestive tract by analyzing the motion of feature points in the image sequence and combining a visual odometer and a graph optimization SLAM algorithm, and finally maps the recognized focus to specific coordinates of the map. According to the invention, automatic identification and accurate positioning of the nidus of the digestive tract are realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical devices, more particularly, the present application relates to a capsule endoscopy lesion automatic identification and positioning system. BACKGROUND

[0002] Capsule endoscopy technology is an important field of non-invasive diagnosis of digestive tract diseases, and its development has experienced evolution from simple image acquisition to preliminary wireless transmission and image processing functions. Currently, this field is rapidly developing towards intelligence and precision, aiming to introduce computer vision and artificial intelligence technology to assist doctors in improving the efficiency and accuracy of lesion identification, and further exploring the accurate judgment of lesion location, so as to provide more comprehensive decision support for clinical diagnosis. Realizing the automatic identification and accurate positioning of lesions has become one of the key trends in the evolution of this technology.

[0003] The prior art mainly has the following deficiencies: First, in the aspect of lesion identification, most systems rely on traditional image processing algorithms or relatively basic convolutional neural networks, which have limited feature extraction capabilities for lesions with various shapes, fuzzy boundaries and small sizes, are prone to missed or false detection, and have insufficient model generalization ability, making it difficult to adapt to the differences in different patients' digestive tract environments. Secondly, in the aspect of spatial positioning, existing solutions are mostly limited to estimating the approximate trajectory of the capsule by receiving signal strength or rough inter-frame motion, which cannot construct a coherent and accurate topological map of the inner wall of the digestive tract, making it difficult to accurately associate the identified lesions with the specific anatomical location in the digestive tract, and limiting the clinical guidance value. In addition, the modules of existing systems usually work independently, and the identification and positioning processes are disconnected, lacking an efficient coordination mechanism to integrate visual information and spatial information, which not only affects the overall operation efficiency of the system, but also limits the further improvement of positioning accuracy.

[0004] Therefore, in view of the above problems, a capsule endoscopy lesion automatic identification and positioning system is proposed, which aims to solve the deficiencies in the prior art and mainly addresses two core problems: one is how to improve the automatic identification accuracy and robustness of multiple types of lesions in capsule endoscopy images; the other is how to achieve accurate positioning of lesions in the three-dimensional space inside the digestive tract without using external positioning devices. SUMMARY

[0005] In order to overcome the above-mentioned defects of the prior art, the embodiments of the present application provide a capsule endoscopy lesion automatic identification and positioning system to solve the problems raised in the background art.

[0006] To achieve the above-mentioned purpose, the present application provides the following technical scheme: a capsule endoscopy lesion automatic identification and positioning system, comprising an image acquisition module, an intelligent analysis module and a spatial positioning module; The image acquisition module is configured to receive and store a sequence of color or grayscale images captured continuously by the capsule endoscope while moving inside the digestive tract, with an image resolution of 512x512 pixels to 2048x2048 pixels and a frame rate of 2 frames / s to 30 frames / s, and transmit the image sequence to the intelligent analysis module. The intelligent analysis module is integrated with a pre-trained deep convolutional neural network model, which adopts a multi-scale feature extraction and fusion architecture, contains 3 to 8 down-sampling stages and a corresponding number of up-sampling stages, each stage contains 1 to 3 convolutional layers, and the convolutional kernel size is 3x3 pixels to 7x7 pixels, which can process the input image sequence frame by frame, and output a pixel-level segmentation mask of the lesion area in each frame, wherein the segmentation mask identifies the outline and type of the lesion. The spatial positioning module is connected to the intelligent analysis module, receives the segmentation mask and the original image sequence, extracts key point features in the image and tracks the motion changes of these features in consecutive frames, combines the approximate motion dynamics model of the capsule endoscope, estimates the real-time pose of the capsule endoscope in the digestive tract using an incremental three-dimensional reconstruction algorithm, and constructs a two-dimensional or three-dimensional topological map of the inner wall of the digestive tract, thereby mapping the identified lesion area to specific location coordinates on the map. The system further comprises a data interface module for outputting the identification and positioning results to external display or storage devices.

[0007] Further, the deep convolutional neural network model is implemented based on an encoder-decoder structure, the encoder part uses convolutional layers and pooling layers to gradually down-sample and extract image features, the feature map size is gradually reduced to 1 / 2 to 1 / 16 of the original image, the decoder part uses deconvolutional layers and skip connections to gradually up-sample and restore spatial resolution, and finally outputs the class probability of each pixel through a softmax classifier, wherein the model training uses a total loss function for optimization, and the total loss function is composed of a cross-entropy loss and a Dice loss, and the expression is: wherein Ltotal is the total loss, Lcross-entropy is the cross-entropy loss for measuring the difference between the predicted probability distribution and the true distribution, Ldice is the Dice similarity coefficient loss for handling the imbalance between foreground and background pixels, w1 and w2 are weight coefficients for balancing the contribution of the two losses, and the model uses the loss function for end-to-end learning on a dataset containing 100,000 to 500,000 labeled digestive tract lesions.

[0008] Further, the spatial localization module estimates the capsule endoscope pose by first extracting 50 to 1000 key points from the images using a FAST corner detector or SIFT descriptor, then tracking the motion of the key points in the sequence using an LK optical flow method or descriptor matching, and optimizing the pose transformation matrix by solving a nonlinear least squares problem whose objective is to minimize the sum of the reprojection errors of all feature points, which is mathematically expressed as: wherein is the optimal pose transformation matrix to be solved, T is a candidate value of the pose transformation matrix, is the jth three-dimensional map point in the world coordinate system, is the corresponding two-dimensional image feature point observed in the ith frame of image, π(·) is the camera projection function for projecting a three-dimensional space point to a two-dimensional image pixel coordinate, ρ(·) is a robust kernel function for reducing the impact of false matches, and finally the global trajectory and map points are optimized by Bundle Adjustment, wherein the pose estimation includes position and direction parameters, and the position accuracy is in the range of 1 mm to 5 mm.

[0009] Further, the intelligent analysis module further includes an online adaptive submodule, which can dynamically adjust the parameters of the deep learning model according to the newly input image data, update the model weights by using the real-time feedback of the labeled information in an incremental learning manner, and the model update frequency is 1 time per 10 to 100 frames of images processed, so as to adapt to the changes of different patients or lesion shapes.

[0010] Further, the spatial localization module adopts a SLAM method based on graph optimization when constructing the topological map of the digestive tract inner wall, which represents the capsule endoscope pose and map points as graph nodes, minimizes the reprojection error by adding key frame and loop detection constraints, wherein the key frame selection interval is 5 to 20 frames, and the loop detection uses a bag-of-words model for scene recognition, thereby generating a consistent environment map.

[0011] Further, the image acquisition module includes a preprocessing unit that performs standardization processing on the received image sequence, including gamma correction with a gamma correction coefficient of 0.8 to 1.2, histogram equalization, and Gaussian filtering with a standard deviation of 0.5 to 1.5 pixels, to eliminate the effects of light changes and noise, and adjusts the image size to a fixed resolution range of 512×512 pixels to 1024×1024 pixels.

[0012] Further, the system also comprises a result fusion module, which receives the segmentation mask output by the intelligent analysis module and the position coordinates generated by the spatial positioning module, aligns the lesion information with the time stamp, and integrates into a structured data report, which is stored in JSON or XML format, containing the fields of lesion type, size, position and confidence, wherein the confidence threshold is 0.7 to 0.95.

[0013] Further, the training process of the deep convolutional neural network model uses data enhancement techniques, including random rotation of-15 degrees to +15 degrees, scaling of 0.8 times to 1.2 times, and color jittering, including brightness adjustment of-10% to +10% and contrast adjustment of-10% to +10%, to improve the model generalization ability, and the training data comes from a multi-center cooperative anonymous endoscopic image database.

[0014] Further, the system also comprises a quality control module, which performs reliability evaluation on the segmentation results output by the intelligent analysis module, and when the lesion detection confidence of consecutive 3 to 10 frames of images is lower than 0.6 to 0.8, triggers a re-analysis mechanism to send the related images into an auxiliary verification process.

[0015] Further, the spatial positioning module also comprises a motion compensation unit, which smoothes the motion trajectory of the capsule endoscope through a Kalman filter, and the filter state variables include position, velocity and acceleration, and the observation period is 0.1 seconds to 0.5 seconds, to reduce the positioning error caused by the peristalsis of the digestive tract.

[0016] Technical effects and advantages of the present application: Compared with the prior art, the present application constructs a deep convolutional neural network model integrating multi-scale feature extraction and fusion architecture, which adopts an encoder-decoder structure and utilizes a skip connection to combine deep semantic information with shallow contour information, realizing simultaneous capture of image details and global context. In operation, the model processes the input image frame by frame and outputs a pixel-level lesion segmentation mask. In addition, the system also introduces an online adaptive sub-module, which can fine-tune the model parameters in an incremental learning manner according to the newly input patient image data, significantly improving the recognition ability of lesions of different shapes and sizes, especially for small and irregular boundary lesions, with more accurate segmentation. The advantage is that it reduces the misjudgment rate caused by individual differences between patients and the variability of lesion morphology, and enhances the generalization ability and reliability of the system in actual clinical application.

[0017] Compared with the prior art, the present application realizes accurate lesion spatial positioning by fusing visual odometry and graph optimization SLAM (simultaneous localization and mapping) technology. The system extracts and tracks key point features in the image sequence, and uses epipolar geometry and Bundle Adjustment technology to estimate the six-degree-of-freedom pose change of the capsule endoscope in a random manner. In operation, the system constructs a graph optimization problem with the pose and the observed map points, and corrects the cumulative error through loop closure detection, thereby constructing a consistent two-dimensional or three-dimensional topological map of the inside of the digestive tract. The two-dimensional image lesions identified by the intelligent analysis module are accurately mapped to specific three-dimensional coordinates on the topological map. The advantage is that it overcomes the problems of rough positioning and large cumulative error of traditional methods, and can provide the doctor with relatively accurate position information of the lesion in the digestive tract without external sensors, greatly improving the intuitiveness of diagnosis and the potential possibility of surgical navigation. BRIEF DESCRIPTION OF DRAWINGS

[0018] Fig. 1 is the overall framework diagram of the system of the present application.

[0019] Fig. 2 is the workflow diagram of the intelligent analysis module of the present application.

[0020] Fig. 3 is the workflow diagram of the spatial positioning module of the present application.

[0021] Fig. 4 is the result fusion and output flowchart of the present application. DETAILED DESCRIPTION

[0022] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0023] Embodiment one As shown in the accompanying drawings, a capsule endoscope lesion automatic recognition and positioning system is as follows: Figs. 1 to 4 The complete implementation process of the system is as follows: Implementation step 1 (image data acquisition and preprocessing) After the system is started, the image acquisition module first receives the original image sequence continuously captured by the capsule endoscope when moving in the digestive tract. The capsule endoscope takes pictures at a preset frame rate, usually in the range of 2 frames / second to 30 frames / second, and the resolution of the obtained images is between 512×512 pixels and 2048×2048 pixels. These original image sequences are received and temporarily stored in the image buffer of the system through wireless transmission.

[0024] Subsequently, the preprocessing unit is activated to perform a series of standardization operations on the raw images to improve the accuracy of subsequent analysis. The preprocessing operations mainly include three links: first, gamma correction is performed, and the correction coefficient γ is set in the range of 0.8 to 1.2, aiming to adjust the overall illumination intensity of the image and enhance the visibility of details of the captured image in the dark environment of the digestive tract; second, histogram equalization is performed to widen the gray level distribution of the image and enhance the contrast of the image; and finally, Gaussian filtering is applied, and the standard deviation σ of the filter is set in the range of 0.5 to 1.5 pixels, which can effectively suppress the random noise introduced by the image sensor and the image artifacts caused by the contents of the digestive tract.

[0025] After preprocessing is completed, all images are uniformly scaled to a fixed resolution range, such as 512x512 pixels to 1024x1024 pixels, to ensure that the data specifications of the subsequent intelligent analysis module are consistent. The preprocessed image sequence and its associated timestamp information are transmitted synchronously to the intelligent analysis module and the spatial positioning module for further processing.

[0026] Step 2 (lesion recognition and segmentation based on deep neural network) After the intelligent analysis module receives the preprocessed image sequence, it starts its core deep convolutional neural network model to analyze the images frame by frame. The model uses an encoder-decoder architecture, and the encoder part usually contains 3 to 8 down-sampling stages, which gradually reduces the input image size to 1 / 2 to 1 / 16 of the original image through the combination of convolutional layers and pooling layers, and extracts multi-scale features from low-level edges to high-level semantics in the process. The decoder part contains symmetric up-sampling stages with the encoder, which gradually restores the spatial resolution of the image through deconvolution layers and jump connection technology, and fuses the high-resolution features of the corresponding layers in the encoder with the deep semantic features of the decoder, thereby realizing accurate reconstruction of the lesion boundary details. The final output layer of the network is a softmax classifier that assigns a probability to each pixel belonging to different categories (such as normal tissue, various lesions), generating a pixel-level segmentation mask.

[0027] In order to train the network model, a training set containing a large amount of labeled data (usually 100,000 to 500,000 images) is needed, and a complex loss function is defined to guide the optimization of model parameters. The loss function combines cross-entropy loss and Dice loss, and its mathematical expression is as follows: where represents the total target of model optimization, i.e. the total loss value, represents the cross-entropy loss term, which quantifies the difference between the model's predicted class probability distribution for each pixel and the ground-truth class label annotated by human, aiming to drive the model's prediction probability as close to the true distribution as possible, represents the Dice loss term, which is derived from the Dice similarity coefficient and is specifically designed to handle the severe imbalance between the number of foreground (e.g., lesions) and background pixels in medical images. By optimizing the model to maximize the overlap between the predicted segmentation and the ground-truth annotation, the model becomes more sensitive to the shape and location of the lesions. and are two pre-defined weight coefficients that balance the relative importance of the cross-entropy loss and the Dice loss in the total loss. Their specific values are usually determined through cross-validation.

[0028] During model training, a gradient descent algorithm (e.g., Adam optimizer) is used to minimize the total loss function, thereby learning the end-to-end mapping from input images to accurate segmentation masks. For new input image frames, the trained model will directly output a binary or multi-valued segmentation mask that clearly identifies the contours of suspected lesion regions and their predicted types.

[0029] Implementation Step 3 (Capsule Endoscopy Pose Preliminary Estimation Based on Visual Odometry) In parallel with Step 2, the spatial localization module begins its work, with its primary task being to estimate the motion trajectory and pose of the capsule endoscope within the digestive tract. This is achieved using a visual odometry technique based on feature point methods. For the input image sequence, the system first extracts significant key points from each frame using a feature point detection algorithm, such as the FAST corner detector or SIFT descriptor. The number of extracted points typically ranges between 50 and 1000, depending on the image content and the pre-set complexity.

[0030] Subsequently, a feature tracking algorithm, such as the LK optical flow method or a descriptor-based matching method, is used to establish the correspondence between key points in consecutive image frames, thereby obtaining the motion vectors of these feature points in the image plane.

[0031] Based on these matched pairs of feature points, the system estimates the relative motion of the capsule endoscope between two frames, i.e., a six-degree-of-freedom pose transformation matrix, by solving a non-linear optimization problem. The core of this optimization is to minimize the sum of the re-projection errors of all tracked feature points. The re-projection error refers to the difference between the projected point coordinates in the two-dimensional image plane and the actual observed image point coordinates based on the estimated camera pose. The mathematical formulation of this optimization problem is as follows: where T is the candidate solution of the optimal pose transformation matrix that minimizes the total reprojection error, represents the jth three-dimensional map point defined in the coordinate system of the previous frame (in the system initialization phase, these points can be obtained by triangulating the feature points of the first two frames), represents the actual observed two-dimensional image feature point corresponding to in the current ith frame, and π(·) is the camera projection function that maps a three-dimensional space point (after the current pose transformation) to a two-dimensional image coordinate according to the camera intrinsic parameters, and ρ(·) is a robust kernel function, such as the Huber kernel, which reduces the negative impact of abnormal points caused by mismatching on the overall optimization objective; represents the square of the L2 norm of the calculation vector, i.e., the Euclidean distance. By solving this problem through an iterative optimization algorithm (such as the Gauss-Newton method or the Levenberg-Marquardt method), an accurate estimation of the inter-frame motion of the capsule endoscope can be obtained, and its motion trajectory and a series of sparse three-dimensional map points can be preliminarily recovered.

[0032] Implementation Step 4 (SLAM and Topological Map Construction Based on Graph Optimization) In order to overcome the inevitable cumulative error in visual odometry and construct a globally consistent model of the digestive tract environment, the system introduces a simultaneous localization and mapping method based on graph optimization on the basis of Step 3. The entire system state is represented as a graph structure. The nodes in the graph mainly include two categories: one is the pose node of the capsule endoscope at key time points, usually selecting frames with rich feature information and sufficient viewpoint changes as key frames; the other is the three-dimensional environmental map point node obtained by triangulation. The edges of the graph represent the constraints: one type of edge comes from visual odometry, connecting consecutive pose nodes and constraining from the inter-frame relative motion estimated in Step 3; the other type of edge is the observation constraint, connecting a pose node and the three-dimensional map point node observed at that pose, and its constraint is defined by the reprojection error.

[0033] The global optimization objective of the SLAM system is to minimize the sum of errors caused by all these constraints. When a new key frame is added, local graph optimization is performed.

[0034] In addition, the system integrates a loop detection mechanism that determines whether the current key frame has observed the same scene as a past key frame through techniques such as the bag-of-words model. Once a loop is detected, a new constraint edge is added between the two non-consecutive pose nodes, and this constraint can effectively correct the trajectory drift caused by error accumulation.

[0035] The optimization problem of the entire graph model can be formally represented as minimizing the following objective function: Where ξ represents the set of all keyframe poses, and each pose... Lie algebras are used for representation to facilitate unconstrained optimization; p represents the set of all 3D map points; This is the reprojection error term; Indicates Lie algebra The pose transformation matrix (Lie group element) is converted through exponential mapping. In position Below map points The actual observed pixel coordinates, It is the distance to Maharanobis, among which This is the covariance matrix corresponding to the observation, used to characterize the reliability of different observations. By solving this large-scale sparse least squares problem, the system finally outputs a globally optimized, highly consistent capsule endoscope trajectory and a dense or semi-dense 3D point cloud map of the digestive tract wall, i.e., a topological map.

[0036] Step 5 (Lexus location mapping and result fusion output) The results fusion module receives outputs from the intelligent analysis module (i.e., segmentation masks with timestamps and lesion type labels) and from the spatial localization module (i.e., optimized capsule endoscopy pose sequences and 3D topology maps). This module localizes lesions identified in 2D images to 3D gastrointestinal spatial coordinates. For a lesion region identified in an image frame at a certain timestamp t, the system first finds its corresponding SLAM-optimized capsule endoscopy pose based on the timestamp of that frame. Then, using the camera's projection model and the pixel position of the lesion area in the image (e.g., taking the centroid of its segmentation mask), the direction of a ray in three-dimensional space originating from the camera's optical center and passing through the pixel is calculated using the principle of back projection.

[0037] Since the digestive tract is a hollow structure, the system treats the inner surface of the constructed 3D topological map as a simplified geometric model (such as a tubular model). By calculating the intersection of this ray with the inner surface of the 3D topological map, the approximate 3D spatial coordinates of the lesion on the inner wall of the digestive tract can be determined. Simultaneously, other attributes of the lesion, such as its type obtained in step 2, the calculated pixel area (which can be converted into approximate physical size), and the detection confidence score output by the model, are also extracted. All this information, including lesion type, spatial location, size, confidence score, and associated timestamps, is integrated into a structured data report.

[0038] The report is usually organized in a common data format such as JSON or XML, facilitating storage and subsequent transmission. Finally, through the data interface module, the data report integrating the recognition and positioning results is output to an external display device (such as a physician workstation) for visualization, or stored in a database for subsequent query and review, thereby completing the entire process of automatic lesion recognition and positioning of the capsule endoscope.

[0039] Finally, it should be noted that: first, in the description of the present application, it should be noted that, unless otherwise specified and limited, the terms "mounting", "connection", "connection" should be broadly understood, which can be mechanical connection or electrical connection, or the communication between two elements, or direct connection, "up", "down", "left", "right" and the like are only used to indicate the relative positional relationship, when the absolute position of the described object changes, the relative positional relationship may change; Secondly: the drawings of the disclosed embodiments of the present application only involve the structures involved in the disclosed embodiments, other structures can refer to the usual design, and in the case of no conflict, the same embodiment and different embodiments of the present application can be combined with each other; Finally: the above only describes the preferred embodiments of the present application and is not intended to limit the present application, and any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. An automatic identification and localization system for lesions in capsule endoscopy, characterized in that, It includes an image acquisition module, an intelligent analysis module, and a spatial positioning module; The image acquisition module is used to receive and store a sequence of color or grayscale images continuously captured by the capsule endoscope as it moves in the digestive tract. The image resolution is 512×512 pixels to 2048×2048 pixels and the frame rate is 2 frames / second to 30 frames / second. The image sequence is then transmitted to the intelligent analysis module. The intelligent analysis module integrates a pre-trained deep convolutional neural network model. This model adopts a multi-scale feature extraction and fusion architecture, which includes 3 to 8 downsampling stages and a corresponding number of upsampling stages. Each stage contains 1 to 3 convolutional layers with kernel sizes ranging from 3×3 pixels to 7×7 pixels. It can process the input image sequence frame by frame and output a pixel-level segmentation mask of the lesion region in each frame image. The segmentation mask identifies the contour and type of the lesion. The spatial positioning module is connected to the intelligent analysis module, receives the segmentation mask and the original image sequence, extracts key point features in the image and tracks the motion changes of these features in consecutive frames, combines the approximate motion dynamics model of the capsule endoscope, uses an incremental three-dimensional reconstruction algorithm to estimate the real-time pose of the capsule endoscope in the digestive tract, and constructs a two-dimensional or three-dimensional topological map of the digestive tract wall, thereby mapping the identified lesion area to the specific location coordinates on the map. The system also includes a data interface module for outputting the identification and positioning results to an external display or storage device.

2. The capsule endoscopy lesion automatic identification and localization system according to claim 1, characterized in that, The deep convolutional neural network model is implemented based on an encoder-decoder structure. The encoder uses convolutional and pooling layers to progressively downsample and extract image features, gradually reducing the feature map size to 1 / 2 to 1 / 16 of the original image. The decoder uses deconvolutional layers and skip connections to progressively upsample and restore spatial resolution. Finally, a softmax classifier outputs the class probability for each pixel. The model training is optimized using a total loss function, which consists of cross-entropy loss and Dice loss, and its expression is: in For the total loss, The cross-entropy loss is used to measure the difference between the predicted probability distribution and the true distribution. To address the Dice similarity coefficient loss used to handle foreground and background pixel imbalance, To balance the weighting coefficients of the two losses, the model uses this loss function for end-to-end learning on a dataset containing 100,000 to 500,000 labeled gastrointestinal lesions.

3. The capsule endoscopy lesion automatic identification and localization system according to claim 1, characterized in that, When estimating the pose of the capsule endoscope, the spatial positioning module first extracts 50 to 1000 key points from the image using a FAST corner detector or SIFT descriptor. Then, it tracks the movement of these key points in the sequence using LK optical flow or descriptor matching. Finally, it optimizes the pose transformation matrix by solving a nonlinear least squares problem. The objective of this optimization problem is to minimize the sum of reprojection errors of all feature points, and its mathematical expression is: in Let be the optimal pose transformation matrix to be solved, and T be a candidate value for the pose transformation matrix. Let j be the j-th 3D map point in the world coordinate system. For the observed in the i-th frame image and The corresponding two-dimensional image feature points, π(⋅) is the camera projection function that projects three-dimensional spatial points to two-dimensional image pixel coordinates, ρ(⋅) is the robust kernel function used to reduce the impact of mismatches, and finally the global trajectory and map points are optimized by BundleAdjustment, where the pose estimation includes position and orientation parameters, and the position accuracy is in the range of 1mm to 5mm.

4. The capsule endoscopy lesion automatic identification and localization system according to claim 1, characterized in that, The intelligent analysis module also includes an online adaptive submodule, which can dynamically adjust the parameters of the deep learning model based on the newly input image data. It updates the model weights using real-time feedback annotation information through incremental learning. The model is updated once every 10 to 100 frames of images processed to adapt to changes in the morphology of different patients or lesions.

5. The capsule endoscopy lesion automatic identification and localization system according to claim 1, characterized in that, When constructing a topological map of the digestive tract wall, the spatial positioning module adopts a graph-based SLAM method. This method represents the capsule endoscope pose and map points as graph nodes and minimizes the reprojection error by adding keyframes and loop closure detection constraints. The keyframe selection interval is 5 to 20 frames, and the loop closure detection uses a bag-of-words model for scene recognition, thereby generating a consistent environmental map.

6. The capsule endoscopy lesion automatic identification and localization system according to claim 1, characterized in that, The image acquisition module includes a preprocessing unit that performs standardization processing on the received image sequence, including gamma correction with a gamma correction coefficient of 0.8 to 1.2, histogram equalization, and Gaussian filtering with a standard deviation of 0.5 to 1.5 pixels, to eliminate the effects of illumination changes and noise, and adjusts the image size to a fixed resolution range of 512×512 pixels to 1024×1024 pixels.

7. The capsule endoscopy lesion automatic identification and localization system according to claim 1, characterized in that, The system also includes a result fusion module, which receives the segmentation mask output by the intelligent analysis module and the location coordinates generated by the spatial positioning module, aligns the lesion information with the timestamp, and integrates them into a structured data report. The report is stored in JSON or XML format and includes lesion type, size, location, and confidence fields, with a confidence threshold of 0.7 to 0.

95.

8. The capsule endoscopy lesion automatic identification and localization system according to claim 1, characterized in that, The training process of the deep convolutional neural network model uses data augmentation techniques, including random rotation from -15 degrees to +15 degrees, scaling from 0.8 times to 1.2 times, and color jittering. Color jittering includes brightness adjustment from -10% to +10% and contrast adjustment from -10% to +10% to improve the model's generalization ability. The training data comes from an anonymized endoscopic image database from a multi-center collaboration.

9. The capsule endoscopy lesion automatic identification and localization system according to claim 1, characterized in that, The system also includes a quality control module, which evaluates the credibility of the segmentation results output by the intelligent analysis module. When the confidence of lesion detection in 3 to 10 consecutive frames is lower than 0.6 to 0.8, a re-analysis mechanism is triggered to send the relevant images into the auxiliary verification process.

10. The capsule endoscopy lesion automatic identification and localization system according to claim 1, characterized in that, The spatial positioning module also includes a motion compensation unit, which smooths the motion trajectory of the capsule endoscope using a Kalman filter. The filter state variables include position, velocity, and acceleration, and the observation period is 0.1 to 0.5 seconds, in order to reduce positioning errors caused by gastrointestinal peristalsis.

Images