A method and device for relative positioning of wireless capsule endoscope in vivo

Abstract

A method and device for relative positioning of a wireless capsule endoscope in vivo, which realizes the relative positioning of a wireless capsule endoscope in a deformable tubular environment in vivo by training a visual odometer network based on unsupervised learning, without changing the hardware of the wireless capsule endoscope, and using the trained visual odometer network, the position and depth of each image can be accurately estimated according to a given image sequence, thereby improving the positioning accuracy. The visual information of the camera that comes with the capsule can be directly used to develop a visual odometer without adding additional sensors or modifying the capsule. The visual odometer based on unsupervised learning does not require a complicated manual labeling process, and can also solve the problem of inelastic deformation of the environment through a trust mask matrix. The present invention uses visual information to perform relative positioning of a wireless capsule endoscope in vivo, which is more practical in actual clinical applications than existing methods.

Description

Technical Field

[0001] The invention relates to the positioning of a wireless capsule endoscope, and in particular to a method and a device for relative positioning of a wireless capsule endoscope in vivo. Background Art

[0002] Wireless capsule endoscope has become an important technical means for digestive tract examination, especially in the small intestine, which is an area that ordinary gastroscope and colonoscope cannot examine. Wireless capsule endoscope is a capsule-sized robot. It is equipped with a camera module, an image processing module, and a wireless transmission module. It can take images in the patient's body and transmit them to the outside in real time. Doctors can make diagnoses based on the transmitted images. At the same time, patients do not suffer the pain of gastroscopy and colonoscopy. However, if a doctor finds a lesion in a certain picture, the doctor does not have enough information to determine where the picture was taken in the body. If the position of the capsule when taking the lesion picture is known, the doctor can make a diagnosis and subsequent treatment plan faster. Therefore, it is very important to calculate and determine the position of the capsule. At the same time, the doctor hopes to get the distance between the position of the lesion picture taken by the capsule and an important position in a certain organ, such as how many centimeters the lesion picture is relative to the end of the stomach pylorus. Such a relative distance description is more in line with the doctor's needs. The present invention obtains the relative position of the current capsule relative to the biological anatomical landmark based on the picture taken by the capsule through the visual mileage calculation method based on unsupervised learning.

[0003] The existing wireless capsule endoscope positioning systems mainly include capsule radio signal positioning, capsule magnetic positioning and capsule visual positioning. Capsule radio signal positioning technology uses a sensor array outside the human body to obtain the radio signal strength emitted by the capsule for positioning, but the positioning result of this method has a high error. Capsule magnetic positioning technology is to put a small permanent magnet inside the capsule and place a magnetic sensor array outside the human body to obtain the magnetic field strength to calculate the capsule posture information. This solution has a small error and high accuracy. However, the above two methods obtain six-degree-of-freedom coordinate information in a fixed three-dimensional coordinate system, that is, spatial position information (x, y, z), and spatial rotation information (α, β, γ). Such coordinate position information is "absolute positioning" information, and "relative positioning" information cannot be obtained, which does not meet the needs of doctors. At the same time, some existing capsule visual positioning technologies are mainly used as auxiliary positioning for radio signal positioning, which is also "absolute positioning", or only visual feature extraction and matching are performed. And their feature extraction and matching technologies are all performed between two full capsule endoscope images. Mainstream spot feature extraction and matching technologies such as SIFT, SURF or ORB are mainly used to extract and match features of two continuous images, because the resolution of the images taken by the capsule endoscope is not high and is relatively blurry, and the human intestine is in motion all the time, and the intestine is elastically deformed. Directly matching the two entire images is prone to a large number of mismatches. In addition, there are studies that suggest that a deep learning-based visual algorithm can be used to locate the capsule, which can overcome the elastic deformation of the intestine to a certain extent, but it is mainly tested in a relatively bulging organ such as the pig stomach, not in a tubular environment such as the large intestine and small intestine.

[0004] It should be noted that the information disclosed in the above background technology section is only used for understanding the background of the present application, and therefore may include information that does not constitute prior art known to ordinary technicians in the field. Summary of the invention

[0005] The main purpose of the present invention is to overcome the defects of the above-mentioned background technology and provide a method and device for relative positioning of a wireless capsule endoscope in vivo, so as to improve the positioning accuracy and practicality when performing relative positioning of a wireless capsule endoscope in vivo.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] An in vivo relative positioning method for a wireless capsule endoscope includes a training process and a testing process;

[0008] The training process includes the following steps:

[0009] S1. Inputting a wireless capsule endoscope image sequence into a visual odometer network based on unsupervised learning, wherein the image sequence includes a source image and a target image, and the visual odometer network includes a "pose network" and a "depth network";

[0010] S2, the "pose network" outputs the pose transformation matrix of the target image converted to the source image and outputs the confidence mask matrix of the target image;

[0011] S3, the “depth network” outputs a pixel depth prediction matrix of the target image;

[0012] S4, based on the pose transformation matrix, the pixel depth prediction matrix and the trust mask matrix, reconstructing the source image onto the target image according to the camera model, and calculating the pixel difference between the target image and the reconstructed image, which is called pixel loss;

[0013] S5. Calculate the cross entropy loss of the confidence mask matrix, called mask loss;

[0014] S6, directly estimating the pixel relative depth from the target image, calculating the difference between the depth matrix and the pixel depth prediction matrix, which is called depth loss;

[0015] S7, calculating the smoothness loss of the pixel depth prediction matrix;

[0016] S8, forming a loss function with the pixel loss, the mask loss, the depth loss and the smoothness loss, optimizing the loss function, and updating network parameters until convergence;

[0017] The testing process includes the following steps:

[0018] T1, inputting a sequence of wireless capsule endoscope images, including source images and target images, into the trained visual odometer network;

[0019] T2, the pose transformation matrix of the target image converted to the source image output by the "pose network";

[0020] T3. Calculate the current posture of the wireless capsule endoscope relative to the previous biological anatomical landmark based on the newly calculated posture transformation matrix and the historical posture transformation matrix accumulated from the previous biological anatomical landmark as the starting point.

[0021] Further:

[0022] The picture sequence includes three consecutive pictures, wherein the first picture and the third picture are the source pictures, and the second picture is the target picture.

[0023] The confidence mask matrix is ​​a confidence mask matrix of the target image at four different scales, and the pixel depth prediction matrix is ​​a pixel depth prediction matrix of the target image at four different scales.

[0024] In step S6, the pixel relative depth is directly estimated from the target image according to the Shape from Shading algorithm, which is called the SfS depth matrix, and then the difference between the SfS depth matrix and the pixel depth prediction matrix is ​​calculated.

[0025] In step S7, the sum of the absolute values ​​of the derivatives of each element of the pixel depth prediction matrix is ​​calculated as the smoothness loss of the pixel depth prediction matrix.

[0026] In step S8, the pixel loss, the mask loss, the depth loss and the smoothness loss are added together to form the loss function.

[0027] A training method for a visual odometry network based on unsupervised learning, wherein the visual odometry network is used for in vivo relative positioning of a wireless capsule endoscope, the training method comprising the following steps:

[0028] S1. Inputting a wireless capsule endoscope image sequence into a visual odometer network based on unsupervised learning, wherein the image sequence includes a source image and a target image, and the visual odometer network includes a "pose network" and a "depth network";

[0029] S2, the "pose network" outputs the pose transformation matrix of the target image converted to the source image and outputs the confidence mask matrix of the target image;

[0030] S3, the “depth network” outputs a pixel depth prediction matrix of the target image;

[0031] S4, based on the pose transformation matrix, the pixel depth prediction matrix and the trust mask matrix, reconstructing the source image onto the target image according to the camera model, and calculating the pixel difference between the target image and the reconstructed image, which is called pixel loss;

[0032] S5. Calculate the cross entropy loss of the confidence mask matrix, called mask loss;

[0033] S6, directly estimating the pixel relative depth from the target image, calculating the difference between the depth matrix and the pixel depth prediction matrix, which is called depth loss;

[0034] S7, calculating the smoothness loss of the pixel depth prediction matrix;

[0035] S8. Construct a loss function with the pixel loss, the mask loss, the depth loss and the smoothness loss, optimize the loss function, and update network parameters until convergence.

[0036] A method for in vivo relative positioning of a wireless capsule endoscope, using a visual odometer network trained by the training method, the method comprising the following steps:

[0037] T1, inputting a sequence of wireless capsule endoscope images, including source images and target images, into the trained visual odometer network;

[0038] T2, the pose transformation matrix of the target image converted to the source image output by the "pose network";

[0039] T3. Calculate the current posture of the wireless capsule endoscope relative to the previous biological anatomical landmark based on the newly calculated posture transformation matrix and the historical posture transformation matrix accumulated from the previous biological anatomical landmark as the starting point.

[0040] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the in vivo relative positioning method of a wireless capsule endoscope is implemented.

[0041] An in vivo relative positioning device for a wireless capsule endoscope comprises a processor and a computer-readable storage medium. When the processor executes a computer program stored on the computer-readable storage medium, the in vivo relative positioning method for the wireless capsule endoscope is implemented.

[0042] The present invention has the following beneficial effects:

[0043] The present invention provides a method and device for relative positioning of a wireless capsule endoscope in vivo. By training a visual odometer network based on unsupervised learning, the relative positioning of a wireless capsule endoscope in a deformable tubular environment in vivo can be achieved without changing the hardware of the wireless capsule endoscope. In addition, by using the trained visual odometer network, the position and depth of each image can be accurately estimated according to a given image sequence, thereby improving the positioning accuracy. The present invention can directly use the visual information of the camera that comes with the capsule to develop a visual odometer without adding additional sensors or modifying the capsule. The visual odometer based on unsupervised learning does not require a complicated manual labeling process, and can also solve the problem of inelastic deformation of the environment through a trust mask matrix. The present invention uses visual information to perform relative positioning of a wireless capsule endoscope in vivo, which is more practical in actual clinical applications than existing methods.

[0044] The preferred solution of the present invention adopts depth constraints based on shape from shading (SfS), which can further improve the depth prediction accuracy and positioning accuracy. The pixel depth matrix restriction based on the Shape from Shading theory improves the estimation accuracy of the pixel depth, thereby improving the positioning accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 The present invention is a flowchart of a training process of a method for relative positioning in vivo of a wireless capsule endoscope according to an embodiment of the present invention.

[0046] Figure 2 The present invention is a flowchart of a testing process of a method for relative positioning in vivo of a wireless capsule endoscope according to an embodiment of the present invention.

[0047] Figure 3 A structural diagram of a visual odometer network according to an embodiment of the present invention.

[0048] Figure 4 A diagram showing the relationship between the input and output of a visual odometer network according to an embodiment of the present invention.

[0049] Figure 5 (a) shows a picture taken while the capsule is moving in the pig intestine, and (b) shows the pixel depth prediction matrix estimated after the picture is input into the "deep network". DETAILED DESCRIPTION

[0050] The following is a detailed description of the embodiments of the present invention. It should be emphasized that the following description is only exemplary and is not intended to limit the scope and application of the present invention.

[0051] See also Figures 1 to 3 In some embodiments, a relative positioning method based on unsupervised learning for a wireless capsule endoscope in a deformable tubular environment includes the following typical but non-limiting training and testing processes.

[0052] The training process includes the following steps:

[0053] In the first step, the wireless capsule endoscope image sequence is input into the visual odometry network based on unsupervised learning. The visual odometry network contains two sub-networks, namely "pose network" and "depth network". The image sequence contains three consecutive images, of which the first image and the third image are two source images, respectively referred to as the first source image and the second source image, and the second image is referred to as the target image.

[0054] In the second step, three consecutive images are input into the first sub-network "pose network", which converts the output target image into the pose transformation matrix of the two source images. At the same time, the network will also output the trust mask matrix of the target image at four different scales.

[0055] In the third step, the target image is input into the second sub-network "deep network", which will output the pixel depth prediction matrix of the target image at four different scales.

[0056] In the fourth step, based on the two pose transformation matrices, pixel depth prediction matrix and confidence mask matrix output by the two sub-networks in the previous two steps, the two source images are reconstructed and projected onto the target image according to the camera model. The pixel difference between the target image and the two reconstructed images is calculated, which is called pixel loss.

[0057] The fifth step is to calculate the cross entropy loss of the confidence mask matrix, which is called the mask loss.

[0058] The sixth step is to estimate the pixel relative depth directly from the target image according to the ShapefromShading algorithm, which is called the SfS depth matrix. Next, the difference between the SfS depth matrix and the pixel depth prediction matrix output by the network is calculated, which is called the depth loss.

[0059] The seventh step is to calculate the smoothness of the pixel depth prediction matrix, that is, to calculate the sum of the absolute values ​​of the derivatives of each element of the matrix, which is called the smoothness loss.

[0060] Step 8: Add the above four losses together to form a loss function. Optimize the loss function and update the network parameters until the loss value converges.

[0061] The testing process includes the following steps:

[0062] The first step is to input the wireless capsule endoscope image sequence into the trained visual odometry network. The visual odometry network contains two sub-networks, namely the "pose network" and the "depth network". The image sequence contains three consecutive images. The first and third images are two source images, respectively called the first source image and the second source image. The second image is called the target image.

[0063] In the second step, three consecutive images are input into the first sub-network "pose network", which converts the output target image into the pose transformation matrix of the two source images. At the same time, the network will also output the trust mask matrix of the target image at four different scales.

[0064] In the third step, based on the two newly calculated posture transformation matrices and the historical posture transformation matrix accumulated from the previous biological anatomical landmark as the starting point, the current posture of the capsule relative to the previous biological anatomical landmark can be calculated.

[0065] The present invention can be used for relative positioning of a wireless capsule endoscope during inspection of the human digestive tract. A doctor can transmit an image sequence or video captured by the wireless capsule endoscope to the software, which will analyze the image sequence / video to estimate the relative distance of the capsule endoscope relative to a biological anatomical landmark.

[0066] The following further describes the training and testing process of a specific embodiment of the present invention in conjunction with the accompanying drawings.

[0067] Training process

[0068] The first step is to input the wireless capsule endoscope image sequence into the visual odometry network based on unsupervised learning. Figure 3 As shown, the visual odometry network contains two sub-networks, namely the "pose network" and the "depth network". The image sequence contains three consecutive images, the first image and the third image are two source images, respectively referred to as the first source image and the second source image, and the second image is referred to as the target image.

[0069] The second step is, Figure 3 In (a), three consecutive images I t-1 , I t , I t+1 Input into the first sub-network "pose network", which will output the target image I t Convert to two source images I s , s∈{t-1,t+1}’s pose transformation matrix T1, T2. At the same time, the network will also output the trust mask matrix M of the target image at four different scales s，l , s∈{t-1, t+1}, l∈{0, 1, 2, 3}.

[0070] The third step is Figure 3 In (b), the target image I t Input into the second sub-network "deep network", which will output the pixel depth prediction matrix D of the target image at four different scales t，l , l∈{0, 1, 2, 3}.

[0071] The fourth step is to use the two pose transformation matrices T1, T2 and the pixel depth prediction matrix D output by the two sub-networks in the previous two steps. t，l and the trust mask matrix M s，l , reconstruct the two source images and project them onto the target image according to the camera model. Calculate the pixel difference between the target image and the two reconstructed images, which is called pixel loss.

[0072] Since unsupervised learning is used, the input in the entire training process is only the image sequence, without label input, and the output includes pose transformation, pixel depth prediction, and confidence mask. The relationship between all these inputs and outputs is as follows Figure 4 As shown, these quantities form a self-supervisory constraint between each other.

[0073] The camera model is used as a self-supervisory network. If the target image I t Source Image I s , the reconstruction target of s∈{t-1,t+1}, the projected reconstructed image of the source image is I′ s , then the pixel difference between the target image and the projected reconstructed image should be 0, so the pixel loss is defined as:

[0074]

[0075] I t (p) and M′ s (p) are the values ​​of any corresponding pixel points on the target image and the projected reconstructed image respectively.

[0076] Because the dynamic variability of the environment and the reflection caused by the liquid make some pixels violate the camera model, it is beneficial to ignore these pixels. Therefore, we use the confidence mask matrix output by the "pose network". At different scales, the corresponding mask matrix has the same pixel size as the source image of the corresponding scale. Each element in the confidence mask matrix corresponds to the projected reconstructed image M' s For each pixel in (p), if an element in the trust mask matrix is ​​1, the corresponding pixel in the projected reconstructed image can be trusted, otherwise the corresponding pixel cannot be trusted. So the pixel loss is redefined as:

[0077]

[0078] The fifth step is to calculate the cross entropy loss of the confidence mask matrix, which is called the mask loss.

[0079] If we directly minimize the loss in the previous step, the trust mask matrix M s，l will all become 0, because this will make L pixel Minimum to 0. However, this is a meaningless solution, so s，l Minimize the cross entropy loss, denoted as L mask (M s ).

[0080] The sixth step is to estimate the pixel relative depth directly from the target image according to the ShapefromShading algorithm, which is called the SfS depth matrix. Next, the difference between the SfS depth matrix and the pixel depth prediction matrix output by the network is calculated, which is called the depth loss.

[0081] Figure 5 (a) shows a picture of the capsule moving in the pig intestine. Figure 5 (b) shows the pixel depth prediction matrix estimated after the image is input into the "deep network". It can be observed that the surface of the pig intestine wall without texture in the upper left corner is close to the camera lens and has little motion change, but the depth prediction result mistakenly estimates this part as a distant pixel. In other words, since only distant scenes may not change much over a period of time, the algorithm mistakenly estimates the nearby part with less texture that does not move for a short period of time as a distant scene.

[0082] To solve this problem, we used the Shape from Shading (SfS) theory. The SfS algorithm is used to estimate the relative depth of the scene from a grayscale image. According to the SfS algorithm, the reflection equation of the lambertian surface is:

[0083]

[0084] E (x，y) is the grayscale value at the pixel (x, y). s ,q s , 1) is the position of the light source above the target surface. p and q are the partial derivatives of depth Z in the x and y directions respectively. The above equation can be rewritten as:

[0085]

[0086] For a fixed point (x, y) and a given image E, after first-order Taylor expansion, the above equation can be written as:

[0087]

[0088] All values ​​of the depth matrix are initialized to 0. As the iterative calculation proceeds, the value of f tends to 0, which is the depth Z convergence. Therefore, given the target image I t , the depth matrix based on the SfS algorithm can be calculated and recorded as D SfS (I t ). After normalization, each element D in the SfS depth matrix SfS (p) corresponds to each element D in the pixel depth prediction matrix output by the network tThe difference in (p) should be 0. So the depth loss is defined as follows:

[0089]

[0090] The seventh step is to calculate the smoothness of the pixel depth prediction matrix, that is, to calculate the sum of the absolute values ​​of the derivatives of each element of the matrix, which is called the smoothness loss.

[0091] The smoothness loss is defined as:

[0092]

[0093] Step 8: Add the above four losses together to form a loss function. Optimize the loss function and update the network parameters until the loss value converges.

[0094] The total loss is defined as:

[0095]

[0096] Here λ m ,λ d ,λ s Represent the weights of confidence loss, depth loss and smoothness loss respectively.

[0097] Testing Process

[0098] First, the wireless capsule endoscopy image sequence is input into the trained visual odometry network.

[0099] Second, three consecutive images are input into the trained sub-network "pose network", which converts the output target image into the pose transformation matrix of the two source images.

[0100] Third, based on the two newly calculated pose transformation matrices and the historical pose transformation matrix accumulated from the previous biological anatomical landmark as the starting point, the current pose of the capsule relative to the previous biological anatomical landmark can be calculated.

[0101] The background section of the present invention may contain background information about the problem or environment of the present invention, but does not necessarily describe the prior art. Therefore, the content included in the background section is not an admission by the applicant that the prior art is available.

[0102] The above content is a further detailed description of the present invention in combination with specific / preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, without departing from the concept of the present invention, it can also make several substitutions or modifications to these described embodiments, and these substitutions or modifications should be regarded as belonging to the protection scope of the present invention. In the description of this specification, the description of reference terms "an embodiment", "some embodiments", "preferred embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in combination with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily target the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In the absence of mutual contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples. Although the embodiments of the present invention and their advantages have been described in detail, it should be understood that various changes, substitutions and modifications can be made herein without departing from the scope of protection of the patent application.

Claims

1. A method for relative positioning of a wireless capsule endoscope in vivo, characterized in that: Including training process and testing process; The training process includes the following steps: S1. Inputting a wireless capsule endoscope image sequence into a visual odometer network based on unsupervised learning, wherein the image sequence includes a source image and a target image, and the visual odometer network includes a "pose network" and a "depth network"; S2, the "pose network" outputs the pose transformation matrix of the target image converted to the source image and outputs the confidence mask matrix of the target image; S3, the "depth network" outputs a pixel depth prediction matrix of the target image; S4, based on the pose transformation matrix, the pixel depth prediction matrix and the trust mask matrix, reconstruct the source image and project it onto the target image according to the camera model to obtain a reconstructed image of the source image, and calculate the pixel difference between the target image and the reconstructed image, which is called pixel loss; S5. Calculate the cross entropy loss of the confidence mask matrix, called mask loss; S6, directly estimating the relative depth of pixels from the target image, calculating the difference between the depth matrix and the pixel depth prediction matrix, which is called depth loss; S7, calculating the smoothness loss of the pixel depth prediction matrix; S8, forming a loss function with the pixel loss, the mask loss, the depth loss and the smoothness loss, optimizing the loss function, and updating network parameters until convergence; The testing process includes the following steps: T1, inputting a sequence of wireless capsule endoscope images, including source images and target images, into the trained visual odometer network; T2, the pose transformation matrix of the target image converted to the source image output by the "pose network"; T3. Calculate the current posture of the wireless capsule endoscope relative to the previous biological anatomical landmark based on the newly calculated posture transformation matrix and the historical posture transformation matrix accumulated from the previous biological anatomical landmark as the starting point.

2. The in vivo relative positioning method of a wireless capsule endoscope according to claim 1, characterized in that: The picture sequence includes three consecutive pictures, wherein the first picture and the third picture are the source picture, and the second picture is the target picture.

3. The in vivo relative positioning method of a wireless capsule endoscope according to claim 1 or 2, characterized in that: The confidence mask matrix is ​​a confidence mask matrix of the target image at four different scales, and the pixel depth prediction matrix is ​​a pixel depth prediction matrix of the target image at four different scales.

4. The in vivo relative positioning method of a wireless capsule endoscope according to claim 1 or 2, characterized in that: In step S6, the pixel relative depth is directly estimated from the target image according to the Shape from Shading algorithm, which is called the SfS depth matrix, and then the difference between the SfS depth matrix and the pixel depth prediction matrix is ​​calculated.

5. The in vivo relative positioning method of a wireless capsule endoscope according to claim 1 or 2, characterized in that: In step S7, the sum of the absolute values ​​of the derivatives of each element of the pixel depth prediction matrix is ​​calculated as the smoothness loss of the pixel depth prediction matrix.

6. The in vivo relative positioning method of a wireless capsule endoscope according to claim 1 or 2, characterized in that: In step S8, the pixel loss, the mask loss, the depth loss and the smoothness loss are added together to form the loss function.

7. A training method for a visual odometry network based on unsupervised learning, wherein the visual odometry network is used for in vivo relative positioning of a wireless capsule endoscope, characterized in that: The training method comprises the following steps: S1. Inputting a wireless capsule endoscope image sequence into a visual odometer network based on unsupervised learning, wherein the image sequence includes a source image and a target image, and the visual odometer network includes a "pose network" and a "depth network"; S2, the "pose network" outputs the pose transformation matrix of the target image converted to the source image and outputs the confidence mask matrix of the target image; S3, the "depth network" outputs a pixel depth prediction matrix of the target image; S4, based on the pose transformation matrix, the pixel depth prediction matrix and the trust mask matrix, reconstruct the source image and project it onto the target image according to the camera model to obtain a reconstructed image of the source image, and calculate the pixel difference between the target image and the reconstructed image, which is called pixel loss; S5. Calculate the cross entropy loss of the confidence mask matrix, called mask loss; S6, directly estimating the relative depth of pixels from the target image, calculating the difference between the depth matrix and the pixel depth prediction matrix, which is called depth loss; S7, calculating the smoothness loss of the pixel depth prediction matrix; S8. Construct a loss function with the pixel loss, the mask loss, the depth loss and the smoothness loss, optimize the loss function, and update network parameters until convergence.

8. A method for relative positioning of a wireless capsule endoscope in vivo, characterized in that: Using the visual odometer network trained by the training method according to claim 7, the in vivo relative positioning method comprises the following steps: T1, inputting a sequence of wireless capsule endoscope images, including source images and target images, into the trained visual odometer network; T2, the pose transformation matrix of the target image converted to the source image output by the "pose network"; T3. Calculate the current posture of the wireless capsule endoscope relative to the previous biological anatomical landmark based on the newly calculated posture transformation matrix and the historical posture transformation matrix accumulated from the previous biological anatomical landmark as the starting point.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

10. An in vivo relative positioning device for a wireless capsule endoscope, comprising a processor and a computer-readable storage medium, characterized in that: When the processor executes the computer program stored on the computer-readable storage medium, the method according to any one of claims 1 to 8 is implemented.

Images