Intestinal tract three-dimensional reconstruction method and device based on magnetic positioning technology, equipment and medium

Abstract

The invention discloses an intestinal tract three-dimensional reconstruction method and device based on a magnetic positioning technology, equipment and a medium, and relates to the technical field of intestinal tract three-dimensional reconstruction. Respectively processing the monocular intestinal tract image by adopting a pre-trained depth estimation model and a contour extraction algorithm to correspondingly obtain depth data and intestinal tract three-dimensional point cloud data; processing the magnetic field data by adopting a magnetic positioning technology to obtain endoscope pose data; and along with slow advancing of the endoscope, reconstruction of the three-dimensional model of the intestinal tract is finally realized according to the corresponding depth data, the three-dimensional point cloud data of the intestinal tract and the pose data of the endoscope, so that the timeliness of three-dimensional reconstruction of the intestinal tract is remarkably improved.

Description

Technical Field

[0001] The present application relates to the technical field of three-dimensional intestinal reconstruction, and particularly to a method, device, equipment and medium for three-dimensional intestinal reconstruction based on magnetic positioning technology. Background Technique

[0002] In medical diagnosis, three-dimensional intestinal reconstruction technology is gradually becoming an important auxiliary tool. Through precise three-dimensional reconstruction, doctors can clearly identify the location of lesions, providing a reliable basis for subsequent treatment and surgery. In addition, the three-dimensional model provides more intuitive visual information, helping doctors more comprehensively understand the characteristics of lesions and reducing the risk of misdiagnosis or missed diagnosis. In clinical practice, three-dimensional reconstruction technology also provides support for preoperative simulation and intraoperative navigation. Doctors can conduct detailed planning and simulation before surgery, identify potential difficulties and risks, thereby improving the safety and success rate of surgery.

[0003] Existing three-dimensional intestinal reconstruction technologies such as shape from shading (SFS), shape from focus (SFF), etc. involve complex image processing and calculations, with high computational complexity and serious time consumption. Especially when processing high-resolution images, the reconstruction process may take several hours to complete. This results in limited efficiency of these technologies in clinical applications and cannot meet the needs of real-time or rapid diagnosis. In cases where quick decisions are urgently needed, doctors may not be able to obtain the reconstruction results in time, thus delaying the evaluation and treatment of lesions.

[0004] Therefore, how to develop more efficient processing algorithms, improve the timeliness of three-dimensional intestinal reconstruction, and meet the clinical requirements for immediate diagnosis and treatment planning is a technical problem to be solved by those skilled in the art. Summary of the Invention

[0005] The purpose of the present application is to provide a method, device, equipment and medium for three-dimensional intestinal reconstruction based on magnetic positioning technology, which can improve the timeliness of three-dimensional intestinal reconstruction.

[0006] To achieve the above purpose, the present application provides the following solutions:

[0007] In the first aspect, the present application provides a method for three-dimensional intestinal reconstruction based on magnetic positioning technology, including:

[0008] Real-time acquisition of monocular intestinal images and corresponding magnetic field data, where the monocular intestinal images are images captured by an endoscope during the forward movement in the intestine, and the corresponding magnetic field data refers to the magnetic field data of the permanent magnet at the front end of the endoscope corresponding to each monocular intestinal image;

[0009] Taking each of the monocular intestinal images as input, depth data of each pixel point in each of the monocular intestinal images is obtained by using a pre-trained depth estimation model;

[0010] Processing each of the monocular intestinal images by using a contour extraction algorithm to obtain two-dimensional intestinal contour data;

[0011] According to the depth data and the corresponding two-dimensional intestinal contour data, three-dimensional intestinal point cloud data is obtained;

[0012] Processing the magnetic field data by using magnetic positioning technology to obtain endoscopic pose data corresponding to each of the monocular intestinal images;

[0013] According to all the three-dimensional intestinal point cloud data and the corresponding endoscopic pose data, a three-dimensional model of the entire intestine is reconstructed.

[0014] In a second aspect, the present application provides an intestinal three-dimensional reconstruction device based on magnetic positioning technology, including:

[0015] An endoscope, a permanent magnet, a sensor array board and a controller;

[0016] Both the endoscope and the sensor array board are connected to the controller;

[0017] The endoscope is used to collect monocular intestinal images in real time;

[0018] The permanent magnet is located at the front end of the endoscope;

[0019] The sensor array board is used to collect magnetic field data of the permanent magnet in real time;

[0020] The controller is used to execute the intestinal three-dimensional reconstruction method based on magnetic positioning technology described in the first aspect.

[0021] In a third aspect, the present application provides a computer device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the computer program to implement the intestinal three-dimensional reconstruction method based on magnetic positioning technology described in the first aspect above.

[0022] In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the intestinal three-dimensional reconstruction method based on magnetic positioning technology described in the first aspect above is implemented.

[0023] According to the specific embodiments provided by the present application, the present application has the following technical effects:

[0024] The present application provides a method, apparatus, device and medium for three-dimensional reconstruction of the intestine based on magnetic positioning technology. The method obtains monocular intestine images and magnetic field data of a permanent magnet at the front end of an endoscope in real time, and processes the monocular intestine images respectively by using a pre-trained depth estimation model and a contour extraction algorithm to obtain depth data and three-dimensional point cloud data of the intestine; processes the magnetic field data by using magnetic positioning technology to obtain endoscope pose data; as the endoscope slowly advances, according to the corresponding depth data, three-dimensional point cloud data of the intestine and endoscope pose data, the three-dimensional model of the intestine is finally reconstructed, significantly improving the timeliness of three-dimensional reconstruction of the intestine. Description of the Drawings

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required to be used in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0026] Figure 1 It is an application environment diagram of a method for three-dimensional reconstruction of the intestine based on magnetic positioning technology in Embodiment 1 of the present application;

[0027] Figure 2 It is a schematic flowchart of a method for three-dimensional reconstruction of the intestine based on magnetic positioning technology provided in Embodiment 1 of the present application;

[0028] Figure 3 It is a conceptual diagram of a method for three-dimensional reconstruction of the intestine based on magnetic positioning technology provided in Embodiment 1 of the present application;

[0029] Figure 4 It is a schematic diagram of the mapping process from the world coordinate to the image in Embodiment 1 of the present application;

[0030] Figure 5 It is a schematic structural diagram of the magnetic positioning device in Embodiment 1 of the present application;

[0031] Figure 6 It is a schematic structural diagram of a computer device provided in Embodiment 3 of the present application.

[0032] Reference Signs:

[0033] 1. Magnetic Sensor; 2. Sensor Array Board; 3. Endoscope Lens; 4. Ring Magnet; 5. Magnet Housing; 6. Endoscope Body. Detailed Embodiments

[0034] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0035] To make the above objects, features, and advantages of the present application more obvious and understandable, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0036] The three-dimensional intestinal reconstruction method based on magnetic positioning technology provided by the embodiments of the present application can be applied to an application environment as Figure 1 shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be set up separately, integrated on the server 104, or placed on the cloud or other servers. The terminal 102 can send the to-be-processed monocular intestinal images and magnetic field data to the server 104. After receiving the to-be-processed monocular intestinal images and magnetic field data, the server 104 uses each of the monocular intestinal images as an input, and uses a pre-trained depth estimation model to obtain the depth data of each pixel point in each of the monocular intestinal images; uses a contour extraction algorithm to process each of the monocular intestinal images to obtain two-dimensional intestinal contour data; according to the depth data and the corresponding two-dimensional intestinal contour data, obtains intestinal three-dimensional point cloud data; uses magnetic positioning technology to process the magnetic field data to obtain the endoscopic pose data corresponding to each of the monocular intestinal images; according to all the intestinal three-dimensional point cloud data and the corresponding endoscopic pose data, reconstructs a three-dimensional model of the entire intestine. The server 104 can feedback the obtained three-dimensional model of the entire intestine to the terminal 102. In addition, in some embodiments, the three-dimensional intestinal reconstruction method based on magnetic positioning technology can also be implemented separately by the server 104 or the terminal 102. For example, the terminal 102 can directly process the to-be-processed monocular intestinal images and magnetic field data using the three-dimensional intestinal reconstruction method based on magnetic positioning technology, or the server 104 can obtain the to-be-processed monocular intestinal images and magnetic field data from the data storage system and process them using the three-dimensional intestinal reconstruction method based on magnetic positioning technology.

[0037] Among them, the terminal 102 can be, but is not limited to, various desktop computers, laptop computers, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. The portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented by an independent server or a server cluster composed of multiple servers, and can also be a cloud server.

[0038] In an exemplary embodiment, as Figure 2 shown, a three-dimensional intestinal reconstruction method based on magnetic positioning technology is provided. This method is executed by a computer device, and specifically can be executed alone by a computer device such as a terminal or a server, or can be jointly executed by a terminal and a server. In the embodiment of the present application, taking this method applied to Figure 1 the server 104 in it as an example for illustration, it includes the following steps 201 to step 206.

[0039] Among them:

[0040] Step 201, obtain a monocular intestinal image and corresponding magnetic field data in real time. Among them, the monocular intestinal image is an image captured by an endoscope during the forward movement in the intestine, and the corresponding magnetic field data refers to the magnetic field data of the permanent magnet at the front end of the endoscope corresponding to each monocular intestinal image.

[0041] Step 202, take each monocular intestinal image as an input, and use a pre-trained depth estimation model to obtain the depth data of each pixel point in each monocular intestinal image. In this embodiment, the FastDepth model is selected as the depth estimation model.

[0042] Step 203, process each monocular intestinal image by using a contour extraction algorithm to obtain two-dimensional intestinal contour data;

[0043] Step 203-1, perform denoising processing on each monocular intestinal image to obtain a denoised monocular intestinal image.

[0044] Step 203-1, perform binarization processing on the denoised monocular intestinal image to obtain a binarized image.

[0045] Step 203-1, perform edge detection on the binarized image, and take the points on the largest contour as two-dimensional intestinal contour data.

[0046] Step 204, according to the depth data and the corresponding two-dimensional intestinal contour data, obtain intestinal three-dimensional point cloud data.

[0047] Step 205, process the magnetic field data by using magnetic positioning technology to obtain the endoscopic pose data corresponding to each monocular intestinal image.

[0048] Step 205-1: Calculate the magnetic moment of the permanent magnet according to the magnetic field data corresponding to each monocular intestinal image.

[0049] Step 205-2: Calculate the pose data of the permanent magnet according to the magnetic moment.

[0050] Step 205-3: Calculate the pose data of the endoscope according to the pose data of the permanent magnet.

[0051] Step 206: Reconstruct the three-dimensional model of the entire intestine according to all the intestinal three-dimensional point cloud data and the corresponding endoscope pose data.

[0052] As an alternative implementation, the selection process of the permanent magnet in this embodiment specifically includes:

[0053] 1) Establish different magnetic field calculation models based on the Biot-Savart law, where the magnetic field calculation models are used to calculate the magnetic field data under different permanent magnet configurations, and the magnetic field data includes the magnetic field magnitude and the magnetic field direction.

[0054] 2) Simulate the different permanent magnets to obtain simulated magnetic field data.

[0055] 3) Calculate the difference between the simulated magnetic field data and the magnetic field data calculated by the corresponding magnetic field calculation model.

[0056] 4) Select the permanent magnet corresponding to the magnetic field calculation model with the smallest difference as the optimal permanent magnet.

[0057] To make those skilled in the art more clearly understand the technical concept of the above-mentioned three-dimensional intestinal reconstruction method based on magnetic positioning technology in this embodiment, the following is specifically explained in combination with Figure 3 for specific illustration.

[0058] As Figure 3 shown in the conceptual diagram of the three-dimensional intestinal reconstruction method based on magnetic positioning technology, this method includes the following steps:

[0059] Step S1: Process the monocular intestinal images captured from the endoscope through a deep learning method to estimate the depth information. In this embodiment, a self-supervised learning method is used to train a convolutional neural network (CNN) to obtain a pre-trained depth estimation model, so as to predict the depth information of the monocular intestinal images.

[0060] Specifically, in this embodiment, the monocular depth estimation model FastDepth is used for depth prediction. This model has a high inference speed to improve the real-time performance of depth estimation. The encoder part uses the lightweight convolutional neural network MobileNet, and the decoder restores the feature map to a depth map of the same size as the input image through a series of deconvolution and upsampling operations. During the training process, a self-supervised learning method is adopted, in which the difference between the output of the model and the true depth label is measured by a loss function.

[0061] Step S2: Use a contour extraction algorithm to perform a series of image processing operations on the monocular intestinal image captured by the endoscope, including Gaussian blur, threshold segmentation, and contour detection, etc., to extract the contour of the intestine (i.e., two-dimensional intestinal contour data). Specifically, it includes the following sub-steps:

[0062] S21: First, apply Gaussian blur to the monocular intestinal image to reduce the noise in the image and enhance the stability of subsequent processing.

[0063] S22: Perform binarization on the Gaussian-blurred image. Set a threshold (obtained through multiple experiments) to convert the image into a black-and-white image, where the pixel points above the threshold are set to white, and the pixel points below the threshold are set to black, and then invert it to highlight the region of interest.

[0064] S23: Detect the contour in the binarized image. Among all the detected contours, find the largest contour, which usually corresponds to the main region of the intestine, and save the points of the largest contour as a two-dimensional array for subsequent processing.

[0065] Step S3: Convert the extracted two-dimensional intestinal contour data into a three-dimensional point cloud. By combining the depth information, each point on the contour is combined with its corresponding depth value to generate a point cloud representation in three-dimensional space (i.e., intestinal three-dimensional point cloud data).

[0066] Specifically, in step S2, a two-dimensional array of the intestinal contour is obtained. Combining the depth information of the corresponding points obtained in step S1, a point cloud in three-dimensional space can be generated. Figure 4 The following figure shows a schematic diagram of the mapping process from the world coordinate to the image. Assume a point P = [x w , y w , z w T in the world coordinate system. Through the camera model, it can be projected onto the image plane of the camera, and the corresponding point in the camera coordinate system is P c = [x c , y c , z c TAccording to the projection relationship, for the two-dimensional coordinates (u, v) of each point on the contour to the point cloud X point_cloud =(x point_cloud y point_cloud , z point_cloud ), there is the following transformation formula:

[0067]

[0068] where z c is the depth value of this point, f x , y , u 0 , v 0 are camera calibration data and can be obtained through camera calibration.

[0069] Step S4: Adopt magnetic positioning technology. By establishing a magnetic field model of the permanent magnet and using a sensor array board to detect the magnetic field, accurately calculate the magnetic moment of the annular magnet at the front end of the endoscope. Use a magnetic tracking system to obtain the accurate position and attitude of the annular small magnet at the front end of the endoscope in real time (i.e., the permanent magnet pose data), so as to realize the real-time tracking of the position and attitude of the endoscope (i.e., the endoscope pose data).

[0070] Figure 5 FIG. shows the magnetic positioning device adopted in this embodiment. The front end of the endoscope is sleeved with an annular magnet for generating a magnetic field for subsequent pose calculation. At the same time, the outside of the annular magnet is wrapped with an annular shell to protect the magnet. The sensor array board is composed of a series of high-precision magnetic sensors and is arranged in a 4×4 matrix form to provide a comprehensive measurement of the magnetic field. When the endoscope approaches the sensor array board, the annular magnet will generate a specific magnetic field signal, and the sensor array board calculates the position and attitude of the endoscope by capturing the magnetic induction intensity data of the annular magnet. The magnetic positioning device includes a magnetic sensor 1, a sensor array board 2 composed of 4×4 magnetic sensors, an endoscope lens 3, an annular magnet 4 (i.e., a permanent magnet), a magnet housing 5 (i.e., a shell), and an endoscope body 6. The wall thickness of the annular magnet is 0.5 mm, which basically does not increase the volume. The thickness of the shell is 0.3 mm and is made of polyetheretherketone (PEEK) material, which has good biocompatibility.

[0071] Adopt a magnetic positioning system to perform pose calculation on the annular magnet sleeved on the front end of the endoscope, specifically including the following sub-steps:

[0072] S41: Through the magnetic field calculation model established based on the Biot-Savart law and COMSOL software simulation, establish a magnetic field calculation model suitable for magnetic tracking of the endoscope (i.e., the optimal magnetic field calculation model).

[0073] Specifically, first, several suitable magnetic field calculation models are established based on the Biot-Savart law. Taking the size and dimensions of the selected toroidal magnet as an example, the magnetic field magnitude and direction of each toroidal magnet are calculated using the established magnetic field calculation models. At the same time, the selected toroidal magnet is simulated using COMSOL software. By comparing the magnetic field data calculated by the magnetic field calculation models with the same size and shape with the simulated magnetic field data, the magnetic field calculation model with the smallest error is selected, and the magnetic field data of toroidal magnets with different sizes and shapes are compared according to the selected magnetic field calculation model to select the most suitable toroidal magnet for the endoscope, and a magnetic field calculation model suitable for magnetic tracking of the micro intestinal robot is established.

[0074] S42. Derive the permanent magnet magnetic moment calculation model based on the established magnetic field calculation model and calculate the magnetic moment data.

[0075] Specifically, first, the magnitude and direction of the interfering magnetic field in space are detected, and then the toroidal magnet to be measured is placed in the sensing array board to detect the combined magnetic field of the interfering magnetic field and the magnetic field of the toroidal magnet. The error interference of the interfering magnetic field is eliminated from the detected data, and the magnetic moment magnitude of the toroidal magnet is calculated by solving the established permanent magnet magnetic moment calculation model. Then, mutual verification and compensation are performed according to the values measured by the individual chips in the sensing array board to obtain the magnetic moment data suitable for the magnetic tracking system.

[0076] S43. Perform pose calculation of the toroidal magnet and the endoscope.

[0077] Specifically, the sensor array board after interference difference compensation collects the magnetic induction intensity data of the toroidal magnet and sends it to the computer. The computer performs a linear calculation once to estimate the approximate position and pose of the toroidal magnet. The estimation result is transmitted in real time and used as the initial value of the LM algorithm. The LM algorithm is executed, and after multiple iterations, the estimated result of the pose data of the toroidal magnet approaches the true value. Since the tracking and positioning are updated in real time, the solution result of the previous time can be used as the initial value of the next solution, and the LM algorithm can be used alone for subsequent tracking until the tracking task ends.

[0078] The position and pose of the toroidal magnet obtained here are the relative position and pose of the toroidal magnet with respect to the sensor array board. Since the position of the sensor array board in the world coordinate system is known, the relative position of the toroidal magnet can be converted from the sensor array board coordinate system to the world coordinate system, and coordinate conversion is performed using the known position of the sensor array board to finally obtain the absolute position and pose of the toroidal magnet in the world coordinate system, which are the pose data of the endoscope.

[0079] Step S5. Perform coordinate transformation on the local point cloud to reflect the position and pose of the intestine in the actual space. As the endoscope moves forward slowly, data is continuously collected and the point cloud is updated, and finally, three-dimensional reconstruction of the entire intestine is achieved.

[0080] Convert the endoscopic pose data obtained in step S4 to the world coordinate system, and convert the local point cloud to the world coordinate system. The formula is:

[0081] X world = M · X point_ccloud ;

[0082] where M is the endoscopic pose data. As the endoscope moves forward slowly, data is continuously collected and the point cloud is updated, ultimately achieving the three-dimensional reconstruction of the entire intestine.

[0083] The three-dimensional intestinal reconstruction method based on magnetic positioning technology provided by the present invention can reconstruct a three-dimensional intestinal model at a relatively fast speed, improving the speed and efficiency of three-dimensional reconstruction. This method extracts depth information and intestinal contour information from monocular intestinal images captured by the endoscope, and combines magnetic positioning technology to achieve rapid intestinal reconstruction. Compared with existing technologies, this embodiment significantly improves the timeliness of three-dimensional reconstruction, and can reconstruct a three-dimensional intestinal model in a short time, which is of great significance for improving the real-time performance of clinical diagnosis and the formulation of treatment plans. In addition, since the magnetic positioning technology directly solves the absolute pose of the endoscope at the current moment, the generation of cumulative errors is avoided, thereby improving the accuracy of the reconstruction result.

[0084] In this embodiment, deep learning uses FastDepth to perform depth prediction on monocular intestinal images. This model has a high inference speed and thus good real-time performance; the magnetic positioning technology can quickly provide the absolute pose information of the endoscope by tracking the pose of the endoscope in real time, enabling three-dimensional reconstruction to be carried out synchronously during the movement of the endoscope to ensure that the model is consistent with the actual intestinal position. In addition, since the magnetic positioning technology directly solves the absolute pose of the endoscope at the current moment, the generation of cumulative errors is avoided, improving the accuracy of the reconstruction result; the contour extraction algorithm uses an efficient image processing algorithm to quickly identify and extract the intestinal boundary. The combination of the three enables this embodiment to achieve rapid three-dimensional reconstruction, improving the speed and efficiency of three-dimensional reconstruction.

[0085] This application also provides an application scenario that applies the above-mentioned three-dimensional intestinal reconstruction method based on magnetic positioning technology. Specifically: The three-dimensional intestinal reconstruction method based on magnetic positioning technology provided by this embodiment can be applied in the scenario of medical image diagnosis. This scenario includes an intestinal data acquisition link, an image processing and reconstruction link, and a clinical analysis and evaluation link. The three-dimensional intestinal reconstruction method based on magnetic positioning technology provided by this embodiment is a key technology in the image processing and reconstruction link. This method can efficiently and accurately reconstruct the three-dimensional structure of the intestine, helping doctors accurately evaluate the lesion location and degree of the intestine, thereby providing important support in the diagnosis, surgical planning, and postoperative evaluation of intestinal diseases.

[0086] Example 2

[0087] This embodiment also provides an intestinal three-dimensional reconstruction device based on magnetic positioning technology, including:

[0088] An endoscope, a permanent magnet, a sensor array board, and a controller.

[0089] Both the endoscope and the sensor array board are connected to the controller.

[0090] The endoscope is used to collect monocular intestinal images in real time.

[0091] The permanent magnet is located at the front end of the endoscope, and the permanent magnet is also wrapped with a housing

[0092] The sensor array board is used to collect the magnetic field data of the permanent magnet in real time.

[0093] The controller is used to execute the intestinal three-dimensional reconstruction method based on magnetic positioning technology described in Embodiment 1.

[0094] Based on the same inventive concept, the embodiment of the present application also provides an intestinal three-dimensional reconstruction device based on magnetic positioning technology for implementing the intestinal three-dimensional reconstruction method based on magnetic positioning technology involved above. The solution provided by this device to solve the problem is similar to the solution described in the above method. The specific limitations of this device can refer to the limitations of the intestinal three-dimensional reconstruction method based on magnetic positioning technology in Embodiment 1, and will not be elaborated here.

[0095] This embodiment provides an intestinal three-dimensional reconstruction device based on magnetic positioning technology. This device collects monocular intestinal images through an endoscope, and uses deep learning technology to perform depth estimation on the monocular intestinal images. Extract the contour of the monocular intestinal image, and generate intestinal three-dimensional point cloud data in combination with the depth information. Then, the position and posture of the endoscope in the intestine are obtained in real time through magnetic positioning technology. Using the obtained position and posture of the endoscope, the local point cloud is gradually transformed into the world coordinate system. As the endoscope moves forward slowly, the point cloud is continuously updated, and finally the rapid three-dimensional reconstruction of the intestine is realized, significantly improving the timeliness of the intestinal three-dimensional reconstruction.

[0096] Example 3

[0097] This embodiment provides a computer device, which can be a server or a terminal, and its internal structure diagram can be as Figure 6As shown in the figure. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data in the three-dimensional intestinal reconstruction method based on magnetic positioning technology. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements the three-dimensional intestinal reconstruction method based on magnetic positioning technology in Embodiment 1.

[0098] Those skilled in the art can understand that Figure 6 the structure shown in the figure is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements. In an exemplary embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the steps in the above method embodiments are implemented.

[0099] Embodiment 4

[0100] This embodiment provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, it implements the three-dimensional intestinal reconstruction method based on magnetic positioning technology in Embodiment 1.

[0101] Embodiment 5

[0102] This embodiment provides a computer program product including a computer program, and when the computer program is executed by a processor, it implements the three-dimensional intestinal reconstruction method based on magnetic positioning technology in Embodiment 1.

[0103] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0104] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the various embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memory can include Read-Only Memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0105] The databases involved in the various embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the various embodiments provided in the present application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0106] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0107] Specific examples are used in this article to elaborate on the principles and implementation manners of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A three-dimensional reconstruction method of the intestine based on magnetic positioning technology, characterized in that, the three-dimensional reconstruction method of the intestine based on magnetic positioning technology includes: Obtaining a monocular intestine image and corresponding magnetic field data in real time, where the monocular intestine image is an image captured by an endoscope during its advancement in the intestine, and the corresponding magnetic field data refers to the magnetic field data of the permanent magnet at the front end of the endoscope corresponding to each monocular intestine image; Taking each monocular intestine image as an input, and using a pre-trained depth estimation model to obtain the depth data of each pixel point in each monocular intestine image; Processing each monocular intestine image by using a contour extraction algorithm to obtain two-dimensional intestine contour data; Obtaining three-dimensional intestine point cloud data according to the depth data and the corresponding two-dimensional intestine contour data; Processing the magnetic field data by using magnetic positioning technology to obtain the endoscope pose data corresponding to each monocular intestine image; Reconstructing a three-dimensional model of the entire intestine according to all the three-dimensional intestine point cloud data and the corresponding endoscope pose data.

2. The three-dimensional reconstruction method of the intestine based on magnetic positioning technology according to claim 1, characterized in that, Processing the magnetic field data by using magnetic positioning technology to obtain the endoscope pose data corresponding to each monocular intestine image, specifically including: Calculating the magnetic moment of the permanent magnet according to the magnetic field data corresponding to each monocular intestine image; Calculating the permanent magnet pose data according to the magnetic moment; Calculating the endoscope pose data according to the permanent magnet pose data.

3. The three-dimensional reconstruction method of the intestine based on magnetic positioning technology according to claim 2, characterized in that, The process of selecting the permanent magnet specifically includes: Establishing different magnetic field calculation models based on the Biot-Savart law, where the magnetic field calculation model is used to calculate the magnetic field data under different permanent magnet configurations, and the magnetic field data includes the magnetic field magnitude and the magnetic field direction; Simulating the different permanent magnets to obtain simulated magnetic field data; Calculating the difference between the simulated magnetic field data and the magnetic field data calculated by the corresponding magnetic field calculation model; Selecting the permanent magnet corresponding to the magnetic field calculation model with the smallest difference as the optimal permanent magnet.

4. The three-dimensional reconstruction method of the intestine based on magnetic positioning technology according to claim 1, characterized in that, The depth estimation model is the FastDepth model.

5. The three-dimensional reconstruction method of the intestine based on magnetic positioning technology according to claim 1, characterized in that, Processing each monocular intestine image by using a contour extraction algorithm to obtain two-dimensional intestine contour data, specifically including: Performing denoising processing on each monocular intestine image to obtain a denoised monocular intestine image; Performing binarization processing on the denoised monocular intestine image to obtain a binarized image; Performing edge detection on the binarized image, and taking the points on the largest contour as two-dimensional intestine contour data.

6. The three-dimensional reconstruction method of the intestine based on magnetic positioning technology according to claim 1, characterized in that, The calculation formula of the three-dimensional intestine point cloud data is: where X point_cloud =(x point_cloud , y point_cloud , z point_cloud ) is the three-dimensional coordinate of the three-dimensional point cloud data of the intestine, z c is the depth data of the pixel point, (u, v) is the two-dimensional coordinate of the corresponding two-dimensional intestinal contour data, f x , f y , u 0 , v 0 are camera calibration data.

7. A three-dimensional reconstruction device of the intestine based on magnetic positioning technology, characterized in that, The intestinal three-dimensional reconstruction device based on magnetic positioning technology includes: an endoscope, a permanent magnet, a sensor array board, and a controller; both the endoscope and the sensor array board are connected to the controller; the endoscope is used to collect monocular intestinal images in real time; the permanent magnet is located at the front end of the endoscope; the sensor array board is used to collect the magnetic field data of the permanent magnet in real time; the controller is used to execute the intestinal three-dimensional reconstruction method based on magnetic positioning technology according to any one of claims 1-6.

8. The intestinal three-dimensional reconstruction device based on magnetic positioning technology according to claim 7, wherein, the permanent magnet is further wrapped with a housing.

9. A computer device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the computer program to implement the intestinal three-dimensional reconstruction method based on magnetic positioning technology according to any one of claims 1-6.

10. A computer-readable storage medium, on which a computer program is stored, wherein, when the computer program is executed by a processor, it implements the intestinal three-dimensional reconstruction method based on magnetic positioning technology according to any one of claims 1-6.

Images