Medical equipment, stomach three-dimensional model reconstruction method and device

Abstract

The present invention discloses a medical device, a method for reconstructing a three-dimensional stomach model, and an apparatus thereof. The method comprises the following steps: obtaining original point cloud information, which is a set of multiple original coordinates, wherein the original coordinates are the coordinates of sampling points located on the inner surface of the stomach; establishing a stomach coordinate system based on the original point cloud information, and calculating transformed point cloud information in the stomach coordinate system; resampling the transformed point cloud information to generate stomach point cloud information, and performing three-dimensional surface reconstruction based on the stomach point cloud information to generate a three-dimensional stomach model. Using this three-dimensional model reconstruction method, a three-dimensional stomach model can be established, solving the problem of capsule-taken photos being unclear and unintuitive. The three-dimensional stomach model is generated based on the capsule's positioning information within the stomach. The operation method is simple, the stomach is presented intuitively, and the stomach can be observed from any angle, assisting doctors in clinical diagnosis.

Description

Technical Field

[0001] The present invention relates to medical equipment, and in particular to a method and device for reconstructing a three-dimensional stomach model. Background Art

[0002] Capsule endoscopy allows for examination of the digestive tract. Compared to other digestive tract imaging technologies, such as CT imaging, capsule endoscopy is simpler to use and does not pose a significant risk of radiation exposure. Typically, the capsule endoscopy's camera module captures multiple images, allowing doctors to examine the health of various sections of the patient's digestive tract.

[0003] However, the images obtained by capsule endoscopy may be unclear, the specific location of the corresponding stomach may not be clear, and multiple images may still not accurately reflect the overall picture of the stomach. The overall structure of the stomach is not intuitive and accurate enough, and doctors need to rely on experience to make judgments, which increases the threshold for the use of capsule endoscopy. Summary of the Invention

[0004] In order to solve the problem that the images in the prior art are not clear and intuitive enough, the purpose of the present invention is to provide a medical device, a method for reconstructing a three-dimensional stomach model and a device thereof.

[0005] To achieve the above-mentioned object of the invention, one embodiment of the present invention provides a method for reconstructing a three-dimensional stomach model, comprising the steps of:

[0006] Acquiring original point cloud information, where the original point cloud information is a set of multiple original coordinates, wherein the original coordinates are coordinates of sampling points located on the inner surface of the stomach;

[0007] Establishing a stomach coordinate system based on the original point cloud information, and calculating transformed point cloud information in the stomach coordinate system;

[0008] resampling the converted point cloud information to generate stomach point cloud information, wherein the resampling is used to make the stomach point cloud information smooth and evenly distributed, and at least partially remove point data located inside the inner surface of the stomach;

[0009] Three-dimensional surface reconstruction is performed based on the stomach point cloud information to generate a three-dimensional stomach model.

[0010] As a further improvement of the present invention, the step of "establishing a stomach coordinate system based on the original point cloud information, and calculating the transformed point cloud information in the stomach coordinate system" further includes:

[0011] Get standard model point cloud information;

[0012] Performing an affine transformation on the original point cloud information according to the standard model point cloud information to generate adapted point cloud information;

[0013] A stomach coordinate system based on the adapted point cloud information is established, and transformed point cloud information of the adapted point cloud information in the stomach coordinate system is calculated.

[0014] As a further improvement of the present invention, the step of "establishing a stomach coordinate system based on the original point cloud information" includes:

[0015] Calculate the average values ​​of all the original coordinates on the X, Y, and Z axes to obtain average coordinates;

[0016] Determine the average value coordinates as the origin coordinates of the stomach coordinate system;

[0017] or,

[0018] Filter the minimum values ​​of all the original coordinates on the X, Y, and Z axes to obtain the minimum coordinates;

[0019] Determining the minimum value coordinates as the origin coordinates of the stomach coordinate system;

[0020] or,

[0021] Filter the maximum values ​​of all the original coordinates on the X, Y, and Z axes to obtain the maximum value coordinates;

[0022] The maximum value coordinates are determined as the origin coordinates of the stomach coordinate system.

[0023] As a further improvement of the present invention, the converted point cloud information includes a set of multiple converted coordinates;

[0024] The step of “calculating the transformed point cloud information in the stomach coordinate system” includes:

[0025] Calculating the offset of the original coordinates relative to the origin of the stomach coordinate system in the direction of each coordinate axis;

[0026] Each transformed coordinate corresponds to an offset from each original coordinate.

[0027] As a further improvement of the present invention, the resampling further includes:

[0028] De-noising the converted point cloud information to obtain de-noised point cloud information, wherein the converted point cloud information is a set of multiple converted coordinates, and the de-noised point cloud information is a set of multiple de-noised coordinates;

[0029] Applying a voxel filtering algorithm, a uniform sampling algorithm, or a least squares method to the denoised point cloud information to generate uniform point cloud information;

[0030] A concave hull algorithm is used on the uniform point cloud information to generate stomach point cloud information.

[0031] As a further improvement of the present invention, the denoising process includes:

[0032] Calculating the mean and standard deviation of all the transformed coordinates;

[0033] Calculate the average distance between each transformed coordinate and a preset number of adjacent transformed coordinates;

[0034] Calculate an offset corresponding to each transformed coordinate, where the offset is the absolute value of the difference between the mean and the average distance;

[0035] The transformed coordinates whose offset is less than or equal to the offset extreme value are set as the denoised coordinates, wherein the offset extreme value is the product of the standard deviation and the control factor.

[0036] As a further improvement of the present invention, when the voxel filtering algorithm is adopted, the steps include:

[0037] Filter the minimum coordinate and the maximum coordinate among all the denoised coordinates;

[0038] Calculating a step size of a voxel grid according to the minimum coordinate and the maximum coordinate;

[0039] Calculate the centroid coordinates of each voxel grid and traverse all voxel grids;

[0040] The centroid coordinates of all voxel grids are set as the uniform point cloud information.

[0041] As a further improvement of the present invention, the step of "reconstructing a three-dimensional surface based on the stomach point cloud information to generate a three-dimensional stomach model" includes:

[0042] Performing three-dimensional surface reconstruction on the stomach point cloud information using a three-dimensional point cloud reconstruction algorithm to generate a three-dimensional stomach model;

[0043] The three-dimensional point cloud reconstruction algorithm is a marching cube algorithm, a greedy projected triangle algorithm or a Poisson reconstruction algorithm.

[0044] To achieve one of the above-mentioned objectives, an embodiment of the present invention provides a device for reconstructing a three-dimensional stomach model, comprising:

[0045] an acquisition module, configured to acquire original point cloud information, wherein the original point cloud information is a set of multiple original coordinates, wherein the original coordinates are coordinates of sampling points located on the inner surface of the stomach;

[0046] a coordinate calculation module, configured to establish a stomach coordinate system based on the original point cloud information and calculate transformed point cloud information in the stomach coordinate system;

[0047] a resampling module, configured to resample the converted point cloud information to generate stomach point cloud information, wherein the resampling is configured to smooth and evenly distribute the stomach point cloud information and at least partially remove point data located inside the inner surface of the stomach;

[0048] The three-dimensional reconstruction module is used to perform three-dimensional surface reconstruction based on the stomach point cloud information to generate a three-dimensional model of the stomach.

[0049] To achieve one of the above-mentioned objectives, an embodiment of the present invention provides an electronic device, including:

[0050] a storage module storing a computer program;

[0051] The processing module can implement the steps in the above-mentioned stomach three-dimensional model reconstruction method when executing the computer program.

[0052] To achieve one of the above-mentioned objects of the invention, an embodiment of the present invention provides a readable storage medium storing a computer program, which, when executed by a processing module, can implement the steps in the above-mentioned method for reconstructing a three-dimensional stomach model.

[0053] Compared with the existing technology, the present invention has the following beneficial effects: the three-dimensional model reconstruction method can be used to establish a three-dimensional model of the stomach, avoiding the radiation problem of CT imaging and solving the problem that the capsule photos are not clear and intuitive enough. The three-dimensional model of the stomach is generated by the positioning information of the capsule in the stomach. The operation method is simple, the presentation of the stomach is intuitive, and the stomach can be observed from any angle, assisting doctors in clinical diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 is a flow chart of a method for reconstructing a three-dimensional stomach model according to an embodiment of the present invention;

[0055] Figure 2 This is a schematic structural diagram of a magnetically controlled capsule system according to an embodiment of the present invention applied to a human body;

[0056] Figure 3 is a schematic diagram of the calculation process of a method for reconstructing a three-dimensional stomach model according to an embodiment of the present invention;

[0057] Figure 4 This is a structural block diagram of a stomach three-dimensional model reconstruction device according to an embodiment of the present invention;

[0058] Figure 5 is a schematic diagram of a module of a magnetically controlled capsule system according to an embodiment of the present invention;

[0059] Among them, 1000, magnetic control capsule system; 100, magnetic control system; 200, capsule; 300, human body; 301, stomach; 10, control magnet; 20, signal transmission module; 30, storage module; 40, processing module; 50, communication bus; 60, magnetic component; 70, magnetic sensor; 80, signal transmission module. DETAILED DESCRIPTION

[0060] The present invention will be described in detail below with reference to the specific embodiments shown in the accompanying drawings. However, these embodiments do not limit the present invention, and any structural, methodological, or functional changes made by those skilled in the art based on these embodiments are all within the scope of protection of the present invention.

[0061] An embodiment of the present invention provides a medical device, a method for reconstructing a three-dimensional stomach model, and an apparatus thereof, which generates a three-dimensional stomach model based on the positioning information of a capsule in the stomach. The operation method is simple and the presentation method is intuitive.

[0062] The medical device of this embodiment can be a magnetically controlled capsule system 1000, a computing device such as a desktop computer, a laptop, a PDA, or a cloud server. The magnetically controlled capsule system 1000 can directly generate a three-dimensional model after obtaining the positioning information of the capsule 200. Alternatively, the magnetically controlled capsule system 1000 can obtain the positioning information of the capsule 200 and then transmit the data to another computing device, which then generates the three-dimensional model.

[0063] The magnetically controlled capsule system 1000 includes a magnetic control system 100, a capsule 200 and a capsule positioning system. The capsule positioning system can obtain the position and posture of the capsule 200. The magnetic control system 100 includes a control magnet 10. The control magnet 10 includes a magnetic source for emitting a magnetic field. The capsule 200 is a capsule endoscope and can be used to take pictures inside the human body 300. There is a magnetic part 60 inside the capsule 200. The magnetic source exerts force on the magnetic part 60 to realize the control of the position and posture of the capsule 200 by the magnetic control system 100. In addition, there is a magnetic sensor 70 inside the capsule 200. The magnetic control system 100 can obtain the position and posture of the capsule 200 in space.

[0064] Figure 2 This is a schematic diagram of a magnetically controlled capsule system 1000 according to one embodiment of the present application being applied to a human body 300. Capsule 200 is located within human body 300, specifically within the stomach 301 of the digestive tract. Human body 300 lies flat on a bed, while magnetic control system 100 is positioned externally. During an examination, the magnetic field emitted by magnet 10 controls the movement of capsule 200 within human body 300, specifically within the stomach 301.

[0065] The 3D model reconstruction method of this embodiment primarily focuses on the 3D reconstruction of the stomach 301, specifically the internal space of the stomach, excluding the outer wall. This includes the space at various locations, including the cardia, fundus, body, and pylorus. Accordingly, capsule 200 is controlled to sequentially pass through these locations.

[0066] Figure 1 This is a flowchart of a method for reconstructing a three-dimensional model of the stomach according to one embodiment of the present application. Although the present application provides method operation steps as described in the following embodiment or flowchart, based on routine or no creative labor, the method has steps in which there is no necessary causal relationship logically, and the execution order of these steps is not limited to the execution order provided in the embodiment of the present application.

[0067] The specific method for reconstructing a three-dimensional stomach model includes the following steps:

[0068] Step S1: obtaining original point cloud information, wherein the original point cloud information is a set of multiple original coordinates, wherein the original coordinates are coordinates of sampling points located on the inner surface of the stomach.

[0069] During the entire process of the user taking the capsule 200 for examination, the above-mentioned capsule positioning system collects multiple positioning information of the location of the capsule 200 in real time. These locations include the positioning information of the capsule 200 at multiple locations of the stomach 301, such as the cardia, fundus, body, and pylorus.

[0070] The magnet 10 can be driven and controlled to make the capsule 200 move in the stomach 301, especially to move in as wide a range as possible and in as many different postures as possible, such as translation, rolling, jumping, etc., to traverse as many positions in the stomach 301 as possible.

[0071] During the movement of capsule 200, multiple positioning information can be continuously acquired. Capsule 200 primarily appears in the top-surface and bottom-sinking states, with less frequent capture in the middle of the stomach. However, at the top and bottom of the stomach, where capsule 200 is in close contact with the inner surface of the stomach, the acquired positioning information primarily consists of the coordinates of the inner surface of the stomach.

[0072] The positioning information can be provided in real time by the capsule positioning system. The 6-DOF state data provided by the capsule positioning system is [Cx, Cy, Cz, Ch, Cv, Cs] in the original coordinate system. The original coordinate system can be a local coordinate system or a world coordinate system.

[0073] Ideally, the more positioning information is obtained, the closer the final generated 3D model will be to the real situation.

[0074] Step S2: establishing a stomach coordinate system based on the original point cloud information, and calculating the transformed point cloud information in the stomach coordinate system.

[0075] Before establishing the stomach coordinate system, two situations can be handled:

[0076] In the first case, the quality of the original point cloud information is good, and the original point cloud information is directly used to establish the stomach coordinate system and calculate the transformed point cloud information.

[0077] The second case is that the quality of the original point cloud information is poor. Due to various reasons, the collected point cloud sometimes cannot cover all parts of the stomach. The three-dimensional surface of the stomach obtained by directly using the original point cloud information for three-dimensional reconstruction is not very ideal. In this case, it needs to be processed. In this case, step S2 includes:

[0078] S211: Obtaining standard model point cloud information;

[0079] S212: Performing an affine transformation on the original point cloud information according to the standard model point cloud information to generate adapted point cloud information.

[0080] In step S211, the standard model point cloud information corresponds to the point cloud collected under ideal conditions to cover the entire inner surface of the stomach, and the point cloud is three-dimensionally reconstructed according to steps S1-S4 of this application to obtain a complete stomach model.

[0081] Step S212 specifically includes:

[0082] S2121: Use PCA (Principal Component Analysis) to obtain the centroid and three main directions of the original point cloud information. The main directions coincide with the coordinate system directions corresponding to the original point cloud information.

[0083] S2122: Use the main direction and center of mass to form an affine transformation (including rotation vector and translation vector), and convert the standard model point cloud information and the original point cloud information to the origin respectively based on two affine transformation matrices.

[0084] S2123: Establish a model bounding box of the standard model point cloud information transformed to the origin, establish an original bounding box of the original point cloud information transformed to the origin, and deform the standard model point cloud information based on the original bounding box to obtain the deformed model point cloud information.

[0085] Specifically, the two bounding boxes are composed of the maximum and minimum values ​​of the three coordinate axes and the center point respectively. The standard model point cloud information is scaled and translated according to the size and center position of the two bounding boxes, so that the center point of the scaled and translated model bounding box coincides with the center point of the original bounding box.

[0086] S2124: Convert the deformed model point cloud information to the position of the original point cloud information to obtain the adapted point cloud information after deformation based on the standard model point cloud information.

[0087] Specifically, it is achieved through the inverse transformation from the original point cloud information to the deformed model point cloud information.

[0088] In the second case, in subsequent steps, the stomach coordinate system is established using the adapted point cloud information and the transformed point cloud information is calculated.

[0089] The following continues to explain the first case where the quality of the original point cloud information is good. If it is the second case, the original point cloud information can be replaced with the converted point cloud information.

[0090] The original coordinates are [Cx, Cy, Cz] in the 6-DOF state data given by the capsule positioning system, which represent the XYZ coordinates of the capsule endoscope in the original coordinate system. middle, , the set of multiple original coordinates is:

[0091] ,

[0092] Where N is the number of data, and its size depends on the sampling frequency and sampling time of the positioning device. Among them, the sampling frequency and sampling time are preset parameters and are set before the inspection.

[0093] The "establishing a stomach coordinate system" in step S2 includes multiple implementations, some of which are as follows:

[0094] In one embodiment, the average values ​​of all the original coordinates on the X, Y, and Z axes are calculated to obtain average coordinates; and the average coordinates are determined as the origin coordinates of the stomach coordinate system.

[0095] In this embodiment, the average values ​​of the original coordinate set in the three spatial dimensions {x, y, z} are respectively counted to obtain {x_mean, y_mean, z_mean}, which are calculated as follows:

[0096] .

[0097] Step S2 converts the original coordinate set obtained into a coordinate system from the original coordinate system to a coordinate system with the center of the stomach 301 as the origin, i.e., the stomach coordinate system. This reduces the computational complexity of subsequent coordinate data processing operations and makes the three-dimensional model visualization process more convenient.

[0098] In another embodiment of step S2, the minimum values ​​of all the original coordinates on the X, Y, and Z axes are first screened to obtain the minimum coordinates {x_min, y_min, z_min}; the minimum coordinates {x_min, y_min, z_min} are then determined as the origin coordinates of the stomach coordinate system. The original coordinates can be defined to increase in size from left to right and from top to bottom, such that the minimum coordinate corresponds to the upper left corner of all coordinates. In other words, the coordinates of the upper left corner of the stomach are used as the origin coordinates of the stomach coordinate system.

[0099] In another embodiment of step S2, the maximum values ​​{x_max, y_max, z_max} of all the original coordinates on the X, Y, and Z axes can be first screened to obtain the maximum value coordinates {x_max, y_max, z_max}; the maximum value coordinates can then be determined as the origin coordinates of the stomach coordinate system. Referring to the above-mentioned left-right and top-bottom definitions, the original coordinates can be defined to increase in size from left to right and from top to bottom. In this way, the maximum value coordinates correspond to the lower right corner of all coordinates. In other words, the coordinates of the lower right corner of the stomach are used as the origin coordinates of the stomach coordinate system.

[0100] The converted point cloud information includes a set of multiple converted coordinates, and step S2 further includes:

[0101] S221: Calculating the offset of the original coordinate relative to the origin of the stomach coordinate system in the direction of each coordinate axis;

[0102] S222: The offset of each transformed coordinate corresponding to each original coordinate.

[0103] Corresponding to the three embodiments above, the original coordinates are subtracted from their average / minimum / maximum values ​​in the corresponding dimensions to obtain new stomach coordinates. The set of all stomach coordinates is cloud_t. The specific calculation formula is as follows:

[0104]

[0105]

[0106] Step S3: resampling the converted point cloud information to generate stomach point cloud information.

[0107] The resampling is used to make the stomach point cloud information smooth and evenly distributed, and to at least partially remove point data located inside the inner surface of the stomach.

[0108] Specifically, step S3 includes:

[0109] S31: De-noising the converted point cloud information to obtain de-noised point cloud information, so as to smooth the stomach point cloud information.

[0110] The converted point cloud information is a set of multiple converted coordinates, and the denoised point cloud information is a set of multiple denoised coordinates.

[0111] S32: applying a voxel filtering algorithm, a uniform sampling algorithm, or a least squares method to the denoised point cloud information to generate uniform point cloud information, so that the stomach point cloud information is evenly distributed.

[0112] S33: applying a concave hull algorithm to the uniform point cloud information to generate stomach point cloud information, so as to at least partially remove point data located inside the inner surface of the stomach.

[0113] In step S31 , the denoising process can reduce the noise (such as positioning errors or deviations) generated during positioning of the magnetically controlled capsule system 1000 , making the processed point cloud data set smoother.

[0114] The outlier removal algorithm can be used. The specific steps are as follows:

[0115] Step S311: Calculate the mean avg and standard deviation std of all the transformed coordinates. The calculation formula is:

[0116]

[0117]

[0118] ;

[0119] Where dist(i,j) is the distance between the points (xi,yi,zi) and (xj,yj,zj) in the set cloud_t consisting of all transformed coordinates, and i,j are (0,N);

[0120] Step S312: Calculate the average distance k_avg from each transformed coordinate to the preset number k of adjacent transformed coordinates, where k is the preset number of nearest neighbors of the current positioning point, generally set between [5, 20], and can be dynamically adjusted according to the distribution of the actual point cloud. The calculation formula for the average distance k_avg is:

[0121] ;

[0122] In addition, the k-nearest points of each positioning point can be searched using the kdtree method.

[0123] Step S313: Calculate the offset delta corresponding to each transformed coordinate. The offset delta is the absolute value of the difference between the mean avg and the average distance k_avg. The calculation formula is:

[0124] ;

[0125] Step S314: setting the converted coordinates whose offset delta is less than the offset extreme value as the denoised coordinates, wherein the offset extreme value is the product of the standard deviation std and the control factor mul.

[0126] If delta>mul*std, then the point is an outlier, that is, a noise point, and it is removed, leaving only delta ≤mul*std

[0127] Step S31 may also be to perform denoising on all the transformed coordinates through Gaussian filtering to obtain denoised point cloud information.

[0128] The specific steps include: generating a Gaussian kernel weight value kernel based on the preset Gaussian kernel size r and standard deviation sigma; then Gaussian weighting each transformed coordinate with the other transformed coordinates within a radius r to obtain the new position of the current transformed coordinate and the denoised coordinate. The Gaussian kernel size r and Gaussian standard deviation sigma are set parameters and can be dynamically adjusted based on the actual point cloud distribution.

[0129] Since the capsule 200 moves at different speeds in different parts of the stomach, step S32 is adopted to make the distribution of the processed stomach point cloud information set more uniform.

[0130] Step S32 takes the voxel filtering algorithm as an example, and the steps are as follows:

[0131] Step S321: Filter the minimum coordinate {x_min, y_min, z_min} and the maximum coordinate {x_max, y_max, z_max} among all the denoised coordinates;

[0132] Step S322: Calculate the step lengths x_step, y_step, and z_step of the voxel grid according to the minimum coordinate and the maximum coordinate. Specifically, the step length of the voxel grid is calculated as follows:

[0133] ;

[0134] Step S323: Calculate the centroid (x_center, y_center, z_center) of each voxel grid, and traverse all voxel grids. The centroid coordinates of each grid are calculated as follows:

[0135]

[0136] Where m is the number of sampling points in the current grid, x_size, y_size, and z_size are the preset numbers of voxel grids, and x_size = 50, y_size = 50, and z_size = 50 can be taken.

[0137] Step S324: setting the center of gravity of all voxel grids as the uniform point cloud information.

[0138] Taking the uniform sampling algorithm as an example, setting the uniform sampling interval d, the coordinates of the point cloud after sampling are:

[0139]

[0140] Taking the least squares method as an example, when the collected point cloud data has many (large) holes, an upsampling algorithm based on the least squares method can be used to fill the holes. The steps are as follows: the surface formed by each point and its neighboring points is solved and fitted using the least squares method. Then, points are added to the surface within the neighborhood according to a certain step size to serve as hole filling points.

[0141] Step S33 at least partially removes the internal point data of the stomach. A concave hull is calculated for the uniform point cloud information to obtain the stomach point cloud information after the concave hull. Three-dimensional surface reconstruction is then performed on the stomach point cloud information. This results in a smoother three-dimensional surface with fewer internal holes.

[0142] Specifically, the concave hull algorithm may be an Alpha-shape algorithm or the like.

[0143] Step S4: performing three-dimensional surface reconstruction based on the stomach point cloud information to generate a three-dimensional stomach model.

[0144] In this step, the stomach point cloud information is reconstructed into a three-dimensional surface using a three-dimensional point cloud reconstruction algorithm to generate a three-dimensional stomach model.

[0145] The stomach point cloud information obtained in step S3 is subjected to three-dimensional surface reconstruction processing by a three-dimensional point cloud reconstruction algorithm to obtain an interactive three-dimensional surface visualization model of the stomach 301 .

[0146] The three-dimensional point cloud reconstruction algorithm is a marching cube algorithm, a greedy projected triangle algorithm or a Poisson reconstruction algorithm.

[0147] The above-mentioned method for reconstructing the three-dimensional model of the stomach can also be referred to Figure 3 , Figure 3 This is a schematic diagram of the calculation process of the stomach three-dimensional model reconstruction method, showing the calculation method and calculation results of each step in the method.

[0148] The three-dimensional model reconstruction method can be used to establish a three-dimensional model of the stomach 301, avoiding the radiation problem of CT imaging and solving the problem that the photos taken by the capsule 200 are not clear and intuitive enough. The three-dimensional model of the stomach 301 is generated through the positioning information of the capsule 200 in the stomach 301. The operation method is simple, the presentation of the stomach 301 is intuitive, and the stomach 301 can be observed from any angle, assisting doctors in clinical diagnosis.

[0149] In one embodiment, a stomach three-dimensional model reconstruction device is provided. Figure 4 The stomach three-dimensional model reconstruction device may include an acquisition module, a coordinate calculation module, a resampling module, and a three-dimensional reconstruction module. The specific functions of each module are as follows:

[0150] an acquisition module, configured to acquire original point cloud information, wherein the original point cloud information is a set of multiple original coordinates, wherein the original coordinates are coordinates of sampling points located on the inner surface of the stomach;

[0151] a coordinate calculation module, configured to establish a stomach coordinate system based on the original point cloud information and calculate transformed point cloud information in the stomach coordinate system;

[0152] a resampling module, configured to resample the converted point cloud information to generate stomach point cloud information, wherein the resampling is configured to smooth and evenly distribute the stomach point cloud information and at least partially remove point data located inside the inner surface of the stomach;

[0153] The three-dimensional reconstruction module is used to perform three-dimensional surface reconstruction based on the stomach point cloud information to generate a three-dimensional model of the stomach.

[0154] In one embodiment, the stomach three-dimensional model reconstruction device may be a magnetically controlled capsule system 1000, which can drive and control the magnet 10 to move the capsule 200 in the stomach 301; and continuously obtain multiple positioning information during the movement of the capsule 200.

[0155] In one embodiment, the stomach three-dimensional model reconstruction device further includes a standard model adaptation module for acquiring standard model point cloud information, and performing affine transformation on the original point cloud information according to the standard model point cloud information to generate adapted point cloud information.

[0156] In one embodiment, the coordinate calculation module calculates the offset of the original coordinates relative to the origin of the stomach coordinate system in the direction of each coordinate axis, and each transformed coordinate corresponds to the offset of each original coordinate.

[0157] In one embodiment, the resampling module is used to perform denoising on the converted point cloud information to obtain denoised point cloud information, wherein the converted point cloud information is a set including a plurality of converted coordinates, and the denoised point cloud information is a set including a plurality of denoised coordinates;

[0158] The resampling module is used to generate uniform point cloud information by applying a voxel filtering algorithm, a uniform sampling algorithm, or a least squares method to the denoised point cloud information;

[0159] The resampling module is used to apply a concave hull algorithm to the uniform point cloud information to generate stomach point cloud information.

[0160] In one embodiment, the three-dimensional reconstruction module performs three-dimensional surface reconstruction on the stomach point cloud information through a three-dimensional point cloud reconstruction algorithm to generate a three-dimensional stomach model; wherein the three-dimensional point cloud reconstruction algorithm is a marching cube algorithm, a greedy projected triangle algorithm or a Poisson reconstruction algorithm.

[0161] It should be noted that for details not disclosed in the stomach three-dimensional model reconstruction device of the embodiment of the present invention, please refer to the details disclosed in the stomach three-dimensional model reconstruction method of the embodiment of the present invention.

[0162] The three-dimensional stomach model reconstruction device according to the present invention can establish a three-dimensional model of the stomach 301, avoiding the radiation problem of CT imaging and solving the problem that the photos taken by the capsule 200 are not clear and intuitive enough. The three-dimensional model of the stomach 301 is generated through the positioning information of the capsule 200 in the stomach 301. The operation method is simple, the presentation of the stomach 301 is intuitive, and the stomach 301 can be observed from any angle, assisting doctors in clinical diagnosis.

[0163] In one embodiment, a medical device is provided. The medical device may be a magnetically controlled capsule system 1000, a computing device such as a desktop computer, a laptop, a PDA, or a cloud server. The medical device includes the aforementioned three-dimensional stomach model reconstruction device. The medical device may further include, but is not limited to, a processing module 40 and a storage module 30. Those skilled in the art will appreciate that the schematic diagram is merely an example of a three-dimensional stomach model reconstruction device and does not constitute a limitation of the medical device. The medical device may include more or fewer components than shown, or a combination of certain components, or different components. For example, the medical device may also include input and output devices, network access devices, buses, etc.

[0164] Take the medical device as an example, which is a magnetically controlled capsule system 1000. Figure 5FIG. 1 shows a schematic diagram of a magnetically controlled capsule system 1000. The magnetically controlled capsule system 1000 includes a magnetic control system 100, a capsule positioning system, a control magnet 10, and a capsule endoscope. The magnetic control system 100 is used to control the movement of the capsule endoscope, while the capsule positioning system is used to position the capsule endoscope. The capsule endoscope is internally equipped with a sensor module comprising a magnetic sensor 70 (magsensor), such as a Hall effect sensor or a magnetoresistive sensor (AMR, GMR, or TMR), for detecting magnetic fields. The control magnet 10 includes a magnetic source for emitting a magnetic field and a servo motor for controlling the movement of the magnetic source. The capsule endoscope also has a magnetic component 60 inside. The magnetic source exerts a force on the magnetic component 60, enabling the magnetic control system 100 to control the position and posture of the capsule endoscope. In this embodiment, the capsule endoscope is located inside the stomach, and the control magnet 10 controls the movement of the capsule endoscope within the stomach 301.

[0165] When acquiring data, the human body 300 lies flat on the bed, and a magnetic control system 100 is provided outside the human body 300. During the examination, the magnetic field emitted by the magnet 10 is controlled to control the movement of the capsule endoscope in the human body 300, especially in the stomach 301.

[0166] The magnetically controlled capsule system 1000 may further include a processing module 40, a storage module 30, and a computer program stored in the storage module 30 and executable on the processing module 40, such as the above-mentioned stomach three-dimensional model reconstruction method program. When the processing module 40 executes the computer program, the steps of each of the above-mentioned stomach three-dimensional model reconstruction method embodiments are implemented, such as Figure 1 Steps shown.

[0167] The magnetically controlled capsule system 1000 may further include a signal transmission module 20 and a communication bus 50. The signal transmission module 20 is used to send data to the processing module 40 or a server. The signal delivery module 80 and the signal transmission module 20 may transmit data via a wireless connection, such as Bluetooth, WiFi, Zigbee, etc. The communication bus 50 is used to establish a connection between the control magnet 10, the signal transmission module 20, the processing module 40, and the storage module 30. The communication bus 50 may include a path to transmit information between the control magnet 10, the signal transmission module 20, the processing module 40, and the storage module 30.

[0168] In addition, the present invention also proposes an electronic device, which includes a storage module 30 and a processing module 40. When the processing module 40 executes the computer program, it can implement the steps in the above-mentioned method for reconstructing a three-dimensional stomach model, that is, implement the steps in any one of the technical solutions in the above-mentioned method for reconstructing a three-dimensional stomach model.

[0169] The electronic device may be a part integrated into a stomach three-dimensional model reconstruction device, or a local terminal device, or a part of a cloud server.

[0170] The processing module 40 can be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), off-the-shelf field-programmable gate arrays (FPGAs), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processing module 40 serves as the control center of the three-dimensional stomach model reconstruction device, connecting the various components of the device using various interfaces and circuits.

[0171] The storage module 30 can be used to store the computer programs and / or modules. The processing module 40 implements the various functions of the three-dimensional stomach model reconstruction method by running or executing the computer programs and / or modules stored in the storage module 30 and accessing the data stored in the storage module 30. The memory may primarily include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function (such as sound playback or image playback); the data storage area may store data generated based on the use of the mobile phone, such as audio data and a phone book. Furthermore, the memory may include high-speed random access memory (RAM) and non-volatile memory, such as a hard disk, internal memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0172] For example, the computer program may be divided into one or more modules / units, which are stored in the storage module 30 and executed by the processing module 40 to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program in the method for reconstructing a three-dimensional stomach model.

[0173] Furthermore, an embodiment of the present invention provides a readable storage medium storing a computer program, which, when executed by the processing module 40, can implement the steps in the above-mentioned method for reconstructing a three-dimensional stomach model, that is, implement the steps in any one of the technical solutions in the above-mentioned method for reconstructing a three-dimensional stomach model.

[0174] If the integrated module of the stomach three-dimensional model reconstruction method is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention can also implement all or part of the process of the above-mentioned embodiment method by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When executed by a processor, the computer program can implement the steps of each of the above-mentioned stomach three-dimensional model reconstruction method embodiments.

[0175] The computer program includes computer program code, which may be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium may include any entity or device capable of carrying the computer program code, recording medium, disk, removable hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium. It should be noted that the content of the computer-readable medium may be appropriately increased or decreased based on the requirements of legislation and patent practice within a jurisdiction. For example, in some jurisdictions, based on legislation and patent practice, computer-readable media does not include electric carrier signals and telecommunication signals.

[0176] It should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each implementation method can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

[0177] The series of detailed descriptions listed above are only specific descriptions of feasible implementation methods of the present invention. They are not intended to limit the scope of protection of the present invention. Any equivalent implementation methods or changes that do not deviate from the technical spirit of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for reconstructing a three-dimensional stomach model, characterized in that: Including steps: Obtaining original point cloud information, wherein the original point cloud information is a set of multiple original coordinates, wherein the original coordinates are coordinates of sampling points located on the inner surface of the stomach, and the original point cloud information is positioning information of the capsule obtained by the magnetically controlled capsule system when the capsule moves in the digestive tract; Establishing a stomach coordinate system based on the original point cloud information, and calculating transformed point cloud information in the stomach coordinate system; resampling the converted point cloud information to generate stomach point cloud information, wherein the resampling is used to make the stomach point cloud information smooth and evenly distributed, and at least partially remove point data located inside the inner surface of the stomach; Performing three-dimensional surface reconstruction based on the stomach point cloud information to generate a three-dimensional stomach model; The step of establishing a stomach coordinate system based on the original point cloud information and calculating the transformed point cloud information in the stomach coordinate system includes: Get standard model point cloud information; The center of mass and three main directions of the original point cloud information are obtained by principal component analysis. The main directions coincide with the coordinate system directions corresponding to the original point cloud information. Using the main direction and the center of mass to form an affine transformation, the standard model point cloud information and the original point cloud information are respectively transformed to the origin based on the affine transformation matrix; Establishing a model bounding box of the standard model point cloud information transformed to the origin, establishing an original bounding box of the original point cloud information transformed to the origin, and deforming the standard model point cloud information based on the original bounding box to obtain deformed model point cloud information; Converting the deformed model point cloud information to the position of the original point cloud information to obtain adapted point cloud information after deformation based on the standard model point cloud information; A stomach coordinate system based on the adapted point cloud information is established, and transformed point cloud information of the adapted point cloud information in the stomach coordinate system is calculated.

2. The method for reconstructing a three-dimensional stomach model according to claim 1, wherein: The step of "establishing a stomach coordinate system based on the original point cloud information" includes: Calculate the average values ​​of all the original coordinates on the X, Y, and Z axes to obtain average coordinates; Determine the average value coordinates as the origin coordinates of the stomach coordinate system; or, Filter the minimum values ​​of all the original coordinates on the X, Y, and Z axes to obtain the minimum coordinates; Determining the minimum value coordinates as the origin coordinates of the stomach coordinate system; or, Filter the maximum values ​​of all the original coordinates on the X, Y, and Z axes to obtain the maximum value coordinates; The maximum value coordinates are determined as the origin coordinates of the stomach coordinate system.

3. The method for reconstructing a three-dimensional stomach model according to claim 2, wherein: The converted point cloud information includes a set of multiple converted coordinates; The step of "calculating the transformed point cloud information in the stomach coordinate system" includes: Calculating the offset of the original coordinates relative to the origin of the stomach coordinate system in the direction of each coordinate axis; Each transformed coordinate corresponds to an offset from each original coordinate.

4. The method for reconstructing a three-dimensional stomach model according to claim 1, wherein: The resampling further comprises: De-noising the converted point cloud information to obtain de-noised point cloud information, wherein the converted point cloud information is a set of multiple converted coordinates, and the de-noised point cloud information is a set of multiple de-noised coordinates; Applying a voxel filtering algorithm, a uniform sampling algorithm, or a least squares method to the denoised point cloud information to generate uniform point cloud information; A concave hull algorithm is used on the uniform point cloud information to generate stomach point cloud information.

5. The method for reconstructing a three-dimensional stomach model according to claim 4, wherein: The denoising process includes: Calculating the mean and standard deviation of all the transformed coordinates; Calculate the average distance between each transformed coordinate and a preset number of adjacent transformed coordinates; Calculate an offset corresponding to each transformed coordinate, where the offset is the absolute value of the difference between the mean and the average distance; The converted coordinates whose offset is less than or equal to the offset extreme value are set as the denoised coordinates, wherein the offset extreme value is the product of the standard deviation and the control factor.

6. The method for reconstructing a three-dimensional stomach model according to claim 4, wherein: When the voxel filtering algorithm is adopted, the steps include: Filter the minimum coordinate and the maximum coordinate among all the denoised coordinates; Calculating a step size of a voxel grid according to the minimum coordinate and the maximum coordinate; Calculate the centroid coordinates of each voxel grid and traverse all voxel grids; The centroid coordinates of all voxel grids are set as the uniform point cloud information.

7. The method for reconstructing a three-dimensional stomach model according to claim 1, wherein: The step of "reconstructing a three-dimensional surface based on the stomach point cloud information to generate a three-dimensional stomach model" includes: Performing three-dimensional surface reconstruction on the stomach point cloud information using a three-dimensional point cloud reconstruction algorithm to generate a three-dimensional stomach model; The three-dimensional point cloud reconstruction algorithm is a marching cube algorithm, a greedy projected triangle algorithm or a Poisson reconstruction algorithm.

8. A stomach three-dimensional model reconstruction device, characterized in that: include: an acquisition module, configured to acquire original point cloud information, wherein the original point cloud information is a collection of multiple original coordinates, wherein the original coordinates are coordinates of sampling points located on the inner surface of the stomach, and the original point cloud information is positioning information of the capsule acquired by the magnetically controlled capsule system when the capsule moves in the digestive tract; A coordinate calculation module is used to establish a stomach coordinate system based on the original point cloud information, calculate the converted point cloud information in the stomach coordinate system, and obtain the standard model point cloud information; use the principal component analysis method to obtain the center of mass and three main directions of the original point cloud information respectively, and the main directions coincide with the coordinate system directions corresponding to the original point cloud information; use the main directions and the center of mass to form an affine transformation, and transform the standard model point cloud information and the original point cloud information to the origin respectively based on the affine transformation matrix; establish a model bounding box of the standard model point cloud information transformed to the origin, establish an original bounding box of the original point cloud information transformed to the origin, and based on the original bounding box, deform the standard model point cloud information to obtain deformed model point cloud information; transform the deformed model point cloud information to the position of the original point cloud information to obtain adapted point cloud information after deformation based on the standard model point cloud information; establish a stomach coordinate system based on the adapted point cloud information, and calculate the converted point cloud information of the adapted point cloud information in the stomach coordinate system; a resampling module, configured to resample the converted point cloud information to generate stomach point cloud information, wherein the resampling is configured to smooth and evenly distribute the stomach point cloud information and at least partially remove point data located inside the inner surface of the stomach; The three-dimensional reconstruction module is used to perform three-dimensional surface reconstruction based on the stomach point cloud information to generate a three-dimensional model of the stomach.

9. A medical device, characterized in that: include: The stomach three-dimensional model reconstruction device according to claim 8.

10. An electronic device, characterized in that: include: a storage module storing a computer program; The processing module can implement the steps of the stomach three-dimensional model reconstruction method according to any one of claims 1 to 7 when executing the computer program.

11. A readable storage medium storing a computer program, characterized in that: When the computer program is executed by the processing module, the steps of the method for reconstructing a three-dimensional stomach model as described in any one of claims 1 to 7 can be implemented.

Images