Control method, positioning method and device of magnetic control capsule system

Abstract

The invention discloses a control method, a positioning method and a positioning device of a magnetic control capsule system. The control method comprises the following steps: continuously acquiring measurement state data of a capsule endoscope; the control magnet is controlled to move until the magnetic field state data is maximum and / or the capsule vertical inclination angle of the capsule endoscope is minimum; and correcting a capsule spin angle of the capsule endoscope so as to enable the direction of the capsule endoscope to be consistent with the direction on the image acquired by the capsule endoscope when the capsule endoscope moves or the posture is adjusted. According to the method, the moving direction of the capsule endoscope can be consistent with the direction on the acquired image only by depending on the measurement state data acquired by the capsule endoscope without depending on positioning equipment, that is, the capsule endoscope can be controlled based on the first visual angle of the image of the capsule endoscope, and the accuracy of the capsule endoscope is improved. Therefore, movement or posture adjustment of the capsule endoscope can be accurately controlled.

Description

Technical Field

[0001] The present invention relates to the technical field of medical equipment, and in particular to a control method, a positioning method and a device for a magnetically controlled capsule system. Background Art

[0002] In-vivo device positioning technology, such as wireless capsule endoscopes and invasive medical devices, has received increasing attention. The magnetically controlled capsule system drives the movement of the magnet to generate magnetic force to drive the capsule endoscope to move in the body. During the movement, the capsule endoscope captures images of the digestive tract. There are two main control methods for driving capsule endoscopes: artificial simulation control that relies on visual feedback from the images captured by the capsule, and automated quantitative control based on the positioning information of the capsule.

[0003] In the process of implementing the present invention, the inventors found that these two control methods have at least the following problems:

[0004] (1) On the one hand, manual simulation control has low accuracy and is difficult to accurately control the capsule endoscope to capture images of a certain position. In addition, since the images of many positions in the digestive tract have a high degree of similarity and the capsule endoscope is almost unconstrained in the gastric cavity, how to accurately identify and control the movement of the capsule endoscope requires a high level of operator skill. This results in low efficiency and a high possibility of missed detection.

[0005] (2) Automated quantitative control requires the real-time posture information fed back by the positioning device, and requires hardware and software that match the positioning device. It has high requirements for the equipment. If the equipment conditions cannot be met, the movement and photo-taking of the capsule endoscope cannot be accurately controlled.

[0006] Therefore, the current control methods are still insufficient and cannot well meet the needs of medical examinations. Summary of the invention

[0007] To solve at least one of the above-mentioned problems in the prior art, an object of the present invention is to provide a control method, a positioning method and a device for a magnetically controlled capsule system that can accurately control the movement of a capsule endoscope without relying on a positioning device.

[0008] To achieve the above-mentioned object of the invention, an embodiment of the present invention provides a control method of a magnetically controlled capsule system, wherein the magnetically controlled capsule system includes a control magnet and a capsule endoscope, and the control method includes the following steps:

[0009] Step S10: continuously acquiring measurement status data of the capsule endoscope, wherein the measurement status data includes acceleration data and / or magnetic field status data;

[0010] Step S20: controlling the control magnet to move at the same height, and adjusting the position of the control magnet until the magnetic field state data is maximum, and / or the capsule vertical tilt angle of the capsule endoscope is minimum, wherein the capsule vertical tilt angle is calculated according to the acceleration data;

[0011] Step S30: calibrating the capsule rotation angle of the capsule endoscope so that the direction of the capsule endoscope when it moves or adjusts its posture is consistent with the direction on the image acquired by the capsule endoscope.

[0012] As a further improvement of the present invention, step S20 includes:

[0013] Controlling the magnetization direction of the control magnet to be in a vertical direction;

[0014] The control magnet is controlled to translate within a plane perpendicular to the vertical direction while keeping the magnetization direction unchanged, wherein the translation is a movement in a direction that gradually reduces the vertical tilt angle of the capsule and / or gradually increases the magnetic field state data.

[0015] As a further improvement of the present invention, the step of controlling the translation of the control magnet comprises:

[0016] The movement of the control magnet is controlled in coordination with the magnetic field state data and the vertical tilt angle of the capsule.

[0017] As a further improvement of the present invention, the step of controlling the movement of the control magnet in coordination with the magnetic field state data and the vertical tilt angle of the capsule comprises:

[0018] In the process of controlling the movement of the control magnet, if the vertical tilt angle of the capsule does not change, continuously moving the control magnet until the vertical tilt angle of the capsule changes;

[0019] In the process of controlling the movement of the control magnet, if the vertical tilt angle of the capsule increases and the magnetic field state data decreases, the control magnet is controlled to move in the opposite direction.

[0020] As a further improvement of the present invention, after step S20, the following steps are further included:

[0021] Calculating the height of the capsule endoscope according to the magnetic field state data;

[0022] The height of the control magnet is adjusted so that the height of the capsule endoscope is within a set range.

[0023] As a further improvement of the present invention, the steps are also included:

[0024] Step S40: Acquire image data captured by the capsule endoscope;

[0025] Step S50: adjusting the posture of the control magnet while keeping the position of the control magnet unchanged, so as to obtain a plurality of peripheral image data outside the field of view corresponding to the image data;

[0026] Step S60: splicing the image data and the peripheral image data to obtain extended image data.

[0027] As a further improvement of the present invention, step S50 includes:

[0028] On the basis of keeping the position of the control magnet unchanged, adjusting the posture of the control magnet so that the position of the capsule endoscope remains unchanged and only the posture is adjusted;

[0029] The posture of the control magnet is adjusted to move the capsule endoscope toward a target direction, wherein the target direction is a direction extending from a field of view corresponding to the image data to a field of view corresponding to the peripheral image data.

[0030] As a further improvement of the present invention, the step S50 further includes:

[0031] The posture of the control magnet is adjusted so that the newly acquired peripheral image data includes images outside the field of view corresponding to the image data while retaining a portion of the images within the field of view corresponding to the image data.

[0032] As a further improvement of the present invention, the step of adjusting the posture of the control magnet includes:

[0033] Acquiring an expected movement amount of the field of view of the capsule endoscope;

[0034] Calculating a posture adjustment amount of the capsule endoscope according to the expected movement amount of the visual field;

[0035] Calculating the posture adjustment amount of the control magnet according to the posture adjustment amount of the capsule endoscope;

[0036] The control magnet is controlled to adjust a corresponding change amount according to the attitude adjustment amount of the control magnet.

[0037] As a further improvement of the present invention, the multiple peripheral image data include peripheral image data at eight positions above, below, left, right, upper left, upper right, lower left and lower right of the image data.

[0038] As a further improvement of the present invention, the steps are also included:

[0039] Step S70: controlling the control magnet to move so that the capsule endoscope moves to the next control node;

[0040] Step S80: On the basis of continuously executing step S10, looping steps S20 to S70 until the inspection is completed.

[0041] To achieve one of the above-mentioned purposes of the invention, an embodiment of the present invention provides a positioning method of a magnetically controlled capsule system, wherein the magnetically controlled capsule system includes a control magnet and a capsule endoscope, and the positioning method includes the following steps:

[0042] Continuously acquiring measurement status data of the capsule endoscope, wherein the measurement status data includes acceleration data and / or magnetic field status data;

[0043] Controlling the control magnet to move at the same height, and adjusting the position of the control magnet until the magnetic field state data is maximum, and / or the capsule vertical tilt angle of the capsule endoscope is minimum, wherein the capsule vertical tilt angle is calculated according to the acceleration data;

[0044] Acquiring magnet state data of the control magnet, wherein the magnet state data includes magnet position data and magnet posture data;

[0045] Determining capsule position data of the capsule endoscope according to the current magnet position data;

[0046] The capsule posture data of the capsule endoscope is determined according to the current measurement state data and the magnet posture data.

[0047] As a further improvement of the present invention, the step of determining the capsule position data of the capsule endoscope according to the current magnet position data comprises:

[0048] The x-axis coordinate of the capsule endoscope is the same as the x-axis coordinate of the capsule endoscope;

[0049] The y-axis coordinate of the capsule endoscope is the same as the y-axis coordinate of the capsule endoscope;

[0050] The z-axis coordinate of the capsule endoscope is calculated according to the magnetic field state data.

[0051] As a further improvement of the present invention, the capsule posture data includes the capsule horizontal azimuth angle, the capsule vertical tilt angle, and the capsule spin angle, and the magnet posture data includes the magnet horizontal azimuth angle;

[0052] When the magnetic field state data is maximum and / or the capsule vertical tilt angle of the capsule endoscope is minimum, the capsule vertical tilt angle is zero;

[0053] The step of determining the capsule posture data of the capsule endoscope according to the current measurement state data and the magnet posture data also includes:

[0054] Calculating the capsule spin angle according to the acceleration data;

[0055] After the magnetic field state data is maximum and / or the capsule vertical tilt angle of the capsule endoscope is minimum, the following steps are performed:

[0056] Adjusting the posture of the control magnet so that the vertical tilt angle of the capsule is not zero;

[0057] The capsule horizontal azimuth angle is equal to the magnet horizontal azimuth angle.

[0058] To achieve one of the above-mentioned objects of the invention, an embodiment of the present invention provides a control device for a magnetically controlled capsule system, the magnetically controlled capsule system comprising a control magnet and a capsule endoscope, the control device comprising:

[0059] A capsule state acquisition module, used for continuously acquiring measurement state data of the capsule endoscope, wherein the measurement state data includes acceleration data and / or magnetic field state data;

[0060] a posture adjustment module, used for controlling the control magnet to move at the same height, and adjusting the posture of the control magnet until the magnetic field state data is maximum, and / or the capsule vertical tilt angle of the capsule endoscope is minimum, wherein the capsule vertical tilt angle is calculated according to the acceleration data;

[0061] The correction module is used to correct the capsule rotation angle of the capsule endoscope so that the direction of the capsule endoscope when it moves or adjusts its posture is consistent with the direction on the image acquired by the capsule endoscope.

[0062] To achieve one of the above-mentioned purposes of the invention, an embodiment of the present invention provides a positioning device for a magnetically controlled capsule system, the magnetically controlled capsule system comprising a control magnet and a capsule endoscope, the positioning device comprising:

[0063] A capsule state acquisition module, used for continuously acquiring measurement state data of the capsule endoscope, wherein the measurement state data includes acceleration data and / or magnetic field state data;

[0064] a posture adjustment module, used for controlling the control magnet to move within the same height, and adjusting the posture of the control magnet until the magnetic field state data is maximum, and / or the capsule vertical tilt angle of the capsule endoscope is minimum, wherein the capsule vertical tilt angle is calculated according to the acceleration data;

[0065] A magnet state acquisition module, used for acquiring magnet state data of the control magnet, wherein the magnet state data includes magnet position data and magnet posture data;

[0066] A capsule position calculation module, used to determine capsule position data of the capsule endoscope according to the current magnet position data;

[0067] The capsule posture calculation module is used to determine the capsule posture data of the capsule endoscope according to the current measurement state data and the magnet posture data.

[0068] To achieve one of the above-mentioned purposes of the invention, an embodiment of the present invention provides an electronic device, including:

[0069] A storage module storing a computer program;

[0070] The processing module can implement the steps of the control method of the magnetically controlled capsule system or the positioning method of the magnetically controlled capsule system when executing the computer program.

[0071] To achieve one of the above-mentioned purposes of the invention, an embodiment of the present invention provides a readable storage medium storing a computer program, which, when executed by a processing module, can implement the steps in the above-mentioned control method of the magnetically controlled capsule system or the positioning method of the magnetically controlled capsule system.

[0072] Compared with the prior art, the present invention has the following beneficial effects: (1) The control method of the magnetically controlled capsule system can be applied without relying on the positioning device, only relying on the measurement status data obtained by the capsule endoscope, so that the subsequent movement or posture adjustment direction of the capsule endoscope can be consistent with the direction on the acquired image, that is, the movement or posture adjustment of the capsule endoscope can be controlled based on the first-person perspective of the image of the capsule endoscope, and the capsule endoscope can be controlled in the form of "what you think is what you get", so that the adjustment of the capsule endoscope can be accurately controlled, which reduces the demand on the ability of doctors, meets the ability conditions of more doctors, and is conducive to the promotion and application of the magnetically controlled capsule system.

[0073] (2) By controlling the capsule endoscope based on the first-person perspective of the capsule endoscope image, images within the required field of view can be obtained, thereby improving the efficiency of the inspection, avoiding the problem of missed inspection, and meeting the needs of physicians.

[0074] (3) Without relying on positioning equipment, the capsule endoscope is positioned by first controlling the magnet to reach directly above the capsule endoscope, thus meeting the need for precise control. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 This is a schematic diagram of the structure of a magnetically controlled capsule system for digestive tract examination according to an embodiment of the present invention.

[0076] Figure 2a is a schematic diagram of magnet state data of a control magnet according to an embodiment of the present invention;

[0077] Figure 2b is a schematic diagram of measurement status data of a capsule endoscope according to an embodiment of the present invention;

[0078] Figure 3 is a flow chart of a control method of a magnetically controlled capsule system according to a first embodiment of the present invention;

[0079] Figure 4 yes Figure 3 A flowchart of a specific implementation of step S20;

[0080] Figure 5 is a characteristic diagram of the change of the capsule vertical tilt angle Cv and the magnetic field state data B with the position according to an embodiment of the present invention;

[0081] Figure 6a is a schematic diagram of the field of view orientation of a capsule endoscope according to an embodiment of the present invention;

[0082] Figure 6b is a schematic diagram of capsule endoscope spin correction according to an embodiment of the present invention;

[0083] Figure 7 is a schematic diagram of adjusting the direction of the capsule endoscope according to the first embodiment of the present invention based on the first viewing angle field;

[0084] Figure 8 is a flow chart of a control method of a magnetically controlled capsule system according to a second embodiment of the present invention;

[0085] Fig. 9 yes Figure 8 A flowchart of a specific implementation of step S50;

[0086] Fig.10a is a structural schematic diagram of the capsule endoscope of the second embodiment of the present invention executing step S40 when it is at the bottom of the digestive tract;

[0087] Fig.10b is a schematic structural diagram of the capsule endoscope of the second embodiment of the present invention executing step S50 when it is at the bottom of the digestive tract;

[0088] Fig.11a is a schematic structural diagram of the capsule endoscope of the second embodiment of the present invention executing step S40 when it is at the top of the digestive tract;

[0089] Fig.11b is a schematic structural diagram of the capsule endoscope of the second embodiment of the present invention executing step S50 when it is at the top of the digestive tract;

[0090] Fig.12 is a schematic diagram of the distribution of the lens field of view angle of the capsule endoscope according to the second embodiment of the present invention;

[0091] Fig.13 is a schematic diagram of a lens movement path of a capsule endoscope when executing step S50 in the second embodiment of the present invention;

[0092] Fig.14 is a rendering of the extended image data obtained in the second embodiment of the present invention;

[0093] Fig.15 is a flow chart of a positioning method of a magnetically controlled capsule system according to a third embodiment of the present invention;

[0094] Fig.16 is a schematic diagram of a module of a control device of a magnetically controlled capsule system according to a first embodiment of the present invention;

[0095] Fig.17 is a module schematic diagram of a control device of a magnetically controlled capsule system according to a second embodiment of the present invention;

[0096] Fig.18 is a schematic diagram of a module of a positioning device of a magnetically controlled capsule system according to a third embodiment of the present invention;

[0097] Fig.19 is a structural block diagram of a magnetically controlled capsule system according to an embodiment of the present invention;

[0098] Among them, 100, magnetically controlled capsule system; 10, capsule endoscope; 11, inertial measurement unit; 12, camera unit; 13, data processing and transmission unit; 14, power supply unit; 15, permanent magnet unit; 20, magnetic control system; 21, control magnet; 22, motor drive device; 23, signal transmission module; 24, storage module; 25, processing module; 26, communication bus. DETAILED DESCRIPTION

[0099] The present invention will be described in detail below in conjunction with the specific embodiments shown in the accompanying drawings. However, these embodiments do not limit the present invention, and any structural, methodological, or functional changes made by a person skilled in the art based on these embodiments are all within the scope of protection of the present invention.

[0100] An embodiment of the present invention provides a control method, a positioning method and a device for a magnetically controlled capsule system that can accurately control the movement of a capsule endoscope without relying on a positioning device.

[0101] Magnetic Control Capsule System

[0102] The magnetically controlled capsule system of this embodiment may include a capsule endoscope 10 and a magnetically controlled system. The magnetically controlled system includes a control magnet 21. The control magnet 21 may drive the capsule endoscope 10 to move. Figure 1As shown, the capsule endoscope 10 may be located in the digestive tract, and the control magnet 21 controls the capsule endoscope 10 to move in the digestive tract, for example, in the stomach, and takes images of the stomach.

[0103] There can be both magnetic force interaction and information data interaction between the capsule endoscope 10 and the magnetic control system. The magnetic control system also includes a motor drive device, which is used to control the movement of the control magnet 21. During the movement, the magnet state data of the control magnet 21 can be obtained, and the magnet state data includes magnet position data and magnet posture data, and the control magnet 21 can also be controlled to move to a desired position and desired posture.

[0104] The capsule endoscope 10 includes a permanent magnet unit and a lens. The permanent magnet unit has an axially magnetized cylindrical structure, and the lens is used to capture images. The motor drive device controls the movement of the control magnet 21 while controlling the magnet 21 to exert a magnetic force on the permanent magnet unit, thereby controlling the movement of the capsule endoscope 10. When the capsule endoscope 10 is located inside the human digestive tract, the human body lies flat on the bed, and a magnetic control system is provided outside the human body. During the inspection, the motor drive device drives the control magnet 21 to move, and controls the magnet 21 to generate a magnetic field, and the generated magnetic field drives the capsule endoscope 10 to move in the human digestive tract. In addition, the magnetic control system can also send a signal instruction to the capsule endoscope 10 when to capture an image.

[0105] The capsule endoscope 10 also includes an inertial measurement unit (IMU), which includes a three-axis accelerometer, a gyroscope and a three-axis magnetometer. The IMU provides necessary navigation information feedback and realizes more accurate and convenient intelligent control of the capsule endoscope based on inertial navigation. The control method of the first perspective based on inertial navigation is described in detail below.

[0106] Furthermore, before introducing the specific method, the spatial coordinates are explained first. Embodiments 1, 2, and 3 of this document can all calculate specific values ​​based on this coordinate system.

[0107] like Figure 2a As shown, a Cartesian world coordinate system is established for the space where the magnetically controlled capsule system is located. Through the control system software, the magnet state data of the control magnet 21 can be obtained in real time, wherein the magnet state data includes magnet position data and magnet attitude data, the magnet position data is [Mx, My, Mz], and the magnet attitude data is [Mh, Mv], wherein [Mx, My, Mz] represents the position offset of the center of the control magnet 21 in the world coordinate system, Mh is the horizontal azimuth, and M vThe magnetic field of the control magnet 21 has the characteristic of rotational symmetry about the attitude orientation axis at the far end. The attitude orientation has two independent degrees of freedom. The attitude orientation angle of the N pole of the control magnet 21 is determined by the horizontal azimuth angle and the vertical tilt angle. As shown in FIG2 , the horizontal azimuth angle is the angle between the magnetic pole orientation of the control magnet 21 and the XY plane projection vector of the coordinate system and the positive direction of the Y axis. The vertical tilt angle is the angle between the magnetic pole orientation of the control magnet 21 and the positive direction of the Z axis of the coordinate system.

[0108] like Figure 2b As shown, the parameters of the capsule endoscope 10 are also based on the world coordinate system, and the component data of the inertial measurement unit are [gx, gy, gz, wx, wy, wz, bx, by, bz], where [gx, gy, gz] represent the three-axis components of the accelerometer; [wx, wy, wz] represent the three-axis components of the gyroscope; and [bx, by, bz] represent the three-axis components of the magnetometer. In theory, the position and posture of the capsule endoscope 10 include capsule position data [Cx, Cy, Cz] and capsule attitude data [Ch, Cv, Cs]. Among them, the angle between the head orientation of the capsule endoscope 10 and the positive direction of the Z axis is defined as the capsule vertical tilt angle Cv; the angle between the projection vector of the head orientation of the capsule endoscope 10 in the XY plane and the positive direction of the Y axis is defined as the capsule horizontal azimuth angle Ch; the capsule spin angle Cs is the orientation angle of the lens of the capsule endoscope 10. In addition, due to the lack of positioning equipment, the capsule position data and capsule attitude data cannot be directly obtained.

[0109] Preferably, the capsule lens of the capsule endoscope 10 is used as a reference to establish a coordinate system, the direction along the long axis pointing to the lens is the Z axis, the plane perpendicular to the Z axis is the XY plane, and the Y axis direction is fixed as the direction in which the lens is upright. At this time, the capsule spin angle Cs = 0. When assembling the capsule endoscope 10, the capsule lens of the capsule endoscope 10 and the XY coordinates of the inertial measurement unit can be aligned to save the later correction of the capsule spin angle. In other words, if the coordinate alignment is not performed, it can be compensated and corrected in the subsequent steps.

[0110] Combine the following Figure 1 to Figure 15 , describing the control method and positioning method of the magnetically controlled capsule system provided by an embodiment of the present invention, which is divided into 3 embodiments for description, wherein: Embodiment 1 is used to control the capsule endoscope 10 to move or adjust its posture along the direction of the first viewing angle of the capsule endoscope 10 to obtain the image. Embodiment 2 is used to control the capsule endoscope 10 to obtain the image within the required field of view on the basis of the first viewing angle direction. Embodiment 3 is used to obtain the position information of the capsule endoscope 10 without the positioning device providing the positioning data.

[0111] In Example 1, the method steps of Example 1 can be completed based only on the acceleration data and / or magnetic field state data acquired by the inertial measurement unit.

[0112] In Example 1, the capsule spin angle Cs is fixed, and Example 2 can be calculated based on only two parameters, the capsule vertical tilt angle Cv and the capsule horizontal azimuth angle Ch, combined with the magnet posture data [Mh, Mv] of the control magnet.

[0113] In both Embodiment 1 and Embodiment 2, it is not necessary to calculate the capsule position data [Cx, Cy, Cz].

[0114] Embodiment 3 can determine the 6DOF complete state parameters [Cx, Cy, Cz, Ch, Cv, Cs] of the capsule endoscope.

[0115] Although the present application provides the method operation steps shown in the following implementation or flow chart, based on routine or no creative work, the execution order of the steps in the method that do not have a necessary causal relationship in logic is not limited to the execution order provided in the implementation of the present application. For example, the acquisition order of step S10 and other steps below can be adjusted arbitrarily, without distinguishing the order in time sequence.

[0116] Embodiment 1: A control method of a magnetically controlled capsule system

[0117] Example 1 Figures 3 to 7 As shown, a control method for a magnetically controlled capsule system is introduced.

[0118] Specifically, Figure 3 As shown, the following steps are included:

[0119] Step S10: continuously acquiring measurement status data of the capsule endoscope, wherein the measurement status data includes acceleration data and / or magnetic field status data.

[0120] Step S20: Control the control magnet to move at the same height, and adjust the position of the control magnet until the magnetic field state data is maximum and / or the vertical tilt angle of the capsule endoscope is minimum, wherein the vertical tilt angle of the capsule is calculated based on the acceleration data.

[0121] Step S30: calibrating the capsule rotation angle of the capsule endoscope so that the direction of the capsule endoscope when it moves or adjusts its posture is consistent with the direction on the image acquired by the capsule endoscope.

[0122] First, in step S10, the acceleration data is [gx, gy, gz] obtained by the accelerometer mentioned above, and the magnetic field state data is [bx, by, bz] obtained by the magnetometer mentioned above. In addition, the measurement state data may also include [wx, wy, wz] obtained by the gyroscope. The measurement state data is real-time data that is continuously obtained and is updated as time goes by in subsequent steps, so as to facilitate the state parameter calculation, state feedback, and capsule control operation required in the subsequent steps.

[0123] In addition, considering that the magnetic field value measured by the magnetometer includes three sources: the environmental background magnetic field, the capsule magnet background magnetic field, and the control magnet magnetic field, since the weaker environmental background magnetic field can usually be ignored and the capsule magnet background magnetic field can be accurately deducted first, the capsule magnetometer can easily separate the more accurate net magnetic field value of the control magnet. In other words, the magnetic field state data in this embodiment can be only the magnetic field generated by the control magnet, that is, B = [bx, by, bz].

[0124] Secondly, in step S20, the control magnet is moved stably, accurately and quickly to the top of the capsule. This is very beneficial because when the control magnet is located above the capsule endoscope, it provides the maximum control magnetic force to the capsule and can provide all-round posture control to the capsule.

[0125] Step S20 includes multiple implementations, including:

[0126] (1) adjusting the position and posture of the control magnet based on the direction that maximizes the magnetic field state data;

[0127] (2) adjusting the position of the control magnet based on the direction that minimizes the vertical tilt angle of the capsule;

[0128] (3) The position and posture of the control magnet are adjusted based on the direction that maximizes the magnetic field state data and minimizes the vertical tilt angle of the capsule.

[0129] The following uses implementation (3) with the maximum magnetic field state data and the minimum capsule vertical tilt angle as examples for explanation. Based on this explanation, those skilled in the art can also understand the implementation principles of implementations (1) and (2).

[0130] Ginseng Figure 4 As shown, step S20 also includes steps S21 to S23:

[0131] Step S21: Control the movement of the control magnet in coordination with the magnetic field state data and the vertical tilt angle of the capsule.

[0132] In step S21, the magnetic field state data is directly determined according to B=[bx, by, bz] obtained by the magnetometer, and the capsule vertical tilt angle Cv can be determined according to [gx, gy, gz] obtained by the accelerometer. Specifically, the calculation formula of the capsule vertical tilt angle Cv is:

[0133] In the process of controlling the movement of the control magnet, the magnetization direction of the control magnet can be controlled to be in a vertical direction, and on the basis of keeping the magnetization direction unchanged within a plane perpendicular to the vertical direction, the control magnet is controlled to translate, wherein the translation is along the direction that gradually reduces the vertical tilt angle of the capsule and / or gradually increases the magnetic field state data.

[0134] Since the body shapes of users are generally the same, the control magnet can be maintained at a certain height according to the system settings, and only move in the XY plane. When it moves to the top of the capsule endoscope, it can be ensured that the capsule endoscope will not completely sink to the bottom or completely stick to the top, but the bottom of the capsule endoscope is supported on the lower wall of the digestive tract, and the top of the capsule endoscope is vertically upward, or the bottom of the capsule endoscope is supported on the upper wall of the digestive tract, and the top of the capsule endoscope is vertically downward. When the height of the control magnet remains unchanged and only moves in the XY plane, the distribution of the capsule vertical tilt angle Cv and the magnetic field state data B is as follows: Figure 5 shown.

[0135] Ginseng Figure 5 As shown, when the control magnet is located directly above the capsule endoscope, the capsule endoscope is vertically upward, and the capsule vertical tilt angle Cv approaches 0 at this time. The amplitude value B of the magnetic field of the control magnet obtained at the magnetometer has a maximum value in the XY plane. If the capsule endoscope deviates from the directly below the control magnet in any direction in the XY plane, the capsule vertical tilt angle Cv will increase, and the magnetic field state data B will decrease. Therefore, the distribution of the capsule vertical tilt angle Cv and the magnetic field state data B are symmetrical about the Z-axis rotation axis. How to gradually reduce the capsule vertical tilt angle Cv and / or gradually increase the magnetic field state data B is a convex optimization problem. According to the state and change trend of Cv and B, in coordination with the magnetic field state data and the capsule vertical tilt angle, it is easy to quickly and accurately move the control magnet to the directly above the capsule endoscope.

[0136] In addition, in the process of controlling the movement of the control magnet, if the vertical tilt angle of the capsule does not change, the control magnet is continuously moved until the vertical tilt angle of the capsule changes. This means that the control magnet is far away from the capsule endoscope and the magnetic force acting on the capsule endoscope is weak, so the vertical tilt angle Cv value of the capsule does not change when the control magnet moves. The control magnet can continue to be driven close to the capsule endoscope until the vertical tilt angle Cv value of the capsule changes to a certain extent, and then the moving path is optimized in coordination with the magnetic field state data and the vertical tilt angle of the capsule to improve the efficiency and accuracy of the control magnet in finding the capsule.

[0137] If the vertical tilt angle of the capsule increases and the magnetic field state data decreases, the control magnet is controlled to move in the opposite direction.

[0138] Step S22: Determine whether the magnetic field state data is maximized and / or the capsule vertical tilt angle is minimized.

[0139] When the judgment result of step S22 is no, the process returns to step S21; when the judgment result of step S22 is yes, the process proceeds to step S23.

[0140] Step S23: Control the magnet to move to just above the capsule endoscope.

[0141] Furthermore, after step S20, the method may further include the following steps:

[0142] Calculating the height of the capsule endoscope according to the magnetic field state data;

[0143] The height of the control magnet is adjusted so that the height of the capsule endoscope is within a set range.

[0144] As mentioned above, although the control magnet can be maintained at a certain height to ensure that the capsule endoscope will not completely sink to the bottom or completely stick to the ceiling, after the magnet moves to the top of the capsule endoscope, the height difference between the capsule endoscope and the control magnet can be calculated, and the height can be further limited. At this time, the calculation formula for the center height distance Z0 of the capsule endoscope and the control magnet is:

[0145] Right now;

[0146] In this way, the height difference Z0 between the control magnet and the capsule endoscope can be quantitatively adjusted to provide a suitable control magnetic force. This can ensure that the capsule endoscope will not exceed the critical height distance and be sucked to the top of the stomach cavity when photographing the upper half of the stomach cavity. It can also be selected to accurately and quickly change the height difference between the control magnet and the capsule endoscope when necessary to ensure that the capsule endoscope is sucked up to achieve the position migration between the bottom and the top of the capsule endoscope.

[0147] After step S20, since there is no restriction on the capsule rotation angle Cs, the value of the capsule rotation angle Cs can be any value. At this time, step S30 is executed to fix the value of the capsule rotation angle Cs, so as to facilitate the control of the capsule endoscope with its own first-view angle. Figure 6a and 6b , Figure 6a It is a schematic diagram of the imaging field of view, where H+ represents the positive direction of the horizontal axis of the image, and V+ represents the positive direction of the vertical axis of the image. Figure 6b In the embodiment, when the directions of H+ and Hw are corrected to be consistent, and the directions of V+ and Vw are corrected to be consistent, Cs=0, and the image can be corrected to the state of Cs=0.

[0148] During the calibration process of step S30, the capsule spin angle value can be obtained by the inertial measurement unit and then corrected to Cs=0, where:

[0149] In addition, considering the complex environment in the digestive tract, such as the peristalsis of the stomach, the capsule endoscope may not be completely static. When calculating the capsule attitude angle, the accelerometer and gyroscope status information can be combined for complementary filtering to enhance the dynamic accuracy of the capsule attitude angle.

[0150] After executing the above steps S10 to S30, in the absence of additional positioning equipment, the control magnet is located directly above the capsule endoscope. The subsequent control of the magnet can be directly based on the orientation of the captured image and accurate direction control based on visual feedback. In other words, the current image obtained is consistent with the cubic orientation during the first-person perspective real-life measurement, and the control is adjusted by referring to the up, down, left, and right field of view angles of the current image in accordance with the intuitive habits of the human eye.

[0151] When the capsule endoscope needs to be controlled to adjust its posture, if the capsule posture is adjusted to the left, the lens of the capsule endoscope will be facing left; if the capsule posture is adjusted upward, the lens of the capsule endoscope will be facing upward; the control effects of other directions are similar. At this time, the schematic diagram of the capsule endoscope adjusting the direction based on the first-person field of view is as follows Figure 7 As shown, if the posture of the capsule endoscope needs to be adjusted, based on the first viewing angle, along Figure 7 By adjusting the posture in 8 directions, the field of vision is expanded.

[0152] When the movement of the capsule endoscope needs to be controlled, if the capsule endoscope is moved to the left, the capsule endoscope moves to the left; if the capsule endoscope is moved upward, the capsule endoscope moves upward; the control effects in other directions are similar.

[0153] Therefore, when controlling the capsule endoscope to adjust its posture or move, the direction of adjustment or movement is consistent with the observation habits of human eyes, and the operating state is "what you think is what you get", achieving a stable capsule endoscope first-person perspective control effect.

[0154] Embodiment 2: Another control method of magnetically controlled capsule system

[0155] Example 2 Figures 8 to 14 As shown, other steps may be continued on the basis of the "first perspective" control in Example 1.

[0156] like Figure 8 As shown, the control method of the magnetically controlled capsule system of this embodiment further includes the following steps S40-S80 based on the steps S10-S30 of the embodiment 1:

[0157] Step S40: Acquire image data captured by the capsule endoscope.

[0158] exist Fig.10a and 10b In the embodiment, the capsule endoscope 10 is located in the lower part of the stomach cavity (sunken to the bottom), and the lens faces upward to take pictures; Fig.11a and 11b In the figure, the capsule endoscope 10 is located in the upper part of the stomach cavity (ceiling), and the lens faces downward to take pictures; Fig.10a and Fig.11a Corresponding to step S40 in the two states respectively. Fig.10b and Fig.11b These correspond to the following step S50 in the two states respectively.

[0159] Step S50: On the basis of keeping the position of the control magnet 21 unchanged, adjusting the posture of the control magnet 21 to obtain a plurality of peripheral image data outside the field of view corresponding to the image data.

[0160] Step S50 is to adjust the posture of the control magnet 21 on the basis of keeping the position of the control magnet 21 unchanged, so that the position of the capsule endoscope 10 remains unchanged and only the posture is adjusted; that is, step S50 is to quantitatively control the magnet posture data [Mh, Mv] of the control magnet 21 to change the capsule posture data [Ch, Cv] of the capsule endoscope 10.

[0161] The capsule endoscope 10 of this embodiment adopts a fisheye lens, that is, a wide-angle lens, with a large field of view (e.g., 140-160 degrees). The spherical scene is compressed and mapped to a plane image. The angular distribution of the image is shown in FIG. Fig.12 , where the value represents the angle away from the central axis of the field of view, and the square area is the actual image acquisition area of ​​the imaging unit.

[0162] like Figures 10a to 10b changes, and Figures 11a to 11b The magnet 21 is controlled to adjust its posture at the original position, so that the capsule endoscope 10 is in a different direction based on the original position. The way of controlling the magnet 21 to drive the lens to different directions is similar to a person controlling himself to stay in the same place but turn his brain to different directions. The direction of the brain movement is controlled based on the first-person perspective of the human eye. Therefore, in this embodiment, Figures 10a to 10b ,as well as Figures 11a to 11b The control process of the capsule endoscope 10 conforms to people's eye habits and is more convenient to operate.

[0163] Step S50 also includes: adjusting the posture of the control magnet 21 so that the field of view of the capsule endoscope 10 moves toward a target direction, wherein the target direction is a direction extending from the field of view corresponding to the image data to the field of view corresponding to the peripheral image data. Through such a field of view adjustment action, the capsule endoscope 10 can scan and shoot the desired target area.

[0164] And step S50 makes the newly acquired peripheral image data include images outside the field of view corresponding to the image data while retaining part of the images within the field of view corresponding to the image data.

[0165] In this embodiment, Fig.12 For example, the field of view angle shifts by 20 to 30 degrees, that is Fig.12 The translation is about 1 / 4 scale, so that the central area of ​​the image after translation can retain about 50% to 80% of the image content before translation. This can take into account both control efficiency and visual effects, and can significantly adjust the center position of the field of view without losing the reference target area, so as to achieve a better first-person perspective field of view translation control effect.

[0166] Further, Figure 7 and 13 As shown, in step S40, the image in the middle is acquired, and step S50 can continue to acquire multiple peripheral image data, including peripheral image data at eight positions above, below, left, right, upper left, upper right, lower left, and lower right of the image data, thereby expanding the small square area in the middle into a large square area.

[0167] Furthermore, if Fig. 9 As shown, step S50 includes the following steps S51-S54:

[0168] Step S51: obtaining the expected movement amount of the field of view of the capsule endoscope.

[0169] Step S52: Calculate the posture adjustment amount of the capsule endoscope according to the expected movement amount of the field of view.

[0170] Step S53: Calculating the posture adjustment amount of the control magnet according to the posture adjustment amount of the capsule endoscope.

[0171] Step S54: controlling the control magnet to adjust a corresponding change amount according to the attitude adjustment amount of the control magnet.

[0172] Specifically, in step S51, the expected movement of the field of view can be regarded as the translation amount [dh, dv] of the two-dimensional HV plane, where dh controls the horizontal translation amplitude of the field of view, and dv controls the vertical translation amplitude of the field of view. When the field of view of the capsule endoscope is adjusted according to the expected movement of the field of view, the center of the field of view of the lens can be directed toward the new target orientation to achieve arbitrary orientation amplitude adjustment of the field of view of the capsule endoscope, for example Fig.13 In the figure, the original field of view center of the lens is toward the middle box position, and the expected field of view movement amount can be used to adjust the field of view center of the lens toward the box on the left.

[0173] For example, take the above-mentioned field of view angle translation range of 20 degrees in the range of 20 to 30 degrees as an example, and it is necessary to make Fig.13 For the translation in the eight directions of up, down, left, right, upper left, upper right, lower left, and lower right, the expected movement of the field of view when the field of view shifts to the right is [20,0], the expected movement of the field of view when the field of view shifts to the left is [-20,0], the expected movement of the field of view when the field of view shifts upward is [0,-20], the expected movement of the field of view when the field of view shifts downward is [0,20], the expected movement of the field of view when the field of view shifts to the lower left is [-20,20], and so on.

[0174] In step S52, assuming that the current capsule posture data is [Ch, Cv], the required posture adjustment amount of the capsule endoscope is [rh, rv], in order to achieve the expected field of view movement amount [dh, dv] in the above HV plane, the calculation formula of the three is as follows:

[0175]

[0176] Among them, the Euler angle used for auxiliary calculation is:

[0177]

[0178] After the posture adjustment amount [rh, rv] of the capsule endoscope is calculated according to the above formula, the posture adjustment amount of the capsule endoscope after adjustment is [Ch+rh, Cv+rv].

[0179] For another example, if the current capsule posture data [Ch, Cv] = [50, 40], the expected movement of the field of view adjusted to the left [dh, dv] = [-20, 0], and the corresponding capsule endoscope posture adjustment amount [rh, rv] = [-29.5, 4.0], then the adjusted capsule endoscope posture data is [20.5, 44.0]. If the expected movement of the field of view adjusted upward [dh, dv] = [0, -20], the corresponding capsule endoscope posture adjustment amount [rh, rv] = [0, -20], the adjusted capsule endoscope posture data is [50, 20].

[0180] In step S53, the attitude adjustment amount of the control magnet can be calculated based on the attitude adjustment amount [rh, rv] of the capsule endoscope through the angle control model of the control magnet, and the value of the attitude data [Mh, Mv] of the control magnet can be adjusted.

[0181] Then, in step S54, the relative angle translation of the expected movement amount [dh, dv] of the capsule field of view under the first viewing angle can be achieved, the center of the field of view is adjusted to the target area, and then multiple peripheral image data are acquired in sequence, for example Fig.13 The moving trajectory shown achieves stable digestive tract wall scanning and shooting in accordance with the operator's expected time sequence correspondence, avoiding the difficulty of determining the location of a local image that lacks image context.

[0182] Step S60: splicing the image data and the peripheral image data to obtain extended image data.

[0183] This step can be combined with existing image registration and image stitching technology to expand the observation field of the target area and expand the image data. Fig.14 As shown, smaller image data are spliced ​​into larger expanded image data.

[0184] Step S70: controlling the control magnet to move so that the capsule endoscope moves to the next control node.

[0185] In the process of moving to the next control node, the forward and backward quantitative control can be performed based on the first viewing angle of the capsule endoscope in the above embodiment 1; or the control can be performed based on the calculated coordinate values ​​in the following embodiment 3.

[0186] The manner of controlling the capsule endoscope to move to the next control node may include dragging, rolling or jumping, among which rolling and jumping control can realize quantitative control with higher quality.

[0187] Taking tumbling as an example, the tumbling action realizes the displacement of the capsule endoscope forward and backward by about the circumference of the long axis of the capsule endoscope. If a larger range of transfer is required, the tumbling action can be performed in multiple steps, that is, moving a distance of n circumferences. For a smaller range of transfer, it can be achieved by two-step zigzag path tumbling. After the tumbling action is performed normally, the control magnet will be located near the top of the new position of the capsule endoscope, so the first-angle control adjustment of the field of view image acquisition of the above steps S40 to S60 can be performed continuously. If the execution fails, you can start again from the above step S20, and restore the control magnet to reach the top of the capsule endoscope and then perform the tumbling action again.

[0188] Taking jumping as an example, the jumping action is combined with the above formula 2, and the quantitative offset of the XY plane and the height adjustment of the Z direction of the control magnet are used to suck up and drop the capsule endoscope. The centripetal characteristics of the magnetic field suction force are used to achieve the capsule endoscope to follow the position of the control magnet and cross larger obstacles. After the jumping action is performed normally, the control magnet is located near the top of the new position of the capsule endoscope, and the first-angle control adjustment of the field of view image acquisition of steps S40 to S60 can be performed continuously. Similarly, if the execution fails, you can start again from the above step S20, and restore the control magnet to reach the top of the capsule endoscope and perform the jumping action again.

[0189] Step S80: On the basis of continuously executing step S10, looping steps S20 to S70 until the inspection is completed.

[0190] Because the water-injected or air-inflated stomach has an irregular 3D cavity structure, the depth of field of the capsule endoscope is limited and it can only obtain clearer images of the stomach wall in local areas. It is necessary to scan at multiple different nodes to achieve a high-quality, complete stomach wall examination without omissions.

[0191] By moving the capsule endoscope to multiple different control nodes, taking pictures and stitching the above steps S40 to S60 are performed at each control node, and the capsule endoscope control based on the first-person perspective is in line with the habits of the human eye, and images of various areas in the complete digestive tract are obtained, thereby improving inspection efficiency.

[0192] The judgment condition for the end of the inspection can be determined based on the completion of the inspection, or the inspection completion time, inspection completion instructions, etc.

[0193] Embodiment 3: Positioning method of magnetically controlled capsule system

[0194] The difference between Example 3 and Example 1 is that Example 1 is for controlling the subsequent movement of the capsule endoscope based on the "first perspective", while Example 3 is for obtaining the position information of the capsule endoscope without the positioning device providing positioning data.

[0195] like Fig.15 As shown, the positioning method includes the following steps:

[0196] Step S10: continuously acquiring measurement status data of the capsule endoscope, wherein the measurement status data includes acceleration data and / or magnetic field status data.

[0197] Step S20: Control the control magnet to move at the same height, and adjust the posture of the control magnet until the magnetic field state data is maximum, and / or the vertical tilt angle of the capsule endoscope is minimum, wherein the vertical tilt angle of the capsule is calculated based on the acceleration data.

[0198] Step S10 ′: acquiring magnet state data of the control magnet, wherein the magnet state data includes magnet position data and magnet posture data.

[0199] The step S10' can be performed simultaneously with the step S10, or the acquisition can be continued all the time.

[0200] Step S21: determining the capsule position data of the capsule endoscope according to the current magnet position data.

[0201] After step S20, the magnet is controlled to reach the top of the capsule endoscope. At this time:

[0202] The x-axis coordinate of the capsule endoscope is the same as the x-axis coordinate of the capsule endoscope;

[0203] The y-axis coordinate of the capsule endoscope is the same as the y-axis coordinate of the capsule endoscope;

[0204] The z-axis coordinate of the capsule endoscope is calculated according to the magnetic field state data.

[0205] For example, if Mx and My are both 0, then Cx=Cy=0, and Cz=-Z0, wherein the calculation formula of Z0 refers to the above formula 2.

[0206] Step S22: determining capsule posture data of the capsule endoscope according to the current measurement state data and the magnet posture data.

[0207] Here, the capsule posture data includes the capsule horizontal azimuth angle, the capsule vertical tilt angle, and the capsule spin angle, and the magnet posture data includes the magnet horizontal azimuth angle;

[0208] When the magnetic field state data is maximum and / or the capsule vertical tilt angle of the capsule endoscope is minimum, the capsule vertical tilt angle is zero.

[0209] At this time, the vertical inclination angle Cv of the capsule does not have to be strictly 0 according to the above description, and can be a value close to 0.

[0210] When the control magnet reaches the top of the capsule endoscope, the capsule endoscope is almost parallel to the Z axis. At this time, the horizontal azimuth angle of the capsule can be any value, which has no practical significance or impact.

[0211] The step of determining the capsule posture data of the capsule endoscope according to the current measurement state data and the magnet posture data also includes:

[0212] The capsule spin angle is calculated based on the acceleration data.

[0213] After the magnetic field state data is maximum and / or the capsule vertical tilt angle of the capsule endoscope is minimum, the following steps are performed:

[0214] Adjusting the posture of the control magnet so that the vertical tilt angle of the capsule is not zero;

[0215] The capsule horizontal azimuth angle is equal to the magnet horizontal azimuth angle.

[0216] At this time, the capsule vertical tilt angle Cv can be calculated according to the above formula 1, the capsule spin angle Cs can be calculated according to the above formula 3, and the capsule horizontal azimuth angle Ch approaches the horizontal azimuth angle Mh of the magnet.

[0217] In addition, the dynamic accuracy of the capsule vertical tilt angle Cv and the capsule spin angle Cs can be enhanced based on the complementary filtering of [wx, wy, wz] obtained by the gyroscope and [gx, gy, gz] obtained by the accelerometer, thereby reducing the limitation on the relative stillness of the capsule endoscope.

[0218] Therefore, in Example 3, without the need for positioning equipment, the magnet can be controlled to reach directly above the capsule endoscope, and then on this basis, the 6DOF complete state parameters [Cx, Cy, Cz, Ch, Cv, Cs] of the capsule endoscope can be calculated, thereby satisfying the positioning requirements when the equipment conditions are low, facilitating the subsequent more accurate scanning control of the capsule endoscope from the first-person perspective, optimizing the shooting angle and distance of the capsule endoscope, and giving full play to the optimal performance of the capsule endoscope imaging hardware system.

[0219] Compared with the prior art, this embodiment has the following beneficial effects:

[0220] (1) The control method of the magnetically controlled capsule system can be applied without relying on positioning equipment, only relying on the measurement status data obtained by the capsule endoscope, so that the subsequent movement direction of the capsule endoscope can be consistent with the direction on the acquired image. In other words, the capsule endoscope can be controlled based on the first-person perspective of the image of the capsule endoscope, and the movement or posture adjustment of the capsule endoscope can be controlled in a "what you think is what you get" form, so that the capsule endoscope can be accurately controlled, reducing the demand on the ability of doctors, meeting the ability conditions of more doctors, and then facilitating the promotion and application of the magnetically controlled capsule system.

[0221] (2) By controlling the capsule endoscope based on the first-person perspective of the capsule endoscope image, images within the required field of view can be obtained, thereby improving the efficiency of the inspection, avoiding the problem of missed inspection, and meeting the needs of physicians.

[0222] (3) Without relying on positioning equipment, the capsule endoscope is positioned by first controlling the magnet to reach directly above the capsule endoscope, thus meeting the demand for precise control.

[0223] (4) Optimize the shooting angle and distance of the capsule endoscope without relying on positioning equipment, give full play to the optimal performance of the capsule endoscope imaging hardware system, improve the image quality of gastrointestinal examinations, improve the accuracy of physicians' film reading results, and help improve the treatment level of patients.

[0224] In one embodiment, a control device for a magnetically controlled capsule system is provided. The module schematic diagram of the control device is as follows: Fig.16 The control device may include modules, and the specific functions of each module are as follows:

[0225] A capsule state acquisition module, used for continuously acquiring measurement state data of the capsule endoscope, wherein the measurement state data includes acceleration data and / or magnetic field state data;

[0226] a posture adjustment module, used for controlling the control magnet to move at the same height, and adjusting the posture of the control magnet until the magnetic field state data is maximum, and / or the capsule vertical tilt angle of the capsule endoscope is minimum, wherein the capsule vertical tilt angle is calculated according to the acceleration data;

[0227] The correction module is used to correct the capsule rotation angle of the capsule endoscope so that the direction of the capsule endoscope when it moves or adjusts its posture is consistent with the direction on the image acquired by the capsule endoscope.

[0228] In one embodiment, a control device for a magnetically controlled capsule system is provided, which, in addition to the modules in the above embodiment, also includes a module schematic diagram as shown in FIG. Fig.17The modules that the positioning device may include and the specific functions of each module are as follows:

[0229] An image acquisition module, used to acquire image data taken by the capsule endoscope;

[0230] A visual field adjustment module, used to adjust the posture of the control magnet while maintaining the position of the control magnet unchanged, so as to obtain a plurality of peripheral image data outside the visual field corresponding to the image data;

[0231] The splicing module is used to splice the image data and the peripheral image data to obtain extended image data.

[0232] In one embodiment, a positioning device for a magnetically controlled capsule system is provided. The module schematic diagram of the positioning device is as follows: Fig.18 The modules that the positioning device may include and the specific functions of each module are as follows:

[0233] A capsule state acquisition module, used for continuously acquiring measurement state data of the capsule endoscope, wherein the measurement state data includes acceleration data and / or magnetic field state data;

[0234] a posture adjustment module, used for controlling the control magnet to move within the same height, and adjusting the posture of the control magnet until the magnetic field state data is maximum, and / or the capsule vertical tilt angle of the capsule endoscope is minimum, wherein the capsule vertical tilt angle is calculated according to the acceleration data;

[0235] A magnet state acquisition module, used for acquiring magnet state data of the control magnet, wherein the magnet state data includes magnet position data and magnet posture data;

[0236] A capsule position calculation module, used to determine capsule position data of the capsule endoscope according to the current magnet position data;

[0237] The capsule posture calculation module is used to determine the capsule posture data of the capsule endoscope according to the current measurement state data and the magnet posture data.

[0238] It should be noted that for details not disclosed in the control device or positioning device of the embodiments of the present invention, please refer to the details disclosed in the control method or positioning method of the embodiments of the present invention.

[0239] The magnetically controlled capsule system 100 of this embodiment is as follows Fig.19As shown, the capsule endoscope 10 may include a magnetic control system 20 and a capsule endoscope 10. In addition to the above-mentioned camera unit 12, permanent magnet unit 15, and inertial measurement unit 11, the capsule endoscope 10 also includes a data processing and transmission unit 13 that is communicatively connected to the camera unit 12, and a power supply unit 14 that supplies power to the above-mentioned active components. The data processing and transmission unit 13 transmits information to an external processing module 25 or server. The control magnet 21 drives the capsule endoscope 10 to move. After the capsule endoscope 10 moves to a specified position, the camera unit 12 takes pictures of the digestive tract, and transmits them to the outside world through the data processing and transmission unit 13, completing the shooting of the body.

[0240] The magnetic control system 20 includes a control magnet 21, a motor drive device 22, a signal transmission module 23, a storage module 24, and a processing module 25. The motor drive device 22 drives the control magnet 21 to move. The signal transmission module 23 and the data processing and transmission unit 13 can transmit data in the form of wireless connection, such as Bluetooth, WiFi, Zigbee, etc. The communication bus 26 is used to establish a connection between the motor drive device 22, the signal transmission module 23, the processing module 25 and the storage module 24. The communication bus 26 may include a passage to transmit information between the above-mentioned motor drive device 22, the signal transmission module 23, the processing module 25 and the storage module 24.

[0241] The magnetically controlled capsule system 100 may also include computing devices such as computers, notebooks, PDAs, and cloud servers, as well as including but not limited to a processing module 25, a storage module 24, and a computer program stored in the storage module 24 and executable on the processing module 25, such as the program of the control method or positioning method described above. When the processing module 25 executes the computer program, the steps in the above-mentioned control method embodiments are implemented, such as FIG. 2, Figure 8 and Fig.15 Steps shown.

[0242] In addition, the present invention also proposes an electronic device, which includes a storage module 24 and a processing module 25. When the processing module 25 executes the computer program, it can implement the steps in the above-mentioned control method or positioning method, that is, implement the steps in any one of the technical solutions in the above-mentioned control method or positioning method.

[0243] The electronic device may be a part integrated into the magnetically controlled capsule system 100 , or a local terminal device, or a part of a cloud server.

[0244] The processing module 25 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processing module 25 is the control center of the magnetically controlled capsule system 100, and uses various interfaces and lines to connect various parts of the entire magnetically controlled capsule system 100.

[0245] The storage module 24 can be used to store the computer program and / or module, and the processing module 25 realizes various functions of the magnetically controlled capsule system 100 by running or executing the computer program and / or module stored in the storage module 24, and calling the data stored in the storage module 24. The storage module 24 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc. In addition, the storage module 24 can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0246] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the storage module 24 and executed by the processing module 25 to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program in the magnetically controlled capsule system 100.

[0247] Furthermore, an embodiment of the present invention provides a readable storage medium storing a computer program, which, when executed by the processing module 25, can implement the steps in the above-mentioned control method or positioning method, that is, implement the steps in any one of the technical solutions in the above-mentioned control method or positioning method.

[0248] If the control device and / or positioning device integrated module of the magnetically controlled capsule system 100 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processing module 25, the steps of each of the above-mentioned method embodiments can be implemented.

[0249] The computer program includes computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, recording medium, disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer readable media do not include electric carrier signals and telecommunication signals.

[0250] It should be understood that although this specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each implementation mode may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

[0251] The series of detailed descriptions listed above are only specific descriptions of feasible implementation methods of the present invention. They are not intended to limit the scope of protection of the present invention. Any equivalent implementation methods or changes that do not deviate from the technical spirit of the present invention should be included in the scope of protection of the present invention.

Claims

1. A control method for a magnetically controlled capsule system, the magnetically controlled capsule system comprising a control magnet and a capsule endoscope, characterized in that: The control method comprises the following steps: Step S10: continuously acquiring measurement status data of the capsule endoscope, wherein the measurement status data includes acceleration data and / or magnetic field status data; Step S20: controlling the control magnet to move at the same height, and adjusting the position of the control magnet until the magnetic field state data is maximum, and / or the capsule vertical tilt angle of the capsule endoscope is minimum, wherein the capsule vertical tilt angle is calculated according to the acceleration data; Step S30: calibrating the capsule rotation angle of the capsule endoscope so that the direction of the capsule endoscope when it moves or adjusts its posture is consistent with the direction on the image acquired by the capsule endoscope.

2. The control method of the magnetically controlled capsule system according to claim 1, characterized in that: The step S20 comprises: Controlling the magnetization direction of the control magnet to be in a vertical direction; The control magnet is controlled to translate within a plane perpendicular to the vertical direction while keeping the magnetization direction unchanged, wherein the translation is a movement in a direction that gradually reduces the vertical tilt angle of the capsule and / or gradually increases the magnetic field state data.

3. The control method of the magnetically controlled capsule system according to claim 2, characterized in that: The step of controlling the translation of the control magnet comprises: The movement of the control magnet is controlled in coordination with the magnetic field state data and the vertical tilt angle of the capsule.

4. The control method of the magnetically controlled capsule system according to claim 3, characterized in that: The step of controlling the movement of the control magnet in coordination with the magnetic field state data and the vertical tilt angle of the capsule comprises: In the process of controlling the movement of the control magnet, if the vertical tilt angle of the capsule does not change, continuously moving the control magnet until the vertical tilt angle of the capsule changes; In the process of controlling the movement of the control magnet, if the vertical tilt angle of the capsule increases and the magnetic field state data decreases, the control magnet is controlled to move in the opposite direction.

5. The control method of the magnetically controlled capsule system according to claim 1, characterized in that: After step S20, the method further includes the following steps: Calculating the height of the capsule endoscope according to the magnetic field state data; The height of the control magnet is adjusted so that the height of the capsule endoscope is within a set range.

6. The control method of the magnetically controlled capsule system according to claim 1, characterized in that: Also includes the steps: Step S40: Acquire image data captured by the capsule endoscope; Step S50: adjusting the posture of the control magnet while keeping the position of the control magnet unchanged, so as to obtain a plurality of peripheral image data outside the field of view corresponding to the image data; Step S60: splicing the image data and the peripheral image data to obtain extended image data.

7. The control method of the magnetically controlled capsule system according to claim 6, characterized in that: The step S50 comprises: On the basis of keeping the position of the control magnet unchanged, adjusting the posture of the control magnet so that the position of the capsule endoscope remains unchanged and only the posture is adjusted; The posture of the control magnet is adjusted to move the capsule endoscope toward a target direction, wherein the target direction is a direction extending from a field of view corresponding to the image data to a field of view corresponding to the peripheral image data.

8. The control method of the magnetically controlled capsule system according to claim 7, characterized in that: The step S50 further includes: The posture of the control magnet is adjusted so that the newly acquired peripheral image data includes images outside the field of view corresponding to the image data while retaining a portion of the images within the field of view corresponding to the image data.

9. The control method of the magnetically controlled capsule system according to claim 8, characterized in that: The step of adjusting the posture of the control magnet comprises: Acquiring an expected movement amount of the field of view of the capsule endoscope; Calculating a posture adjustment amount of the capsule endoscope according to the expected movement amount of the visual field; Calculating the posture adjustment amount of the control magnet according to the posture adjustment amount of the capsule endoscope; The control magnet is controlled to adjust a corresponding change amount according to the attitude adjustment amount of the control magnet.

10. The control method of the magnetically controlled capsule system according to claim 8, characterized in that: The plurality of peripheral image data include peripheral image data at eight positions above, below, left, right, upper left, upper right, lower left, and lower right of the image data.

11. The control method of the magnetically controlled capsule system according to claim 6, characterized in that: Also includes the steps: Step S70: controlling the control magnet to move so that the capsule endoscope moves to the next control node; Step S80: On the basis of continuously executing step S10, looping steps S20 to S70 until the inspection is completed.

12. A positioning method for a magnetically controlled capsule system, the magnetically controlled capsule system comprising a control magnet and a capsule endoscope, characterized in that: The positioning method comprises the following steps: Continuously acquiring measurement status data of the capsule endoscope, wherein the measurement status data includes acceleration data and / or magnetic field status data; Controlling the control magnet to move at the same height, and adjusting the position of the control magnet until the magnetic field state data is maximum, and / or the capsule vertical tilt angle of the capsule endoscope is minimum, wherein the capsule vertical tilt angle is calculated according to the acceleration data; Acquiring magnet state data of the control magnet, wherein the magnet state data includes magnet position data and magnet posture data; Determining capsule position data of the capsule endoscope according to the current magnet position data; The capsule posture data of the capsule endoscope is determined according to the current measurement state data and the magnet posture data.

13. The positioning method of the magnetically controlled capsule system according to claim 12, characterized in that: The step of determining the capsule position data of the capsule endoscope according to the current magnet position data comprises: The x-axis coordinate of the capsule endoscope is the same as the x-axis coordinate of the capsule endoscope; The y-axis coordinate of the capsule endoscope is the same as the y-axis coordinate of the capsule endoscope; The z-axis coordinate of the capsule endoscope is calculated according to the magnetic field state data.

14. The positioning method of the magnetically controlled capsule system according to claim 12, characterized in that: The capsule posture data includes the capsule horizontal azimuth angle, the capsule vertical tilt angle, and the capsule spin angle, and the magnet posture data includes the magnet horizontal azimuth angle; When the magnetic field state data is maximum and / or the capsule vertical tilt angle of the capsule endoscope is minimum, the capsule vertical tilt angle is zero; The step of determining the capsule posture data of the capsule endoscope according to the current measurement state data and the magnet posture data also includes: Calculating the capsule spin angle according to the acceleration data; After the magnetic field state data is maximum and / or the capsule vertical tilt angle of the capsule endoscope is minimum, the following steps are performed: Adjusting the posture of the control magnet so that the vertical tilt angle of the capsule is not zero; The capsule horizontal azimuth angle is equal to the magnet horizontal azimuth angle.

15. A control device for a magnetically controlled capsule system, the magnetically controlled capsule system comprising a control magnet and a capsule endoscope, characterized in that: The control device comprises: A capsule state acquisition module, used for continuously acquiring measurement state data of the capsule endoscope, wherein the measurement state data includes acceleration data and / or magnetic field state data; a posture adjustment module, used for controlling the control magnet to move at the same height, and adjusting the posture of the control magnet until the magnetic field state data is maximum, and / or the capsule vertical tilt angle of the capsule endoscope is minimum, wherein the capsule vertical tilt angle is calculated according to the acceleration data; The correction module is used to correct the capsule rotation angle of the capsule endoscope so that the direction of the capsule endoscope when it moves or adjusts its posture is consistent with the direction on the image acquired by the capsule endoscope.

16. A positioning device for a magnetically controlled capsule system, the magnetically controlled capsule system comprising a control magnet and a capsule endoscope, characterized in that: The positioning device comprises: A capsule state acquisition module, used for continuously acquiring measurement state data of the capsule endoscope, wherein the measurement state data includes acceleration data and / or magnetic field state data; a posture adjustment module, used for controlling the control magnet to move within the same height, and adjusting the posture of the control magnet until the magnetic field state data is maximum, and / or the capsule vertical tilt angle of the capsule endoscope is minimum, wherein the capsule vertical tilt angle is calculated according to the acceleration data; A magnet state acquisition module, used for acquiring magnet state data of the control magnet, wherein the magnet state data includes magnet position data and magnet posture data; A capsule position calculation module, used to determine capsule position data of the capsule endoscope according to the current magnet position data; The capsule posture calculation module is used to determine the capsule posture data of the capsule endoscope according to the current measurement state data and the magnet posture data.

17. An electronic device, characterized in that: include: A storage module storing a computer program; The processing module can implement the steps of the control method of the magnetically controlled capsule system or the positioning method of the magnetically controlled capsule system as described in any one of claims 1 to 14 when executing the computer program.

18. A readable storage medium storing a computer program, characterized in that: When the computer program is executed by the processing module, the steps in the control method of the magnetically controlled capsule system or the positioning method of the magnetically controlled capsule system described in any one of claims 1 to 14 can be implemented.

Images