Capsule endoscope control method, device and control system

Abstract

The embodiments of the present application are applicable to the field of computer-assisted medical technology, and provide a method, device and control system for controlling a capsule endoscope, the method comprising: displaying an image captured by the capsule endoscope in a display device; determining the image coordinates of the position point in the image in response to the operation of clicking any position point in the image; determining the target vector for controlling the movement of the capsule endoscope based on the image coordinates of the position point, the direction of the target vector being the target direction to be turned by the capsule endoscope; determining the current working mode of the capsule endoscope, and controlling the movement of an external magnetic source according to the working mode and the target direction, so as to realize the control of the capsule endoscope. By adopting the above method, the movement direction of the capsule endoscope can be updated by clicking on the image, and the direction of driving the capsule endoscope can be freely selected without knowing the actual position of the capsule endoscope during the control process, thereby reducing the difficulty of controlling the capsule endoscope.

Description

Technical Field

[0001] The embodiments of the present application belong to the field of computer-assisted medical technology, and in particular, relate to a control method, device, and control system for a capsule endoscope. Background Art

[0002] Capsule endoscope is a medical device used for human digestive tract examination, which has the advantage of non-invasive examination. Capsule endoscope is about the same size as ordinary capsule, can be swallowed into the human digestive tract, collect images of the digestive tract and send them to external devices, and doctors can make diagnoses based on the images collected by the capsule endoscope.

[0003] Capsule endoscopes mainly include passive and active types. Among them, the passive capsule endoscope moves with the peristalsis of the human body's gastrointestinal tract and takes images of the inside of the digestive tract during the movement for doctors to diagnose. However, this diagnostic method of randomly taking images of the digestive tract is prone to missing lesions, and because the movement of the capsule endoscope is uncontrolled, doctors cannot make detailed observations of areas of interest inside the digestive tract.

[0004] Active capsule endoscopes are mainly controlled by doctors using operating devices such as external handles. Doctors use operating devices to control the movement and rotation of the external magnetic source, thereby controlling the capsule endoscope to move in different directions, capture images of different positions inside the digestive tract, and observe specific positions. However, the control effect achieved by manual operation in the prior art is poor, the efficiency is low, and certain operating experience is required to complete. Summary of the invention

[0005] In view of this, the embodiments of the present application provide a method, device and system for controlling a capsule endoscope, so as to reduce the difficulty of controlling the capsule endoscope and improve the control efficiency.

[0006] A first aspect of an embodiment of the present application provides a method for controlling a capsule endoscope, comprising:

[0007] Displaying the image captured by the capsule endoscope on a display device;

[0008] In response to an operation of clicking any position point in the image, determining the image coordinates of the position point in the image;

[0009] Determining a target vector for controlling the movement of the capsule endoscope based on the image coordinates of the position point, wherein the direction of the target vector is the target direction in which the capsule endoscope is to be turned;

[0010] The current working mode of the capsule endoscope is determined, and according to the working mode and the target direction, the movement of the external magnetic source is controlled to achieve manipulation of the capsule endoscope.

[0011] A second aspect of an embodiment of the present application provides a capsule endoscope control device, comprising:

[0012] A capsule image display module, used to display images captured by the capsule endoscope on a display device;

[0013] An icon coordinate determination module, configured to determine the image coordinates of any position point in the image in response to an operation of clicking the position point in the image;

[0014] A motion vector determination module, used for determining a target vector for controlling the motion of the capsule endoscope based on the image coordinates of the position point, wherein the direction of the target vector is the target direction in which the capsule endoscope is to be turned;

[0015] The capsule control module is used to determine the current working mode of the capsule endoscope, and control the movement of the external magnetic source according to the working mode and the target direction to realize the control of the capsule endoscope.

[0016] A third aspect of the embodiments of the present application provides a capsule endoscope control system, including a display device, a capsule positioning module and a control unit; wherein:

[0017] The display device is used to display the image captured by the capsule endoscope;

[0018] The control unit is used to determine the image coordinates of any position point in the image in response to an operation of clicking the position point; determine a target vector for controlling the movement of the capsule endoscope based on the image coordinates of the position point, wherein the direction of the target vector is the target direction to which the capsule endoscope is to turn; determine the current working mode of the capsule endoscope, and control the movement of an external magnetic source according to the working mode and the target direction to realize the control of the capsule endoscope.

[0019] A fourth aspect of an embodiment of the present application provides a capsule endoscope control device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the capsule endoscope control method as described in the first aspect above is implemented.

[0020] A fifth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for controlling a capsule endoscope as described in the first aspect above is implemented.

[0021] A sixth aspect of the embodiments of the present application provides a computer program product. When the computer program product runs on a computer, the computer executes the method for controlling a capsule endoscope as described in the first aspect.

[0022] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0023] In the embodiment of the present application, by displaying the image captured by the capsule endoscope in the display device, the image in the digestive tract can be displayed to the operator in real time. The control method provided in the embodiment of the present application adopts the method of clicking the image to update the target direction of the capsule endoscope. The control unit can determine the target vector for controlling the movement of the capsule endoscope by determining the coordinates of the position point in the image clicked by the operator, thereby obtaining the target direction to be turned to by the capsule endoscope. Then, according to the working mode of the capsule endoscope and the determined target direction, the capsule endoscope can be controlled by controlling the movement of the external magnetic source. By applying the control method provided in the embodiment of the present application, it is not necessary to know the actual position of the capsule endoscope during the control process of the capsule endoscope, and the direction of driving the capsule endoscope can be freely selected, which is conducive to the detailed observation of the lesion location. At the same time, the control of the capsule endoscope by actively clicking the image is simpler and more convenient than the operation using external devices such as handles, which reduces the difficulty of controlling the capsule endoscope and improves the control efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or prior art descriptions. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 is a schematic diagram of a capsule endoscope control system provided in an embodiment of the present application;

[0026] Figure 2 is a schematic diagram of a method for controlling a capsule endoscope provided in an embodiment of the present application;

[0027] Figure 3 is a schematic diagram of a possible implementation of S200 in a method for controlling a capsule endoscope provided in an embodiment of the present application;

[0028] Figure 4 is a schematic diagram of a possible implementation of S203 in a method for controlling a capsule endoscope provided in an embodiment of the present application;

[0029] Figure 5is a schematic diagram of a possible implementation of S204 in a method for controlling a capsule endoscope provided in an embodiment of the present application;

[0030] Figure 6 It is a flowchart of a method for controlling a capsule endoscope provided in an embodiment of the present application;

[0031] Figure 7 is a schematic diagram of a control device for a capsule endoscope provided in an embodiment of the present application;

[0032] Figure 8 It is a schematic diagram of a capsule endoscope control device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] In the following description, specific details such as specific system structures, technologies, etc. are proposed for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from hindering the description of the present application.

[0034] The technical solution of the present application is described below through specific embodiments.

[0035] Reference Figure 1 , showing a schematic diagram of a capsule endoscope control system provided in an embodiment of the present application, the system may include Figure 1 The system can be used to display the display device 101, the capsule positioning module 102 and the control unit (not shown in the figure). Figure 1 The capsule endoscope 103 is manipulated to control the movement of the capsule endoscope 103 in the human digestive tract.

[0036] Specifically, the display device 101 is used to display images captured by the capsule endoscope 103 .

[0037] The control unit is used to determine the image coordinates of any position point in the image in response to the operation of clicking on the position point; determine the target vector for controlling the movement of the capsule endoscope 103 based on the image coordinates of the position point, and the direction of the target vector is the target direction to which the capsule endoscope 103 is to turn; determine the current working mode of the capsule endoscope 103, and control the movement of the external magnetic source 104 according to the working mode and the target direction to realize the control of the capsule endoscope 103.

[0038] In a possible implementation of the embodiment of the present application, the display device 101 and the control unit can be integrated into the same device. Exemplarily, the display device 101 and the control unit can be components of an integrated computer device, and the display screen of the computer device can be used as the display device 101 to display the image captured by the capsule endoscope 103; the control unit can be a control unit in the computer device, which is used to implement various algorithms, such as calculation of rotation angle and movement distance, output of control instructions, etc. In addition, in addition to the control unit integrated in the computer device, the control unit can also include other auxiliary operating components, such as a control unit for controlling Figure 1 The mechanical arm of the extracorporeal magnetic source 104 (not shown in the figure), etc., is not limited to this embodiment of the present application.

[0039] like Figure 1 As shown, the capsule endoscope 103 may include a wireless module 1031, a capsule magnet 1032, an orientation measurement module 1033 and a data acquisition module 1034. Among them, the wireless module 1031 can be used to send the data collected by the data acquisition module 1034 to the control unit, and the data collected by the data acquisition module 1034 may include the captured image, its own orientation data, etc. The capsule magnet 1032 can generate a force to drive the capsule endoscope 103 to move under the action of an external magnetic source, thereby driving the capsule endoscope 103 to rotate or move. The orientation measurement module 1033 can be used to measure parameters such as the movement direction of the capsule endoscope 103 in the human digestive tract, and transmit them to the control unit outside the body through the wireless module 1031.

[0040] like Figure 1 As shown, the image of the human digestive tract collected by the data acquisition module 1034 can be sent to the outside of the body through the wireless module 1031 and displayed on the display device 101 outside the body. The doctor can directly select any position point in the image based on the displayed image. The control unit can control the rotation or movement of the external magnetic source 104 by executing the corresponding algorithm, thereby driving the movement of the capsule endoscope 103 in the digestive tract.

[0041] In the embodiment of the present application, the position information, posture information, etc. required in the process of controlling the capsule endoscope 103 can be obtained by Figure 1 The capsule positioning module 102 in the embodiment can be obtained, and the capsule positioning module 102 can be a magnetic field array installed under the examination bed. When the patient swallows the capsule endoscope 102 and lies on the examination bed, the capsule positioning module 102 can locate the position of the capsule endoscope 103 in the digestive tract.

[0042] The capsule endoscope control method provided in the embodiment of the present application can be Figure 1 The control method is implemented in the control system shown in FIG.

[0043] Reference Figure 2 , shows a schematic diagram of a method for controlling a capsule endoscope provided in an embodiment of the present application, which may specifically include the following steps:

[0044] S201. Displaying images captured by the capsule endoscope on a display device.

[0045] It should be noted that the present method can be applied to the control system introduced in the aforementioned embodiment, and each device, module and unit in the control system can jointly implement the present method by executing corresponding steps.

[0046] In the embodiment of the present application, the image captured by the capsule endoscope can be Figure 1 The display device 101 in the capsule endoscope is displayed. Specifically, the data acquisition module in the capsule endoscope can be used to acquire images during the working process, and the acquired images can be sent to the outside of the body through the wireless module and displayed by the display device. The above process can be realized after the capsule endoscope is swallowed by the patient and the capsule endoscope is started.

[0047] In a possible implementation of the embodiment of the present application, a capsule endoscope may be configured with an orientation measurement module, for example Figure 1 The orientation measurement module 1033 in the capsule endoscope. Before the capsule endoscope works, in order to ensure the accuracy of the device operation, the orientation measurement module can be calibrated first. Therefore, before displaying the image captured by the capsule endoscope on the display device in S201, the method can also include S200:

[0048] S200: When the capsule endoscope is started, calibrate the orientation measurement module in the capsule endoscope to obtain a calibration result.

[0049] In a possible implementation of the embodiment of the present application, as Figure 3 As shown, in S200, the orientation measurement module in the capsule endoscope is calibrated to obtain a calibration result, which may specifically include the following steps S2001-S2003:

[0050] S2001. When the capsule endoscope is started, the position information of the capsule endoscope is obtained through the capsule positioning module outside the body, and the position information includes the current direction vector of the capsule endoscope and the quaternion posture of the orientation measurement module expressed as a quaternion.

[0051] In the embodiment of the present application, when the capsule endoscope is started, Figure 1The capsule positioning module 102 outside the body can position the capsule endoscope and obtain the position information of the capsule endoscope. The position information can include not only the current direction vector of the capsule endoscope itself, but also the position information of the orientation measurement module in the capsule endoscope. The direction vector of the capsule endoscope is used to indicate the current direction of the capsule endoscope, and the position information of the orientation measurement module can be the quaternion attitude of the orientation measurement module represented by quaternion.

[0052] S2002. Rotate the unit vector corresponding to the current direction vector of the capsule endoscope based on the quaternion posture of the orientation measurement module.

[0053] In the embodiment of the present application, the unit vector corresponding to the current direction vector of the capsule endoscope can be expressed as (0, 0, 1). During the calibration process, the unit vector can be rotated using a quaternion.

[0054] S2003. Calculate the angle between the rotated unit vector and the current direction vector of the capsule endoscope to obtain the calibration result.

[0055] In an embodiment of the present application, after rotating the unit vector represented by a quaternion, the angle between the rotated unit vector and the current direction vector of the capsule endoscope can be calculated. The angle is the calibration value, which can be used to compensate the data in subsequent coordinate conversion and other processing processes.

[0056] After completing the calibration of the orientation measurement module, the control unit can start the capsule endoscope, and the data acquisition module in the capsule endoscope performs image acquisition, and the acquired image can be displayed in real time on a display device outside the body.

[0057] S202: In response to an operation of clicking any position point in the image, determine the image coordinates of the position point in the image.

[0058] In the embodiment of the present application, for the real-time displayed image, the doctor or other operators can directly click in the image, and the position corresponding to the click operation is the position to which the operator expects to drive the capsule endoscope. In order to achieve the above purpose, when the operator clicks a certain position point in the image, the control unit can determine the image coordinates of the position point in the image, and the above image coordinates are two-dimensional image coordinates. Exemplarily, the two-dimensional image coordinates of the clicked position point can be expressed as (a0, b0).

[0059] S203. Determine a target vector for controlling the movement of the capsule endoscope based on the image coordinates of the position point, where the direction of the target vector is the target direction in which the capsule endoscope is to be turned.

[0060] In the embodiment of the present application, the control unit can convert the two-dimensional image coordinates of the clicked position point into a three-dimensional target vector through a series of coordinate conversion and other processes. The direction of the target vector is the target direction that the capsule endoscope should maintain during the process of controlling the movement of the capsule endoscope. By moving along the target direction, the capsule endoscope can reach the clicked position point.

[0061] In a possible implementation of the embodiment of the present application, as Figure 4 As shown, in S203, based on the image coordinates of the position point, determining the target vector for controlling the movement of the capsule endoscope may specifically include the following steps S2031-S2032:

[0062] S2031. Convert the two-dimensional image coordinates of the position point into a three-dimensional direction vector in the coordinate system of the data acquisition module in the capsule endoscope.

[0063] In an embodiment of the present application, for the two-dimensional image coordinates of a position point, the two-dimensional image coordinates can be converted into a three-dimensional direction vector by adding a dimension based on normalizing the coordinates of each position point in the image.

[0064] For example, for the image coordinates (a0, b0) in the aforementioned example, a three-dimensional direction vector (a1, b1, 1) can be obtained after normalization and adding one dimension.

[0065] Then, the three-dimensional direction vector can be converted into a three-dimensional direction vector in the coordinate system of the data acquisition module in the capsule endoscope through coordinate conversion.

[0066] S2032: Convert the three-dimensional direction vector in the data acquisition module coordinate system to the capsule positioning module coordinate system outside the body to obtain the target vector of the capsule endoscope movement.

[0067] In the embodiment of the present application, for the three-dimensional direction vector in the data acquisition module coordinate system, the three-dimensional direction vector in the data acquisition module coordinate system in the capsule endoscope in vivo can be converted to the capsule positioning module coordinate system outside the body by continuing to perform coordinate conversion, and the target vector of the capsule endoscope movement can be obtained. In this way, the subsequent control process can be implemented based on the capsule positioning module coordinate system, thereby obtaining a more accurate control result.

[0068] In a possible implementation of the embodiment of the present application, in the process of converting the three-dimensional direction vector in the data acquisition module coordinate system to the capsule positioning module coordinate system outside the body to obtain the target vector of the capsule endoscope movement, the fixed position relationship between the data acquisition module and the orientation measurement module in the capsule endoscope can be first determined, and the above-mentioned fixed position relationship can be determined according to the internal structure of the capsule endoscope. Then, the three-dimensional direction vector in the data acquisition module coordinate system can be converted into a three-dimensional direction vector in the orientation measurement module coordinate system according to the fixed position relationship. On this basis, by obtaining the posture data and calibration results of the orientation measurement module, the three-dimensional direction vector in the orientation measurement module coordinate system can be converted into a three-dimensional direction vector in the inertial coordinate system according to the posture data, and then the three-dimensional direction vector in the inertial coordinate system can be converted into the capsule positioning module coordinate system according to the calibration results to obtain the target vector of the capsule endoscope movement.

[0069] S204: Determine the current working mode of the capsule endoscope, and control the movement of the external magnetic source according to the working mode and the target direction to achieve manipulation of the capsule endoscope.

[0070] In an embodiment of the present application, after the capsule endoscope is started, its working modes may include multiple ones. Exemplarily, the working modes of the capsule endoscope may include a driving mode and an observation mode. Among them, the driving mode may refer to a working mode in which the capsule endoscope can be manipulated to move in the human digestive tract according to the trajectory expected by the operator; and the observation mode may be a working mode for observing a specific position. In the observation mode, the capsule endoscope often needs to move to a specific position, such as the lesion position in the digestive tract, and stay at this position, and continuously collect images of this position for the operator to diagnose. Therefore, in different working modes, the capsule endoscope is manipulated differently.

[0071] In a possible implementation of the embodiment of the present application, as Figure 5 As shown, in S204, the movement of the external magnetic source is controlled according to the working mode and the target direction to realize the manipulation of the capsule endoscope, which may specifically include the following steps S2041-S2042:

[0072] S2041. When the working mode is the driving mode, after controlling the external magnetic source to rotate until the magnetic pole direction is the same as the target direction of the capsule endoscope, controlling the external magnetic source to move along the target direction.

[0073] In an embodiment of the present application, when the working mode of the capsule endoscope is the driving mode, the control unit should control the capsule endoscope to move along a specific direction or trajectory in the digestive tract. In order to achieve the above purpose, the control unit can rotate and move by controlling the external magnetic source. The magnetic field generated during the rotation and movement of the external magnetic source acts on the capsule endoscope in the body, which can make the capsule endoscope move in the desired target direction.

[0074] In a possible implementation of the embodiment of the present application, the control unit may first control the external magnetic source to rotate until the magnetic pole direction is the same as the target direction of the capsule endoscope. In this way, the direction of the capsule endoscope movement can be the same as the target direction. Then, the control unit may control the external magnetic source to move a certain distance in the target direction, thereby achieving the effect of controlling the capsule endoscope to move in the target direction.

[0075] In an embodiment of the present application, after controlling the external magnetic source to rotate until the magnetic pole direction is the same as the target direction of the capsule endoscope, the moving position of the magnetic source can be calculated and the magnetic source can be controlled to move to that position, thereby achieving the purpose of controlling the external magnetic source to move a certain distance along the target direction.

[0076] In a possible implementation of the embodiment of the present application, the extracorporeal magnetic source can be installed on a robotic arm, the control unit can control the position and posture of the magnetic source through the robotic arm, and the control unit can obtain the real-time position and posture information of the magnetic source through the capsule positioning module or the robotic arm. That is, the real-time position and posture of the magnetic source can be obtained through the capsule positioning module, or directly through the robotic arm. After determining the target vector of the capsule endoscope, the control unit can control the magnetic source to move a set distance along the target direction of the capsule endoscope based on the set moving distance and taking the current position of the magnetic source as the starting point.

[0077] S2042. When the working mode is the observation mode, control the external magnetic source to rotate until the magnetic pole direction is the same as the target direction of the capsule endoscope, so as to drive the capsule endoscope to turn to the target direction.

[0078] In the embodiment of the present application, when the working mode of the capsule endoscope is the observation mode, the control unit should control the capsule endoscope to stay at a specific position. To achieve the above purpose, the control unit can control the external magnetic source to rotate, thereby driving the capsule endoscope to turn to the target direction.

[0079] It should be noted that the control process in the above-mentioned observation mode can be carried out after the capsule endoscope is moved to a specific position in the driving mode. Therefore, for the observation of the lesion location, the operator can first click on the lesion location found in the image in the driving mode, and drive the capsule endoscope to move to the lesion location according to the steps described above. In order to achieve a detailed diagnosis of the lesion location, when the capsule endoscope moves to the lesion location, the operator can switch the working mode of the capsule endoscope to the observation mode. Thereafter, the operator can click at the lesion location in the image, thereby driving the capsule endoscope to turn to the target direction corresponding to the clicked position point, thereby achieving all-round observation of the lesion location.

[0080] In a possible implementation of an embodiment of the present application, the control unit can calculate the rotation angle of the external magnetic source when controlling the external magnetic source to rotate until the magnetic pole direction is the same as the target direction of the capsule endoscope, and then control the external magnetic source to rotate according to the rotation angle, thereby driving the capsule endoscope to turn to the target direction.

[0081] In the embodiment of the present application, the rotation angle may include a horizontal rotation angle and a vertical rotation angle. The horizontal rotation angle and the vertical rotation angle may be calculated respectively in different ways.

[0082] Specifically, the first projection vector of the capsule endoscope's target vector on the horizontal plane and the second projection vector of the current magnetic pole direction of the external magnetic source on the horizontal plane can be calculated. The horizontal rotation angle is determined according to the first projection vector and the second projection vector, which is also the angle between the two projection vectors.

[0083] For the vertical rotation angle, the first angle between the capsule endoscope's target vector and the vertical direction vector and the second angle between the current magnetic pole direction of the external magnetic source and the vertical direction vector can be calculated. The vertical rotation angle is determined according to the first angle and the second angle, which is the difference between the two angles.

[0084] In an embodiment of the present application, each click of the operator on the display device causes the control unit to update the target vector of the capsule endoscope once, and controls the rotation of the magnetic source or the rotation and movement of the magnetic source according to the current working mode, thereby achieving the effect of driving the capsule endoscope to observe or drive to the clicked position and direction in the image.

[0085] In an embodiment of the present application, by displaying the image captured by the capsule endoscope in a display device, the image in the digestive tract can be displayed to the operator in real time. The control method provided in the embodiment of the present application adopts the method of clicking on the image to update the target direction of the capsule endoscope. The control unit can determine the target vector for controlling the movement of the capsule endoscope by determining the coordinates of the position point in the image clicked by the operator, thereby obtaining the target direction to be turned to by the capsule endoscope. Then, according to the working mode of the capsule endoscope and the determined target direction, the capsule endoscope can be controlled by controlling the movement of the external magnetic source. By applying the control method provided in the embodiment of the present application, the actual position of the capsule endoscope does not need to be known during the control process of the capsule endoscope, and the direction of driving the capsule endoscope can be freely selected, which is conducive to the detailed observation of the lesion location. At the same time, the control of the capsule endoscope by actively clicking on the image is simpler and more convenient than the operation using external devices such as handles, which reduces the difficulty of controlling the capsule endoscope and improves the control efficiency.

[0086] It should be noted that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0087] For ease of understanding, the method for controlling a capsule endoscope provided in an embodiment of the present application is introduced below with reference to a specific example.

[0088] Reference Figure 6 , showing a flow chart of a method for controlling a capsule endoscope provided in an embodiment of the present application. Figure 6 The process shown can be Figure 1 The control system shown is implemented.

[0089] Specifically, Figure 6As shown, in actual use, a patient who has swallowed a capsule endoscope can lie on top of the capsule positioning module, then start the capsule endoscope, and calibrate the orientation measurement module in the capsule endoscope when starting. After completing the calibration of the orientation measurement module, the capsule endoscope can start to collect data, such as images of the digestive tract in the body. The collected images can be sent to the software host computer outside the body through the wireless module of the capsule endoscope for display. The software host computer can be a computer device that integrates the functions of the control unit and the display device in the aforementioned embodiments. The operator can control the capsule endoscope by directly clicking in the image. When the operator clicks on a certain position point in the image, the software host computer can obtain the image coordinates corresponding to the position point, and the direction indicated by the image coordinates is the direction in which the operator expects the capsule endoscope to move or turn. For the determined image coordinates, the software host computer can convert them from two-dimensional to three-dimensional direction vectors by increasing the dimension. In this way, the software host computer can drive the external magnetic source to turn to the target direction.

[0090] After the external magnetic source is turned to the target direction, the capsule endoscope can be controlled according to the working mode it is in. Specifically, Figure 6 As shown, when the external magnetic source turns to the target direction, it can be determined whether the capsule endoscope is in the driving mode. If the capsule endoscope is in the driving mode, the software host computer can drive the magnetic source to move along the target direction, thereby driving the capsule endoscope to follow the movement. If the capsule endoscope is not in the driving mode but in the observation mode, the operator can control the capsule endoscope to turn to the target direction corresponding to the clicked position point by clicking on the image, thereby achieving the purpose of continuous observation at a specific location.

[0091] Reference Figure 7 , shows a schematic diagram of a capsule endoscope control device provided in an embodiment of the present application, which may specifically include a capsule image display module 701, an icon coordinate determination module 702, a motion vector determination module 703 and a capsule control module 704, wherein:

[0092] Capsule image display module 701, used to display the image captured by the capsule endoscope on a display device;

[0093] An icon coordinate determination module 702, configured to determine the image coordinates of any position point in the image in response to an operation of clicking the position point in the image;

[0094] A motion vector determination module 703 is used to determine a target vector for controlling the motion of the capsule endoscope based on the image coordinates of the position point, wherein the direction of the target vector is the target direction in which the capsule endoscope is to be turned;

[0095] The capsule control module 704 is used to determine the current working mode of the capsule endoscope, and control the movement of the external magnetic source according to the working mode and the target direction to realize the control of the capsule endoscope.

[0096] In a possible implementation of the embodiment of the present application, the motion vector determination module 703 may be specifically used to:

[0097] Converting the two-dimensional image coordinates of the position point into a three-dimensional direction vector in the coordinate system of the data acquisition module in the capsule endoscope;

[0098] The three-dimensional direction vector in the data acquisition module coordinate system is converted into the capsule positioning module coordinate system outside the body to obtain the target vector of the capsule endoscope movement.

[0099] In a possible implementation manner of the embodiment of the present application, the motion vector determination module 703 may also be used to:

[0100] Determine the fixed position relationship between the data acquisition module and the orientation measurement module in the capsule endoscope, and convert the three-dimensional direction vector in the coordinate system of the data acquisition module into the three-dimensional direction vector in the coordinate system of the orientation measurement module according to the fixed position relationship; obtain the posture data and calibration result of the orientation measurement module; convert the three-dimensional direction vector in the coordinate system of the orientation measurement module into the three-dimensional direction vector in the inertial coordinate system according to the posture data; convert the three-dimensional direction vector in the inertial coordinate system into the coordinate system of the capsule positioning module according to the calibration result, and obtain the target vector of the capsule endoscope movement.

[0101] In a possible implementation manner of the embodiment of the present application, the device may further include a calibration module, and the calibration module may be specifically used to:

[0102] When the capsule endoscope is started, the orientation measurement module in the capsule endoscope is calibrated to obtain a calibration result.

[0103] In a possible implementation of the embodiment of the present application, the calibration module may also be used for:

[0104] When the capsule endoscope is started, the position information of the capsule endoscope is obtained through the capsule positioning module outside the body, wherein the position information includes the current direction vector of the capsule endoscope and the quaternion posture of the orientation measurement module expressed in quaternion;

[0105] Based on the quaternion posture of the orientation measurement module, rotating the unit vector corresponding to the current direction vector of the capsule endoscope;

[0106] The angle between the rotated unit vector and the current direction vector of the capsule endoscope is calculated to obtain the calibration result.

[0107] In a possible implementation of the embodiment of the present application, the capsule control module 704 may be specifically used for:

[0108] When the working mode is the driving mode, after controlling the external magnetic source to rotate until the magnetic pole direction is the same as the target direction of the capsule endoscope, controlling the external magnetic source to move along the target direction;

[0109] When the working mode is the observation mode, the external magnetic source is controlled to rotate until the magnetic pole direction is the same as the target direction of the capsule endoscope, so as to drive the capsule endoscope to turn to the target direction.

[0110] In a possible implementation of the embodiment of the present application, the capsule control module 704 may also be used to:

[0111] Calculating the rotation angle of the external magnetic source;

[0112] The external magnetic source is controlled to rotate according to the rotation angle to drive the capsule endoscope to turn to the target direction.

[0113] In a possible implementation of the embodiment of the present application, the capsule control module 704 may also be used to:

[0114] Calculate a first projection vector of the target vector of the capsule endoscope on the horizontal plane and a second projection vector of the current magnetic pole direction of the external magnetic source on the horizontal plane;

[0115] Calculating a first angle between a target vector of the capsule endoscope and a vertical direction vector and a second angle between a current magnetic pole direction of the external magnetic source and the vertical direction vector;

[0116] Determine the horizontal rotation angle according to the first projection vector and the second projection vector;

[0117] The vertical rotation angle is determined according to the first angle and the second angle.

[0118] An embodiment of the present application provides a capsule endoscope control device, and by using the device, each step in the aforementioned method embodiments can be implemented.

[0119] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment part.

[0120] Reference Figure 8, shows a schematic diagram of a capsule endoscope control device provided in an embodiment of the present application. Figure 8 As shown, the control device 800 in the embodiment of the present application includes: a processor 810, a memory 820, and a computer program 821 stored in the memory 820 and executable on the processor 810. When the processor 810 executes the computer program 821, the steps in each embodiment of the control method of the capsule endoscope are implemented, such as Figure 2 Alternatively, when the processor 810 executes the computer program 821, the functions of each module / unit in the above-mentioned device embodiments are realized, for example Figure 7 Functions of modules 701 to 704 are shown.

[0121] Exemplarily, the computer program 821 may be divided into one or more modules / units, which are stored in the memory 820 and executed by the processor 810 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which may be used to describe the execution process of the computer program 821 in the control device 800. For example, the computer program 821 may be divided into a capsule image display module, an icon coordinate determination module, a motion vector determination module, and a capsule control module, and the specific functions of each module are as follows:

[0122] A capsule image display module, used to display images captured by the capsule endoscope on a display device;

[0123] An icon coordinate determination module, configured to determine the image coordinates of any position point in the image in response to an operation of clicking the position point in the image;

[0124] A motion vector determination module, used for determining a target vector for controlling the motion of the capsule endoscope based on the image coordinates of the position point, wherein the direction of the target vector is the target direction in which the capsule endoscope is to be turned;

[0125] The capsule control module is used to determine the current working mode of the capsule endoscope, and control the movement of the external magnetic source according to the working mode and the target direction to realize the control of the capsule endoscope.

[0126] The control device 800 may be an electronic device capable of implementing the relevant functions of the control unit in the above-mentioned embodiments, and the electronic device may be a desktop computer, a tablet computer, etc. The control device 800 may include, but is not limited to, a processor 810 and a memory 820. Those skilled in the art will understand that Figure 8It is only an example of the control device 800 and does not constitute a limitation of the control device 800. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the control device 800 may also include input and output devices, network access devices, buses, etc.

[0127] The processor 810 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0128] The memory 820 may be an internal storage unit of the control device 800, such as a hard disk or memory of the control device 800. The memory 820 may also be an external storage device of the control device 800, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the control device 800. Further, the memory 820 may also include both an internal storage unit and an external storage device of the control device 800. The memory 820 is used to store the computer program 821 and other programs and data required by the control device 800. The memory 820 may also be used to temporarily store data that has been output or is to be output.

[0129] An embodiment of the present application also discloses a capsule endoscope control device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the capsule endoscope control method described in the aforementioned embodiments is implemented.

[0130] The embodiments of the present application further disclose a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for controlling a capsule endoscope as described in the aforementioned embodiments is implemented.

[0131] The embodiment of the present application further discloses a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the capsule endoscope control method described in the above-mentioned embodiments.

[0132] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application is described in detail with reference to the above-mentioned embodiments, a person skilled in the art should understand that the technical solutions described in the above-mentioned embodiments can still be modified, or some of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A capsule endoscope control device, characterized in that: The device is used to implement the following method: Displaying an image captured by a capsule endoscope on a display device, wherein the image is acquired by a data acquisition module in the capsule endoscope; In response to an operation of clicking any position point in the image, determining the image coordinates of the position point in the image; Convert the two-dimensional image coordinates of the position point into a three-dimensional direction vector in the coordinate system of the data acquisition module in the capsule endoscope, convert the three-dimensional direction vector in the coordinate system of the data acquisition module into the coordinate system of the capsule positioning module outside the body, and obtain the target vector of the capsule endoscope movement, wherein the direction of the target vector is the target direction to be turned by the capsule endoscope; Determining a current working mode of the capsule endoscope, and controlling the movement of an external magnetic source according to the working mode and the target direction to achieve manipulation of the capsule endoscope; Wherein, controlling the movement of the external magnetic source according to the working mode and the target direction to achieve the manipulation of the capsule endoscope includes: When the working mode is the driving mode, after controlling the external magnetic source to rotate until the magnetic pole direction is the same as the target direction of the capsule endoscope, controlling the external magnetic source to move along the target direction; When the working mode is the observation mode, the external magnetic source is controlled to rotate until the magnetic pole direction is the same as the target direction of the capsule endoscope, so as to drive the capsule endoscope to turn to the target direction.

2. The device according to claim 1, characterized in that The step of converting the three-dimensional direction vector in the data acquisition module coordinate system into the capsule positioning module coordinate system outside the body to obtain the target vector of the capsule endoscope movement includes: Determining a fixed position relationship between a data acquisition module and an orientation measurement module in the capsule endoscope, and converting a three-dimensional direction vector in a coordinate system of the data acquisition module into a three-dimensional direction vector in a coordinate system of the orientation measurement module according to the fixed position relationship; Acquiring attitude data and calibration results of the orientation measurement module; Converting the three-dimensional direction vector in the coordinate system of the orientation measurement module into a three-dimensional direction vector in the inertial coordinate system according to the posture data; The three-dimensional direction vector in the inertial coordinate system is converted into the capsule positioning module coordinate system according to the calibration result to obtain the target vector of the capsule endoscope movement.

3. The device according to claim 1 or 2, characterized in that Before displaying the image captured by the capsule endoscope on the display device, the method further includes: When the capsule endoscope is started, the orientation measurement module in the capsule endoscope is calibrated to obtain a calibration result.

4. The device according to claim 3, characterized in that When the capsule endoscope is started, calibrating the orientation measurement module in the capsule endoscope to obtain a calibration result includes: When the capsule endoscope is started, the position information of the capsule endoscope is obtained through the capsule positioning module outside the body, wherein the position information includes the current direction vector of the capsule endoscope and the quaternion posture of the orientation measurement module expressed in quaternion; Based on the quaternion posture of the orientation measurement module, rotating the unit vector corresponding to the current direction vector of the capsule endoscope; The angle between the rotated unit vector and the current direction vector of the capsule endoscope is calculated to obtain the calibration result.

5. The device according to any one of claims 1, 2 or 4, characterized in that: The controlling the external magnetic source to rotate until the magnetic pole direction is the same as the target direction of the capsule endoscope comprises: Calculating the rotation angle of the external magnetic source; The external magnetic source is controlled to rotate according to the rotation angle to drive the capsule endoscope to turn to the target direction.

6. The device according to claim 5, characterized in that The rotation angle includes a horizontal rotation angle and a vertical rotation angle, and the calculating the rotation angle of the external magnetic source includes: Calculate a first projection vector of the target vector of the capsule endoscope on the horizontal plane and a second projection vector of the current magnetic pole direction of the external magnetic source on the horizontal plane; Calculating a first angle between a target vector of the capsule endoscope and a vertical direction vector and a second angle between a current magnetic pole direction of the external magnetic source and the vertical direction vector; Determine the horizontal rotation angle according to the first projection vector and the second projection vector; The vertical rotation angle is determined according to the first angle and the second angle.

7. A capsule endoscope control device, characterized in that: include: A capsule image display module, used for displaying the image captured by the capsule endoscope in a display device, wherein the image is acquired by the data acquisition module in the capsule endoscope; An icon coordinate determination module, configured to determine the image coordinates of any position point in the image in response to an operation of clicking the position point in the image; a motion vector determination module, used for converting the two-dimensional image coordinates of the position point into a three-dimensional direction vector in the coordinate system of the data acquisition module in the capsule endoscope, converting the three-dimensional direction vector in the coordinate system of the data acquisition module into the coordinate system of the capsule positioning module outside the body, and obtaining a target vector for the movement of the capsule endoscope, wherein the direction of the target vector is the target direction to be turned by the capsule endoscope; A capsule control module, used to determine the current working mode of the capsule endoscope, and control the movement of the external magnetic source according to the working mode and the target direction to realize the control of the capsule endoscope; The capsule manipulation module is specifically used for: When the working mode is the driving mode, after controlling the external magnetic source to rotate until the magnetic pole direction is the same as the target direction of the capsule endoscope, controlling the external magnetic source to move along the target direction; When the working mode is the observation mode, the external magnetic source is controlled to rotate until the magnetic pole direction is the same as the target direction of the capsule endoscope, so as to drive the capsule endoscope to turn to the target direction.

8. A capsule endoscope control system, characterized in that: It includes a display device, a capsule positioning module and a control unit; wherein: The display device is used to display the image captured by the capsule endoscope, and the image is acquired by the data acquisition module in the capsule endoscope; The control unit is configured to determine the image coordinates of any position point in the image in response to an operation of clicking on the position point in the image; convert the two-dimensional image coordinates of the position point into a three-dimensional direction vector in the coordinate system of the data acquisition module in the capsule endoscope, and convert the three-dimensional direction vector in the coordinate system of the data acquisition module into the coordinate system of the capsule positioning module outside the body to obtain a target vector for the movement of the capsule endoscope, wherein the direction of the target vector is the target direction to be turned by the capsule endoscope; determine the current working mode of the capsule endoscope, and control the movement of the external magnetic source according to the working mode and the target direction to realize the control of the capsule endoscope; The control unit is specifically used to: when the working mode is the driving mode, after controlling the external magnetic source to rotate until the magnetic pole direction is the same as the target direction of the capsule endoscope, control the external magnetic source to move along the target direction; When the working mode is the observation mode, the external magnetic source is controlled to rotate until the magnetic pole direction is the same as the target direction of the capsule endoscope, so as to drive the capsule endoscope to turn to the target direction.

9. A capsule endoscope control device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method implemented by the control device of the capsule endoscope according to any one of claims 1 to 6 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method implemented by the control device of the capsule endoscope according to any one of claims 1 to 6 is implemented.

Images