Capsule endoscope cruise control system

Abstract

The application discloses a kind of capsule endoscope cruise control systems, under current preset body position, depth camera shoots the image of the examinee and constructs three-dimensional point cloud;Computer host locates the position of initial stomach model in the three-dimensional point cloud according to the three-dimensional point cloud;Ultrasonic matrix panel detects the ultrasonic echo signal of the stomach of the examinee;The computer host generates three-dimensional stomach contour according to the ultrasonic echo signal, and the shape, size and position of the initial stomach model in the three-dimensional point cloud are corrected accordingly, obtain stomach model, and the current preset cruise point of the stomach model is mapped to the three-dimensional point cloud;Control module controls mechanical arm to drive end magnet to reach the current preset cruise point, so as to drive capsule endoscope to scan each feature part in the feature part set corresponding to the current preset cruise point of stomach to realize scene capture. Precise and effective control of capsule endoscope is realized, and the inspection efficiency of capsule endoscope is improved.

Description

Technical Field

[0001] This invention relates to the field of medical device technology, and in particular to a capsule endoscope cruise control system. Background Technology

[0002] Currently, the location of a magnetically controlled capsule endoscope during gastric examinations can only be roughly determined through image analysis. This involves using a trained AI model to identify several key features of the stomach to roughly estimate the location of the capsule endoscope. Specifically, the stomach is divided into three regions: the upper, middle, and lower parts. Each region includes multiple feature points. The trained AI model analyzes the area occupied by these feature areas in front of the capsule endoscope lens and compares it to the surrounding area to vaguely infer the location of the capsule endoscope and whether the entire region has been scanned. The main limitations of this method include:

[0003] 1) The positioning of capsule endoscopes relies too heavily on image analysis. Due to the lack of other technical support and the inherent error rate of AI model analysis, there is a large error in determining the position of capsule endoscopes based on this method.

[0004] 2) The approximate location of the capsule endoscope can only be determined by a limited number of known feature points, such as the cardia, fundus, angle, antrum, and pylorus. However, the capsule endoscope is prone to getting lost in areas without feature points, such as the anterior and posterior walls of the upper, middle, and lower parts of the stomach and the greater curvature.

[0005] 3) The movement of the magnetically controlled permanent magnet can only guide the capsule endoscope to move in a certain direction, but it cannot provide accurate displacement data based on the magnetic pole axis of the permanent magnet. Therefore, it is impossible to determine whether the capsule endoscope is at the pole center of the permanent magnet during movement. Often, the permanent magnet moves a certain distance while the capsule endoscope remains stationary or moves only a little bit, resulting in low control accuracy and difficulty in positioning of the capsule endoscope.

[0006] 4) Because the magnetically controlled capsule endoscope can only move roughly up and down in the stomach, the image features it identifies through the AI ​​model cannot guarantee that the area has been fully scanned. In addition, the actual shape of each person's stomach differs from the standard stomach model. Therefore, in order to scan the area as many times as possible, the capsule endoscope can only take pictures back and forth repeatedly, resulting in low efficiency. Summary of the Invention

[0007] In order to solve at least one of the above-mentioned technical problems in the prior art, the present invention provides a capsule endoscope cruise control system, which aims to improve the accuracy and efficiency of capsule endoscope control, while ensuring the comprehensiveness and integrity of the examination.

[0008] This invention provides a capsule endoscope cruise control system, comprising: a computer host, a control module, a robotic arm, an end-effector rotation axis, an end-effector magnet, an examination bed, an ultrasound matrix panel, and a depth camera, wherein the ultrasound matrix panel is placed on the abdomen of the patient.

[0009] Under the current preset body position, the depth camera captures images of the subject and constructs a three-dimensional point cloud;

[0010] The computer host determines the shoulder joint position and hip joint position of the subject based on the three-dimensional point cloud; the computer host locates the position of the initial stomach model in the three-dimensional point cloud based on the shoulder joint position and hip joint position of the subject;

[0011] The ultrasound matrix panel detects the ultrasound echo signal of the patient's stomach contour and sends the ultrasound echo signal to the computer host.

[0012] The computer host generates a digital three-dimensional stomach contour based on the ultrasonic echo signal;

[0013] The computer host corrects the shape, size, and position of the initial stomach model in the three-dimensional point cloud based on the digital three-dimensional stomach contour to obtain the stomach model;

[0014] The computer host maps the current preset cruise point of the stomach model onto the subject's three-dimensional point cloud;

[0015] The control module controls the robotic arm to drive the end magnet to the current preset cruise point;

[0016] The control module controls the robotic arm to drive the end magnet to move, thereby driving the capsule endoscope to scan each feature part of the feature part set corresponding to the current preset cruise point in the stomach to achieve scene capture, so as to complete a complete scan of each target part in the target part set corresponding to the current preset cruise point.

[0017] In some embodiments, the computer host uses ultrasound to position the capsule endoscope in three dimensions within the stomach.

[0018] The computer host maps the three-dimensional position of the capsule endoscope in the stomach onto the stomach model;

[0019] After the capsule endoscope has scanned the target area, the computer host marks the area where the target area is located as the scanned area on the stomach model.

[0020] In some embodiments, the ultrasound matrix panel includes crystal probes arranged in an m x n matrix, where m and n are positive integers.

[0021] In some embodiments, the frequency of the crystal probe is 3.5 to 5 MHz.

[0022] In some embodiments, the control module controls the robotic arm to move the end magnet, thereby driving the capsule endoscope to scan each feature part of the feature set corresponding to the current preset cruise point in the stomach to achieve scene capture, so as to complete a complete scan of each target part in the target part set corresponding to the current preset cruise point, including:

[0023] For each target location in the target location set corresponding to the current preset cruise point, the computer host selects the optimal scene combination from the scene combination set corresponding to the target location based on the feature location set corresponding to the current preset cruise point.

[0024] The control module controls the robotic arm to move the end magnet, thereby driving the capsule endoscope to capture the optimal scene combination;

[0025] When the optimal scene combination is captured, the target area is completely scanned.

[0026] In some embodiments, the current preset body position is supine, left lateral decubitus, or right lateral decubitus.

[0027] In some embodiments, in the supine position, the distal magnet moves in a region from the left side of the abdominal cavity to the umbilicus, with the subject's xiphoid process as a reference.

[0028] In the left lateral decubitus position, the end magnet moves in the area below the subject's back near the heart.

[0029] In the right lateral decubitus position, the end magnet moves in the area near the navel on the left side of the subject's abdomen.

[0030] In some embodiments, in the supine position, the current preset cruise points are the fundus of the stomach, the anterior wall of the middle part of the stomach, and the anterior wall of the antrum of the stomach;

[0031] With the patient in a left lateral decubitus position, the current preset cruise point is the cardia;

[0032] In the right lateral decubitus position, the current preset cruise point is the pylorus.

[0033] In some embodiments, the set of feature locations corresponding to the current preset cruise point fundus includes the feature locations cardia, fundus, lesser curvature, and gastric cavity;

[0034] The set of characteristic locations corresponding to the current preset cruise point in the middle forearm of the stomach includes the characteristic locations of the lesser curvature of the stomach, the gastric angle, and the gastric antrum.

[0035] The set of characteristic locations corresponding to the anterior wall of the gastric antrum at the current preset cruise point includes the characteristic locations of the gastric angle, gastric antrum, and pylorus.

[0036] The set of characteristic locations corresponding to the current preset cruise point cardia includes the characteristic locations cardia, greater curvature of the stomach, and gastric cavity;

[0037] The set of characteristic locations corresponding to the current preset cruise point pylorus includes the characteristic locations of gastric angle, gastric antrum, pylorus and gastric cavity.

[0038] In some embodiments, the target location set corresponding to the current preset cruise point at the gastric fundus includes the target location cardia, gastric fundus, posterior wall of the lower gastric cardia, lesser curvature of the upper stomach, posterior wall of the upper stomach, lesser curvature of the middle stomach, and posterior wall of the middle stomach; the target location set corresponding to the current preset cruise point at the forearm of the middle stomach includes the target location lesser curvature of the lower stomach, posterior wall of the lower stomach, greater curvature of the lower stomach, gastric angle, and posterior wall of the gastric angle.

[0039] The target location set corresponding to the anterior wall of the gastric antrum at the current preset cruise point includes the target locations: the greater curvature of the lower stomach, the gastric angle, the posterior wall of the gastric angle, the lesser curvature of the gastric antrum, the posterior wall of the gastric antrum, and the greater curvature of the gastric antrum.

[0040] The target site set corresponding to the current preset cruise point cardia includes the target site anterior wall of the cardia, anterior wall of the upper stomach, greater curvature of the upper stomach, posterior wall of the upper stomach, anterior wall of the middle stomach, greater curvature of the middle stomach, and posterior wall of the middle stomach.

[0041] The target location set corresponding to the current preset cruise point pylorus includes the target locations: anterior wall of the gastric angle, posterior wall of the gastric angle, greater curvature of the lower stomach, anterior wall of the gastric antrum, posterior wall of the gastric antrum, lesser curvature of the gastric antrum, greater curvature of the gastric antrum, and pylorus.

[0042] This invention provides a capsule endoscope cruise control system, comprising: a computer host, a control module, a robotic arm, an end-effector rotation axis, an end-effector magnet, an examination bed, an ultrasound matrix panel, and a depth camera. The ultrasound matrix panel is placed on the abdomen of the patient. In a preset position, the depth camera captures images of the patient and constructs a three-dimensional point cloud. The computer host determines the shoulder and hip joint positions of the patient based on the three-dimensional point cloud. The computer host locates the position of an initial stomach model within the three-dimensional point cloud based on the shoulder and hip joint positions of the patient. The ultrasound matrix panel detects the ultrasound echo signal of the patient's stomach contour and sends the ultrasound echo signal to the computer host. The machine generates a digital three-dimensional stomach contour based on the ultrasonic echo signal; the computer host corrects the shape, size, and position of the initial stomach model in the three-dimensional point cloud based on the digital three-dimensional stomach contour to obtain a stomach model; the computer host maps the current preset cruise point of the stomach model onto the subject's three-dimensional point cloud; the control module controls the robotic arm to drive the end magnet to the current preset cruise point; the control module controls the robotic arm to drive the end magnet to move, thereby driving the capsule endoscope to scan each feature part of the feature set corresponding to the current preset cruise point of the stomach to achieve scene capture, so as to complete the complete scan of each target part of the target part set corresponding to the current preset cruise point. In this system, the three-dimensional point cloud coordinates on the corrected stomach model can provide the precise position of the current preset cruise point, thereby guiding the robotic arm to drive the end magnet to control the movement of the capsule endoscope along each current preset cruise point, realizing precise and effective control of the capsule endoscope, improving the efficiency of capsule endoscopy, and ensuring the integrity of the capsule endoscope examination of the stomach. Attached Figure Description

[0043] The accompanying drawings are provided to further illustrate the embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation on the embodiments of the present invention.

[0044] Figure 1 This is a schematic diagram of a capsule endoscope cruise control system according to an embodiment of the present invention;

[0045] Figure 2 This is a three-dimensional point cloud image of the subject in an embodiment of the present invention;

[0046] Figure 3 This is a schematic diagram of the rotation mode of the end rotation axis R1 and R2 of the capsule endoscope cruise control device in an embodiment of the present invention.

[0047] Explanation of reference numerals in the attached figures:

[0048] Robotic arm 01, end effector rotation axis 02, end effector magnet 03, examination bed 04, ultrasound matrix panel 05, trolley 06, 3D point cloud 07, initial stomach model 08. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0050] like Figure 1 As shown, this embodiment of the invention provides a capsule endoscope cruise control system. The system includes a computer host (not shown), a control module (not shown), a robotic arm 01, an end-effector rotation axis 02, an end-effector magnet 03, an examination bed 04, an ultrasound matrix panel 05, and a depth camera (not shown). The computer host and control module are integrated into a trolley 06, which also integrates a display (not shown) and an input device (not shown). The end-effector rotation axis 02 and the end-effector magnet 03 are integrated into the end of the robotic arm 01. The ultrasound matrix panel 05 is placed on the abdomen of the patient. In the current preset position, the depth camera captures an image of the patient and constructs a three-dimensional point cloud. The computer host determines the shoulder and hip joint positions of the patient based on the three-dimensional point cloud. The computer host locates the position of an initial stomach model in the three-dimensional point cloud based on the shoulder and hip joint positions of the patient. The ultrasound matrix panel 05 detects the ultrasound echo signal of the patient's stomach contour and sends the ultrasound echo signal to the computer host. The computer host generates a digital three-dimensional stomach contour based on the ultrasound echo signal. The computer host corrects the shape and size of the initial stomach model in the three-dimensional point cloud based on the digital three-dimensional stomach contour to obtain a stomach model. The computer host maps the current preset cruise point of the stomach model onto the patient's three-dimensional point cloud. The control module controls the robotic arm 01 to drive the end magnet to the current preset cruise point. The control module controls the robotic arm 01 to drive the end magnet to move, thereby driving the capsule endoscope to scan each feature part of the feature part set corresponding to the current preset cruise point of the stomach to achieve scene capture, so as to complete the complete scan of each target part in the target part set corresponding to the current preset cruise point.

[0051] Specifically, after swallowing the ultrasound contrast agent and capsule endoscope, the patient lies down in a preset position. There is at least one preset position, which can be supine, lateral, etc., and the examination order is not limited. The patient lies down in one of the preset positions. To control the end magnet of the robotic arm 01 to perform the intended work in the patient's stomach, a 3D point cloud reconstruction of the patient is required. Then, the position of the stomach model in the 3D point cloud is located based on the joint positions of the patient, and the end magnet is guided to the designated position by controlling the robotic arm 01. A depth camera can be mounted on the robotic arm 01. In the preset position, the depth camera takes pictures of the patient from multiple angles to stitch together the images to construct the 3D point cloud. The 3D point cloud is as follows: Figure 2 As shown, Figure 2 In the supine position, a reconstructed three-dimensional point cloud image of the subject is generated, where 07 represents the subject's three-dimensional point cloud and 08 represents the initial stomach model. The computer host determines the shoulder and hip joint positions of the subject based on the three-dimensional point cloud. The computer host locates the position of the initial stomach model in the three-dimensional point cloud based on the subject's shoulder and hip joint positions, and the position of the initial stomach model in the subject's three-dimensional point cloud roughly corresponds to the position of the stomach in the subject's body. The ultrasound matrix panel 05 detects the ultrasound echo signal of the subject's stomach contour and sends the ultrasound echo signal to the computer host. The computer host generates a digital three-dimensional stomach contour based on the ultrasound echo signal. The computer host corrects the shape, size, and position of the initial stomach model in the three-dimensional point cloud based on the digital three-dimensional stomach contour to obtain a stomach model. The shape, size, and position of the stomach model in the three-dimensional point cloud are consistent with the shape, size, and position of the stomach in the subject's body, respectively.

[0052] It should be noted that since the origin of the second coordinate system where the robotic arm 01 is located does not coincide with the origin of the first coordinate system where the depth camera is located, it is necessary to transform the three-dimensional point cloud, shoulder joint position and hip joint position of the subject from the first coordinate system with the center of the depth camera as the origin to the second coordinate system with the center of the base of the robotic arm 01 as the origin.

[0053] Each current preset body position corresponds to at least one current preset cruise point, and the scanning order of the current preset cruise points is not limited. The current preset cruise points in the stomach model correspond to the current preset cruise points in the stomach. The current preset cruise point can be the target area to be examined in the stomach. The target areas in the stomach include: fundus, cardia, posterior wall of the lower cardia, anterior wall of the lower cardia, anterior wall of the upper part of the stomach body, posterior wall of the upper part of the stomach body, greater curvature of the upper part of the stomach body, lesser curvature of the upper part of the stomach body, anterior wall of the middle part of the stomach body, posterior wall of the middle part of the stomach body, greater curvature of the middle part of the stomach body, lesser curvature of the middle part of the stomach body, anterior wall of the lower part of the stomach body, posterior wall of the lower part of the stomach body, greater curvature of the lower part of the stomach body, lesser curvature of the lower part of the stomach body, gastric angle, anterior wall of the gastric angle, posterior wall of the gastric angle, anterior wall of the gastric antrum, posterior wall of the gastric antrum, greater curvature of the gastric antrum, lesser curvature of the gastric antrum, and pylorus, a total of 24. Of course, with the development of medicine, the human stomach may be divided into more target areas. The three-dimensional coordinates of each current preset cruise point in the stomach model are known. These three-dimensional coordinates are determined based on a coordinate system with a feature part of the stomach model as the origin. In the current body position, using any one of the corresponding preset cruise points as the current preset cruise point, the computer host maps the current preset cruise point of the stomach model onto the surface of the subject's three-dimensional point cloud, that is, determines the three-dimensional coordinates of the current preset cruise point on the subject's body surface. This can be done by determining the three-dimensional coordinates of the current preset cruise point on the surface of the subject's three-dimensional point cloud based on the relationship between the coordinate system with a feature part of the stomach model as the origin and the coordinate system of the subject's three-dimensional point cloud.

[0054] The control module controls the robotic arm 01 to guide its end magnet to the three-dimensional coordinate position of the current preset cruise point on the subject's body surface, thereby driving the capsule endoscope to the current preset cruise point in the stomach, so as to achieve precise control of the capsule endoscope.

[0055] The feature set corresponding to the current preset cruise point is composed of feature parts adjacent to the current preset cruise point, and the feature set corresponding to the current preset cruise point may include the current preset cruise point itself. For the human stomach, a feature part refers to a biologically distinctive and identifiable part, combination of parts, or feature point within the stomach; a feature part can be a target part. Currently, the trained AI model can identify the following stomach feature parts: cardia, fundus, lesser curvature, greater curvature, gastric cavity, gastric angle, antrum, and pylorus. The target part set corresponding to the current preset cruise point includes at least one target part, and each target part in the target part set corresponding to the current preset cruise point can be fully scanned by capturing and scanning the scene defined by the feature parts in the feature part set corresponding to the current preset cruise point. For specific methods, please refer to Chinese patent application CN114259197A, entitled "A Capsule Endoscopy Quality Control Method and System". This patent application discloses the following technical solution: constructing multiple scenes based on the feature parts that can be identified by the AI ​​model, defining the uniqueness of the scene by the interrelationship between the feature parts in the scene, and each constructed scene includes the main view part and the secondary view part. Based on the correspondence between the 24 target parts of the stomach and their respective adjacent parts, all constructed scenes or scene combinations completely cover the 24 target parts. During the inspection, the magnetically controlled device drives the capsule endoscope to move within the target area via a first magnet; the capsule endoscope acquires images within the target area and sends the images to a terminal device; the terminal device identifies feature regions in the images and outputs the ID (Identity Document) and detection bounding box of each feature region; the terminal device identifies the scene in the image based on the ID and detection bounding box of the feature regions, wherein the scene includes k feature regions and the relationships between the feature regions, and the uniqueness of the scene is defined by the relationships, where k is a positive integer; the terminal device determines whether the target region has been completely inspected based on the scene or a combination of scenes, thereby ensuring that each target region is completely inspected and preventing missed inspections. According to the method described in Chinese patent application CN114259197A, multiple scenes or scene combinations are constructed using feature parts in the feature part set corresponding to the current preset cruise point, so that the constructed multiple scenes or scene combinations completely cover the target parts in the target part set corresponding to the current preset cruise point. A complete scan of each target part in the target part set corresponding to the current preset cruise point is completed by capturing and scanning the above multiple scenes or scene combinations.

[0056] It is understood that each current preset body position may correspond to more than one current preset cruise point. When a current preset body position corresponds to two or more current preset cruise points, the aforementioned execution entities sequentially perform the corresponding operations on each current preset cruise point until the cruise scan of all current preset cruise points under the current preset body position is completed. The completion of the cruise scan of each current preset cruise point includes: completing a complete scan of all target parts in the target part set corresponding to the current preset cruise point by capturing and scanning multiple scenes or scene combinations constructed from the feature parts in the feature part set corresponding to the current preset cruise point.

[0057] Similarly, when there are multiple current preset body positions, each of the above-mentioned execution entities sequentially performs the corresponding operations on each current preset body position until the cruise scan of all the current preset cruise points under all the current preset body positions of the examinee is completed.

[0058] This invention provides a capsule endoscope cruise control system, comprising: a computer host, a control module, a robotic arm 01, an end-effector rotation axis 02, an end-effector magnet, an examination table 04, an ultrasound matrix panel 05, and a depth camera. The ultrasound matrix panel 05 is placed on the abdomen of the patient. In a preset position, the depth camera captures an image of the patient and constructs a three-dimensional point cloud. The computer host determines the shoulder and hip joint positions of the patient based on the three-dimensional point cloud. The computer host locates the position of an initial stomach model in the three-dimensional point cloud based on the shoulder and hip joint positions of the patient. The ultrasound matrix panel 05 detects the ultrasound echo signal of the patient's stomach contour and sends the ultrasound echo signal to the computer host. The computer host generates a digital three-dimensional stomach contour based on the ultrasonic echo signal. The computer host then corrects the shape, size, and position of the initial stomach model in the three-dimensional point cloud based on the digital three-dimensional stomach contour, obtaining a stomach model. The computer host maps the current preset cruise point of the stomach model onto the subject's three-dimensional point cloud. The control module controls the robotic arm 01 to move the end magnet to the current preset cruise point. The control module controls the robotic arm 01 to move the end magnet, thereby driving the capsule endoscope to scan each feature part of the feature set corresponding to the current preset cruise point of the stomach to achieve scene capture, thus completing a complete scan of each target part in the target part set corresponding to the current preset cruise point. In this system, the corrected stomach model's three-dimensional point cloud coordinates can provide precise positions of the current preset cruise point, thereby guiding the robotic arm 01 to move the end magnet along each current preset cruise point to control the movement of the capsule endoscope. This achieves precise and effective control of the capsule endoscope, improving the efficiency of capsule endoscopy while ensuring the integrity of the capsule endoscope examination of the stomach.

[0059] In some embodiments, the computer host locates the three-dimensional position of the capsule endoscope in the stomach using ultrasound; the computer host maps the three-dimensional position of the capsule endoscope in the stomach onto the stomach model; after the capsule endoscope has scanned the target area, the computer host marks the area where the target area is located on the stomach model as the scanned area.

[0060] Specifically, in this system, the ultrasound matrix panel 05 is placed above the patient's abdomen. After the patient swallows the ultrasound contrast agent and the capsule endoscope, the contours of the stomach and the capsule endoscope can be seen under the action of ultrasound echoes. The depth and position of the capsule endoscope within the stomach can also be reflected by the ultrasound echoes. The ultrasound contour of the stomach is used to correct the initial three-dimensional stomach model, resulting in a corrected stomach model. Based on the coordinates provided by the stomach model, the system guides the robotic arm 01 to drive the end magnet 03 to guide the capsule endoscope. Then, the ultrasound echo feedback accurately determines the coordinate position of the capsule endoscope and its target location within the stomach. Therefore, the system can accurately determine whether the capsule endoscope has scanned the target area, and can easily guide the capsule endoscope to observe unscanned areas, thereby greatly improving the efficiency and completeness of stomach examinations. In some embodiments, the ultrasound matrix panel 05 includes crystal probes arranged in an m x n matrix, where m and n are positive integers.

[0061] In this system, the probe of the ultrasound matrix panel 05 is an electronic matrix probe, which is a crystal probe with a multi-row crystal structure. Unlike mechanical probes, electronic matrix probes electronically deflect the ultrasound beam to scan a predetermined volume area for volumetric data acquisition. This results in faster volumetric data acquisition and higher resolution compared to mechanical probes. The ultrasound matrix probe arranges multiple crystal probes in an mxn matrix to achieve real-time three-dimensional image generation. By covering the human stomach with this ultrasound matrix panel 05, the position of the capsule endoscope and the contour of the stomach can be determined in the XY and Z (depth) directions through the reflected echoes of the sound wave matrix. Combined with the image results recognized by the AI ​​model, the position of the capsule endoscope in the stomach can be accurately determined, and the capsule endoscope can be controlled to cruise and detect within the stomach. Its advantage lies in directly converting the current traditional medical B-mode longitudinal section scan into a three-dimensional surface scan. The echo value returned by each crystal probe in the matrix in the depth direction (Z-axis) is combined with the crystal probe array in the XY direction to form a three-dimensional image. This three-dimensional image can obtain both the contour of the stomach and the coordinate position of the capsule endoscope, and the data can be returned in real time.

[0062] In some embodiments, the frequency of the crystal probe is 3.5 to 5 MHz.

[0063] In some embodiments, the control module controls the robotic arm 01 to move the end magnet, thereby driving the capsule endoscope to scan each feature part of the feature part set corresponding to the current preset cruise point in the stomach to achieve scene capture, so as to complete a complete scan of each target part in the target part set corresponding to the current preset cruise point. This includes: for each target part in the target part set corresponding to the current preset cruise point, the computer host selects the optimal scene combination from the scene combination set corresponding to the target part based on the feature part set corresponding to the current preset cruise point; the control module controls the robotic arm 01 to move the end magnet, thereby driving the capsule endoscope to capture the optimal scene combination; when the optimal scene combination is captured, the target part is completely scanned.

[0064] Specifically, based on the quality control method disclosed in Chinese patent application CN114259197A, for the integrity observation of each of the 24 target sites in the stomach, the quality control method will yield a series of scenarios or combinations of scenarios that meet the requirements. If any scenario or combination of scenarios corresponding to a target site is observed, then that target site is considered to have been observed completely. However, it is impossible to search and observe every scenario combination for every target site within the limited inspection time; therefore, it is necessary to select the most suitable scenario or combination of scenarios for searching.Chinese invention patent application CN115251808A, entitled "Scene-Guided Capsule Endoscope Control Method, Device, and Storage Medium," discloses the following solution: A scoring system evaluates, scores, and ranks each scene combination in the scene combination set corresponding to one of the target parts (the current target part) adjacent to the current feature part. The scene combination with the highest score is taken as the optimal scene combination. The capsule endoscope is then controlled to search and scan this optimal scene combination to complete the full observation of the current target part. The next target part adjacent to the current target feature part is taken as the current target part, and the above operation is repeated until the observation of the current target part is completed. Complete observation of all adjacent target parts of a feature region is achieved by searching and scanning for the optimal scene combination for each target part, ensuring observation completeness while improving inspection efficiency. The preferred scene scoring criteria include: determining whether each scene combination is a single scene or multiple scenes; if it is a single scene, the scene combination scores points; otherwise, it scores no points. It also determines whether each scene in the scene combination contains a single feature region or multiple feature regions; if it contains a single feature region, the scene scores points; otherwise, it scores no points. Finally, it determines whether each scene in the scene combination is easily observable; if the scene is easily observable... If a scene is easily observed, it scores a point; otherwise, it scores no point. "Easily observed" means that the characteristic parts of the scene are not obscured by foam or mucus-like suspended matter during scene capture. The scene composition determines whether the main visible parts of each scene belong to "strong observation." If the main visible parts are considered "strong observation," the scene scores a point; otherwise, it scores no point. "Strong observation" means that the main visible parts can be identified and their specific locations are clear. The scene composition also determines whether all scenes are within the currently examined body position. If all scenes are within the currently examined body position... If a scene is described, it scores points; otherwise, it scores no points. The process involves determining whether each scene in a scene combination is shared by at least two other scene combinations. If a scene is shared by at least two other scene combinations, it scores points; otherwise, it scores no points. It also involves determining whether only one scene in a scene combination remains unobserved. If so, the scene scores points; otherwise, it scores no points. Finally, it involves determining whether the currently observed feature in a scene combination is a feature in a candidate scene combination, or an adjacent feature in a candidate scene combination. If so, the scene scores points; otherwise, it scores no points. For detailed implementation methods, please refer to the specific embodiments described in Chinese Patent Application Publication No. CN115251808A, which will not be repeated here.By capturing and scanning the optimal scene combination selected from the scene combination set corresponding to each target part of the stomach, the efficiency of the examination is improved while ensuring the integrity of the stomach observation.

[0065] In some embodiments, the current preset body position is supine, left lateral decubitus, or right lateral decubitus.

[0066] In some embodiments, in the supine position, the distal magnet moves in the region from the left side of the abdomen to the umbilicus, with the xiphoid process as the reference point; in the left lateral position, the distal magnet moves in the region below the back of the subject, close to the heart; and in the right lateral position, the distal magnet moves in the region on the left side of the abdomen of the subject, near the umbilicus.

[0067] Specifically, in the supine position, the end magnet moves in the area from the left side of the abdominal cavity to the navel, with the xiphoid process as the reference. In the second coordinate system where the robotic arm 01 is located, the robotic arm 01 is controlled to guide the end magnet to perform translation and / or rotation in the XY plane, while the Z-axis direction remains unchanged. In this position, the main target areas of the upper and lower parts of the stomach are observed.

[0068] In the left lateral decubitus position, the end magnet moves in the area below the subject's back near the heart. In the second coordinate system where the robotic arm 01 is located, the robotic arm 01 is controlled to guide the end magnet to perform translation and / or rotation in the XZ plane, while the Y-axis direction remains unchanged. In this position, the target area in the middle of the stomach is mainly observed.

[0069] In the right lateral decubitus position, the end magnet moves in the area near the navel on the left side of the subject's abdomen. In the second coordinate system where the robotic arm 01 is located, the robotic arm 01 is controlled to guide the end magnet to perform translation and / or rotation in the XZ plane, while the Y-axis direction remains unchanged. In this position, the target area in the lower part of the stomach is mainly observed.

[0070] In some embodiments, in the supine position, the current preset cruise point is the fundus of the stomach, the anterior wall of the middle part of the stomach, and the anterior wall of the antrum; in the left lateral decubitus position, the current preset cruise point is the cardia; and in the right lateral decubitus position, the preset cruise point is the pylorus.

[0071] In some embodiments, the set of characteristic locations corresponding to the fundus of the stomach at the current preset cruise point includes the cardia, fundus, lesser curvature, and gastric body; the set of characteristic locations corresponding to the forearm of the middle of the stomach at the current preset cruise point includes the lesser curvature, gastric angle, and gastric antrum; the set of characteristic locations corresponding to the anterior wall of the gastric antrum at the current preset cruise point includes the gastric angle, gastric antrum, and pylorus; the set of characteristic locations corresponding to the cardia of the current preset cruise point includes the cardia, greater curvature, and gastric body; and the set of characteristic locations corresponding to the pylorus of the current preset cruise point includes the gastric angle, gastric antrum, pylorus, and gastric body.

[0072] In some embodiments, the target location set corresponding to the current preset cruise point at the gastric fundus includes the target location cardia, gastric fundus, posterior wall of the lower gastric cardia, lesser curvature of the upper stomach, posterior wall of the upper stomach, lesser curvature of the middle stomach, and posterior wall of the middle stomach; the target location set corresponding to the current preset cruise point at the forearm of the middle stomach includes the target location lesser curvature of the lower stomach, posterior wall of the lower stomach, greater curvature of the lower stomach, gastric angle, and posterior wall of the gastric angle.

[0073] The target location set corresponding to the anterior wall of the gastric antrum at the current preset cruise point includes the target locations: the greater curvature of the lower stomach, the gastric angle, the posterior wall of the gastric angle, the lesser curvature of the gastric antrum, the posterior wall of the gastric antrum, and the greater curvature of the gastric antrum.

[0074] The target site set corresponding to the current preset cruise point cardia includes the target site anterior wall of the cardia, anterior wall of the upper stomach, greater curvature of the upper stomach, posterior wall of the upper stomach, anterior wall of the middle stomach, greater curvature of the middle stomach, and posterior wall of the middle stomach.

[0075] The target location set corresponding to the current preset cruise point pylorus includes the target locations: anterior wall of the gastric angle, posterior wall of the gastric angle, greater curvature of the lower stomach, anterior wall of the gastric antrum, posterior wall of the gastric antrum, lesser curvature of the gastric antrum, greater curvature of the gastric antrum, and pylorus.

[0076] like Figure 3 As shown, during the examination, the orientation adjustment of the capsule endoscope from one feature area to another is mainly accomplished by the R1 and R2 axes of the end rotation axis O2. Its steering mechanism is mainly manifested in that the R1 axis rotates around the Z axis in the XY plane, and the R2 axis rotates around an axis perpendicular to the Z axis and in the positive and negative directions of the Z axis.

[0077] The following example illustrates how the R1 and R2 axes of the distal rotation axis O2 are adjusted when the capsule endoscope moves from one feature site to another under different body positions while scanning various feature sites:

[0078] The following is an example of the directional adjustment relationship between characteristic body parts when the subject is in a supine position:

[0079] Turn from the cardia (lower esophageal sphincter) towards the fundus (lower esophageal fundus is in a clockwise direction from the cardia). Once the capsule endoscope lens is focused on the cardia, adjust the R1 axis to rotate clockwise until the capsule endoscope can observe the fundus.

[0080] Starting from the cardia (lower esophageal sphincter), rotate towards the lesser curvature of the stomach. The lesser curvature is counter-clockwise from the cardia. Once the capsule endoscope lens is focused on the cardia, adjust the R1 axis to rotate counter-clockwise until the lesser curvature is visible through the capsule endoscope. An example of the directional adjustment between the key sites is shown below when the patient is in the left lateral decubitus position:

[0081] Turn from the cardia, a key feature, toward the greater curvature of the stomach. The greater curvature is slightly below the cardia in a clockwise direction. Once the capsule endoscope lens is focused on the cardia, adjust the R1 axis to rotate clockwise while simultaneously adjusting the R2 axis to rotate in the positive direction, i.e., rotate the R2 axis upwards, until the greater curvature of the stomach is observed by the capsule endoscope.

[0082] Turn from the characteristic location of the greater curvature of the stomach towards the cardia. The cardia is slightly above the greater curvature of the stomach in a counterclockwise direction. After the capsule endoscope lens focuses on the greater curvature of the stomach, adjust the R1 axis to rotate counterclockwise, and at the same time adjust the R2 axis to rotate in the negative direction, that is, rotate the R2 axis downward, until the cardia is observed by the capsule endoscope.

[0083] With the subject in the right lateral decubitus position, the following is an example of the directional relationship between the features:

[0084] Turn from the gastric angle to the gastric antrum, which is located below the gastric angle in a clockwise direction. After the capsule endoscope lens focuses on the gastric angle, adjust the R1 axis to rotate clockwise, and at the same time adjust the R2 axis to rotate in the positive direction, that is, rotate the R2 axis upward, until the capsule endoscope can observe the gastric antrum.

[0085] Turn from the pylorus, a key feature, toward the gastric angle, which is above the pylorus. Once the capsule endoscope lens is focused on the pylorus, adjust the R2 axis to rotate in the negative direction, i.e., rotate the R2 axis downwards, until the capsule endoscope can observe the gastric angle.

[0086] This invention provides a capsule endoscopy cruise control system. In this system, a capsule endoscopy is performed on the patient's stomach in three positions: supine, left lateral decubitus, and right lateral decubitus. In each position, a three-dimensional point cloud reconstruction is performed to locate the initial stomach model within the three-dimensional point cloud. Ultrasound is then used to correct the initial stomach model's position in the three-dimensional point cloud, ensuring that the corrected position of the stomach model in the patient's three-dimensional point cloud matches the actual position of the stomach within the patient's body. The current preset cruise point of the stomach corresponds to the current preset cruise point in the stomach model. Each current preset cruise point in the stomach model is then controlled via the stomach model... The model is mapped onto the surface of the subject's three-dimensional point cloud. In each position, the control robotic arm 01 guides the end-effector magnet to the three-dimensional coordinate position of the current preset cruise point on the subject's three-dimensional point cloud surface, thereby driving the capsule endoscope to the current preset cruise point in the stomach. Simultaneously, the three-dimensional coordinate position of the capsule endoscope in the stomach is located using ultrasonic echo, thus achieving precise positioning and control of the capsule endoscope. The end-effector magnet drives the capsule endoscope to scan the target area near the current preset cruise point based on a scene-guided cruise method. When the optimal scene combination selected from the scene combination set corresponding to the target area is captured and scanned, the target area is completely examined. The system provided by this embodiment of the invention achieves precise positioning and control of the capsule endoscope, and the system can accurately determine whether the capsule endoscope has scanned the area where the target area is located. It can also easily guide the capsule endoscope to observe unscanned areas, improving examination efficiency and detection completeness.

[0087] The optional embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the embodiments of the present invention are not limited to the specific details in the above embodiments. Within the scope of the technical concept of the embodiments of the present invention, various simple modifications can be made to the technical solutions of the embodiments of the present invention, and these simple modifications all fall within the protection scope of the embodiments of the present invention.

[0088] It should also be noted that the various specific technical features described in the above embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, the embodiments of the present invention will not describe the various possible combinations separately.

[0089] Furthermore, various different implementations of the present invention can be combined arbitrarily, as long as they do not violate the spirit of the present invention, they should also be regarded as the content disclosed in the present invention.

Claims

1. A capsule endoscope cruise control system, characterized in that, include: The system includes a computer host, a control module, a robotic arm, an end-effector, an end magnet, an examination bed, an ultrasound matrix panel, and a depth camera. The ultrasound matrix panel is placed on the abdomen of the patient and includes crystal probes arranged in an m x n matrix, where m and n are positive integers. Under the current preset body position, the depth camera captures images of the subject and constructs a three-dimensional point cloud; The computer host determines the shoulder joint position and hip joint position of the subject based on the three-dimensional point cloud; the computer host locates the position of the initial stomach model in the three-dimensional point cloud based on the shoulder joint position and hip joint position of the subject; The ultrasound matrix panel detects the ultrasound echo signal of the patient's stomach contour and sends the ultrasound echo signal to the computer host. The computer host generates a digital three-dimensional stomach contour based on the ultrasonic echo signal; The computer host corrects the shape, size, and position of the initial stomach model in the three-dimensional point cloud based on the digital three-dimensional stomach contour to obtain the stomach model; The computer host maps the current preset cruise point of the stomach model onto the subject's three-dimensional point cloud; The control module controls the robotic arm to drive the end magnet to the three-dimensional coordinate position of the current preset cruise point on the subject's body surface; The control module controls the robotic arm to drive the end magnet to move, thereby driving the capsule endoscope to scan each feature part of the feature part set corresponding to the current preset cruise point in the stomach to achieve scene capture, so as to complete a complete scan of each target part in the target part set corresponding to the current preset cruise point.

2. The capsule endoscope cruise control system according to claim 1, characterized in that, The computer host uses ultrasound to locate the three-dimensional position of the capsule endoscope in the stomach. The computer host maps the three-dimensional position of the capsule endoscope in the stomach onto the stomach model; After the capsule endoscope has scanned the target area, the computer host marks the area where the target area is located as the scanned area on the stomach model.

3. The capsule endoscope cruise control system according to claim 1, characterized in that, The frequency of the crystal probe is 3.5 to 5 MHz.

4. The capsule endoscope cruise control system according to claim 1, characterized in that, The control module controls the robotic arm to move the end magnet, thereby driving the capsule endoscope to scan the feature areas of the stomach corresponding to the current preset cruise point to achieve scene capture, so as to complete a complete scan of each target area in the target area set corresponding to the current preset cruise point, including: For each target location in the target location set corresponding to the current preset cruise point, the computer host selects the optimal scene combination from the scene combination set corresponding to the target location based on the feature location set corresponding to the current preset cruise point. The control module controls the robotic arm to move the end magnet, thereby driving the capsule endoscope to capture the optimal scene combination; When the optimal scene combination is captured, the target area is completely scanned.

5. The capsule endoscope cruise control system according to claim 1, characterized in that, The currently preset body positions are supine, left lateral decubitus, and right lateral decubitus.

6. The capsule endoscope cruise control system according to claim 5, characterized in that, In the supine position, the distal magnet moves in the area from the left side of the abdominal cavity to the navel, with the xiphoid process of the subject as the reference. In the left lateral decubitus position, the end magnet moves in the area below the subject's back near the heart. In the right lateral decubitus position, the end magnet moves in the area near the navel on the left side of the subject's abdomen.

7. The capsule endoscope cruise control system according to claim 5, characterized in that, In the supine position, the current preset cruise points are the fundus of the stomach, the anterior wall of the middle part of the stomach, and the anterior wall of the antrum of the stomach. In the left lateral decubitus position, the current preset cruise point is the cardia; In the right lateral decubitus position, the current preset cruise point is the pylorus.

8. The capsule endoscope cruise control system according to claim 7, characterized in that, The set of characteristic locations corresponding to the gastric fundus at the current preset cruise point includes the characteristic locations cardia, gastric fundus, lesser curvature, and gastric cavity; the set of characteristic locations corresponding to the anterior wall of the middle part of the stomach at the current preset cruise point includes the characteristic locations lesser curvature, gastric angle, and gastric antrum. The set of characteristic locations corresponding to the anterior wall of the gastric antrum at the current preset cruise point includes the characteristic locations of the gastric angle, gastric antrum, and pylorus. The set of characteristic locations corresponding to the current preset cruise point cardia includes the characteristic locations cardia, greater curvature of the stomach, and gastric cavity; The set of characteristic locations corresponding to the current preset cruise point pylorus includes the characteristic locations of gastric angle, gastric antrum, pylorus and gastric cavity.

9. The capsule endoscope cruise control system according to claim 7, characterized in that, The target location set corresponding to the current preset cruise point gastric fundus includes the target locations cardia, gastric fundus, posterior wall of the cardia, lesser curvature of the stomach, posterior wall of the stomach, lesser curvature of the stomach, and posterior wall of the stomach. The target location set corresponding to the current preset cruise point on the anterior wall of the middle of the stomach includes the target locations: the lesser curvature of the lower stomach, the posterior wall of the lower stomach, the greater curvature of the lower stomach, the gastric angle, and the posterior wall of the gastric angle. The target location set corresponding to the anterior wall of the gastric antrum at the current preset cruise point includes the target locations: the greater curvature of the lower stomach, the gastric angle, the posterior wall of the gastric angle, the lesser curvature of the gastric antrum, the posterior wall of the gastric antrum, and the greater curvature of the gastric antrum. The target location set corresponding to the current preset cruise point cardia includes the target location anterior wall of the cardia, anterior wall of the upper stomach, greater curvature of the upper stomach, posterior wall of the upper stomach, anterior wall of the middle stomach, greater curvature of the middle stomach, and posterior wall of the middle stomach; the target location set corresponding to the current preset cruise point pylorus includes the target location anterior wall of the gastric angle, posterior wall of the gastric angle, greater curvature of the lower stomach, anterior wall of the gastric antrum, posterior wall of the gastric antrum, lesser curvature of the gastric antrum, greater curvature of the gastric antrum, and pylorus.

Images