Control system and method for automatic cavity-following endoscope entering of enteroscope, terminal and medium
Through the autonomous circumcision mirror control system, image acquisition and feature recognition technology are used, combined with expert control models, the autonomous circumcision mirror of colonoscopy is realized, solving the problems of relying on doctors' experience and high discomfort in the existing technology. It improves the intelligence and accuracy of the examination and is suitable for the promotion and popularization of colonoscopy.
Patent Information
- Application Number
- CN202411172294.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-26
- Publication Date
- 2024-11-29
AI Technical Summary
The existing colonoscopy technology relies on doctors' experience and is difficult to operate, which leads to the examination results vary from person to person, and the patient has a strong sense of discomfort, which limits the promotion and popularization of colonoscopy.
The control system of autonomous circulating mirror is adopted. Through the image acquisition module, feature recognition module and expert control model, the orientation of the colonoscopy is automatically adjusted to realize the autonomous circulating mirror of the colonoscopy, reduce the dependence on doctors' operating experience, and improve examinations. degree of intelligence and patient tolerance.
It improves the intelligence and accuracy of colonoscopy, reduces the difficulty of doctors and patient discomfort, and realizes an efficient and accurate examination process, which is suitable for the promotion and popularization of colonoscopy.
Smart Images

Figure CN119033324A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of colonoscopy, and in particular to a control system and method for autonomously advancing a colonoscope into a cavity. Background Art
[0002] Colonoscopy is a diagnostic method used by doctors to examine lesions inside the large intestine and colon. It requires a thin, flexible colonoscope to be passed from the anus into the intestine and eventually reach the cecum. Colonoscopy can effectively detect lesions in the intestine and is the most intuitive and effective method for benign and malignant tumors and precancerous lesions. However, because colonoscopy can cause obvious discomfort, and the operation of different doctors will result in completely different examination results and tolerance, the promotion and popularization of colonoscopy is severely limited.
[0003] There are currently two types of colonoscopy:
[0004] One is manual hand-held insertion of the endoscope. The doctor holds the colonoscope tube to advance and retract the endoscope, manually rotates the endoscope body, and bends the wheel to find the cavity and insert the endoscope. This method has high requirements for the doctor's experience and hand feel, and the control of force must be very fine, otherwise it is easy to lose the vision of the cavity, and it takes a lot of time and energy to find the cavity, and it is very easy to cause discomfort to the patient. In addition, hand-held insertion of the endoscope can easily cause soreness in the doctor's arm, making it difficult to complete multiple colonoscopy operations continuously. The long training time, high difficulty in operation, and high experience requirements result in the effect of colonoscopy surgery always varying from person to person, which seriously hinders the promotion and popularization of colonoscopy.
[0005] Another method is to transform the existing colonoscope into an electronically controlled one, using various motor-driven structural parts to replace the doctor's hands to complete the multi-degree-of-freedom control of the colonoscope tube. The doctor only needs to sit in front of the console and operate the joystick with both hands to complete the colonoscopy, which greatly reduces the doctor's physical burden. The force feedback data can also be fed back to the motor in time to prevent the accidental occurrence of gastric perforation. However, this method still needs to be highly dependent on the doctor's personal experience, how to keep the field of view in the best state, when to turn and bend the lens to achieve the best insertion effect, how to minimize the patient's discomfort, etc. These all require years of accumulation by the doctor to achieve, and this accumulation cannot be quantified, and it is difficult to form a set of effective practical theories for comprehensive promotion. Therefore, the effect of colonoscopy is still different from person to person. The same colonoscopy equipment has completely different examination effects and patient feelings in the hands of different doctors, and there are still great limitations on the promotion and popularization of colonoscopy.
[0006] After searching, the Chinese invention patent application number is CN202011332245.6, which provides an image processing method for providing endoscopic navigation information. It mainly provides an image processing method for providing endoscopic navigation information, by displaying an endoscopic map on a display device and marking the position of the head end of the endoscope on the map, thereby providing navigation information for doctors. This method can reduce the inspection and operation experience requirements of doctors for endoscopic inspection to a certain extent, but it still relies on doctors to manually operate the endoscope. Doctors still need to have certain experience and operation skills, and rely on the markings and navigation information on the endoscopic map. The real-time and accuracy are limited by the accuracy of the map and navigation system. Therefore, the degree of automation, accuracy and real-time performance need to be further improved. Summary of the invention
[0007] In view of the defects in the prior art, the purpose of the present invention is to provide a control system and method for autonomously advancing a colonoscope into the lumen.
[0008] A first aspect of the present invention provides a control system for autonomously advancing a colonoscope through a lumen, comprising:
[0009] An image acquisition module, which acquires images transmitted back from the cavity;
[0010] A feature recognition module, which recognizes the features and directions of the cavity according to the image;
[0011] The control module uses an expert control model to automatically control the enteroscope to adjust the position of the enteroscope according to the characteristics and direction of the cavity, thereby realizing autonomous enteroscope.
[0012] Among them, the expert control model calculates the deviation between the cavity and the intestinal tube lens through visual analysis, and uses the deviation information as a control signal to control the bending of the intestinal tube lens and the rotation of the tube body, so that the field of view center of the intestinal tube lens coincides with the center of the cavity.
[0013] Optionally, the control system: takes the cavity circulation as the optimization control target, always keeps the forward direction of the colonoscope consistent with the intestinal cavity; executes the first action of the action sequence obtained by open-loop control, then receives feedback from the environment, and then performs open-loop simulation on the transfer state of the expert control model, and then takes the first action until the decision is made; after the decision is made, the control instructions formed by the action sequence are output by the expert control model to control the colonoscope to complete automatic cavity circulation.
[0014] Optionally, the control module is connected to the colonoscope controller by signals, and the control module outputs a control signal to the colonoscope controller, which controls the advancement, withdrawal and steering of the colonoscope tube through the colonoscope controller, thereby achieving stable and precise control of the colonoscope and ultimately achieving autonomous advancement of the colonoscope.
[0015] A second aspect of the present invention provides a control method for autonomously advancing a colonoscope through a lumen, comprising:
[0016] Collect images transmitted back from the cavity;
[0017] Identifying the characteristics and direction of the cavity according to the image;
[0018] According to the characteristics and direction of the cavity, an expert control model is used to automatically control the enteroscope to adjust the position of the enteroscope, thereby achieving autonomous enteroscope through the cavity.
[0019] Among them, the expert control model calculates the deviation between the cavity and the intestinal tube lens through visual analysis, and uses the deviation information as a control signal to control the bending of the intestinal tube lens and the rotation of the tube body, so that the field of view center of the intestinal tube lens coincides with the center of the cavity.
[0020] A third aspect of the present invention provides an autonomous enteroscope system for advancing the endoscope through the lumen, comprising:
[0021] Colonoscopy;
[0022] A control system adopts the above-mentioned control system that can realize the autonomous insertion of the colonoscope into the cavity; the control system is connected to the colonoscope respectively.
[0023] A fourth aspect of the present invention provides an electronic terminal, comprising: a memory for storing a computer program; a processor for implementing the control method for autonomously advancing the colonoscope into the cavity when executing the computer program, or running the control system for autonomously advancing the colonoscope into the cavity.
[0024] A fifth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the control method for the autonomous intraluminal advancement of the colonoscope, or runs the control system for the autonomous intraluminal advancement of the colonoscope.
[0025] Compared with the prior art, the embodiments of the present invention have at least one of the following beneficial effects:
[0026] The system and method for autonomous lumen endoscope advancement provided in the embodiments of the present invention construct an expert control model with autonomous learning ability through the operating experience of excellent doctors, image and video data of relevant diagnosis and treatment processes, and corresponding measures. It can recognize the image information transmitted back from the lumen, and automatically control the controller at the front end of the colonoscope to complete the adjustment of the endoscope advancement position according to the characteristics and direction of the lumen. It can also complete autonomous lumen endoscope advancement through the endoscope advancement controller, thereby improving the intelligence level of colonoscopy, reducing the labor cost of colonoscopy, and greatly improving the endoscope advancement effect without the need for excessive experience accumulation. At the same time, it lowers the threshold for colonoscopy, and both real-time and accuracy can be guaranteed, which is of great significance to the promotion and popularization of colonoscopy.
[0027] The system and method for autonomous cavity-tracing endoscope advancement provided in the embodiments of the present invention can continuously optimize control and adaptively perform precise control through an expert control model with autonomous learning capabilities, is less likely to lose the cavity, and has high efficiency.
[0028] The system and method for autonomous transluminal endoscope advancement provided in the embodiments of the present invention adopt real-time force feedback adjustment, which will reduce the tolerance of patients and minimize the discomfort, thereby ultimately improving the tolerance of patients. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:
[0030] Figure 1 This is a block diagram of a control system module for autonomously advancing a colonoscope through a lumen in one embodiment of the present invention;
[0031] Figure 2 This is a flow chart of a control method for autonomously advancing a colonoscope into a lumen according to an embodiment of the present invention;
[0032] Figure 3 FIG. 4 is a structural diagram of a colonoscopy system in one embodiment of the present invention. DETAILED DESCRIPTION
[0033] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements may be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.
[0034] Existing endoscope (colonoscopy) inspection systems rely on doctors’ operation. Although some systems have navigation information / maps to assist doctors’ operation, the degree of automation is low, and fully automated colonoscopy cannot be achieved. The inspection efficiency and effect are highly dependent on the doctor’s operation level, and it is impossible to completely eliminate the loss of vision or patient discomfort caused by improper doctor operation. For complex cavity structures, navigation information may not be sufficient to provide sufficient operational guidance, resulting in reduced inspection efficiency. In addition, if you rely on the markings and navigation information on the endoscope inspection map, the real-time and accuracy of the inspection are limited by the accuracy of the map and navigation system, and cannot meet the requirements well. Because it relies on doctor operation, user experience and cost also need to be improved.
[0035] In order to solve one of the above technical problems, the embodiment of the present invention provides a control system and method for autonomously advancing a colonoscope through a cavity, which adopts an expert control model, automatically calculates the deviation between the cavity and the intestinal tube lens through visual analysis, and automatically controls the colonoscope to complete the adjustment of the position of the colonoscope. Autonomously advancing the colonoscope through a cavity is achieved, which reduces the dependence on the doctor's operating experience and improves the intelligence of the examination. Through the expert control model with autonomous learning ability, the control strategy is continuously optimized to ensure the stability and accuracy of the colonoscopy process.
[0036] Reference Figure 1 The figure shows a block diagram of the control system module of the autonomous enteroscope through the cavity in one embodiment of the present invention. Specifically, the control system in this embodiment can realize the autonomous enteroscope through the cavity, and includes: an image acquisition module, a feature recognition module and a control module, wherein the image acquisition module acquires the image transmitted back from the cavity; the feature recognition module recognizes the characteristics and direction of the cavity according to the image; the control module uses an expert control model to automatically control the enteroscope to adjust the position of the enteroscope according to the characteristics and direction of the cavity, thereby realizing the autonomous enteroscope through the cavity.
[0037] In this embodiment, the image transmitted back from the cavity is acquired in real time through the image acquisition module, the feature recognition module accurately recognizes the features and direction of the cavity, and the expert control model calculates the deviation between the cavity and the intestinal tube lens through visual analysis. The deviation information is used as a control signal, and the control module adjusts the orientation of the colonoscope (controls the bending of the intestinal tube lens and the rotation of the tube body) so that the center of the field of view of the intestinal tube lens coincides with the center of the cavity. This enables autonomous insertion of the scope into the cavity without relying on personal experience, improves the intelligence of colonoscopy, and improves the accuracy and real-time nature of the examination. At the same time, it reduces the labor cost of colonoscopy and greatly improves the effect of inserting the scope.
[0038] In some preferred embodiments of the present invention, in the control module, an expert control model with autonomous learning ability is constructed through the operation experience of excellent doctors, image and video data of historical diagnosis and treatment processes, and corresponding measures. The expert control model with autonomous learning ability can continuously optimize the control and improve the intelligence level of colonoscopy, without the need for the operator to accumulate too much personal experience, greatly improving the effect of inserting the scope.
[0039] Specifically, in some embodiments, in order to obtain an expert control model with autonomous learning capability, the control module may further include:
[0040] A data acquisition module, which acquires video data of the colonoscopy operation and corresponding diagnosis and treatment data, and annotates the video data, wherein the video data includes the endoscope image and the doctor's operation action;
[0041] A feature extraction and modeling module extracts the image features of the cavity from the endoscopic images collected by the data acquisition module, and uses machine learning and deep learning technologies to model the extracted image features and the doctor's operation actions to form the initial state of the expert control model;
[0042] The model training and optimization module uses the labeled endoscopic image data and the doctor's operation action data to train the expert control model, so that the expert control model can simulate the doctor's operation behavior, and through the reinforcement learning method, the expert control model can continuously try and optimize the control strategy in the simulation environment to improve the autonomous learning ability.
[0043] The above data collection and preprocessing module, when implemented, can collect a large amount of video data and related diagnosis and treatment data of excellent doctors performing colonoscopy, including endoscopic images, doctors' operation actions and feedback information. The video data is then annotated to extract cavity features and doctors' operation strategies, including but not limited to rotation angle, advancement speed, etc.
[0044] The above-mentioned feature extraction and modeling module, in which the image features of the cavity, including shape, direction, texture, etc., are extracted from the endoscopic image through an image processing algorithm.
[0045] The above-mentioned model training and optimization module realizes the optimization and self-learning of the expert control model. The expert control model gradually accumulates experience and continuously optimizes its own control strategy through repeated training in a simulated environment. Using the reinforcement learning algorithm, the expert control model adjusts the operation strategy according to the feedback information to achieve the optimal control effect.
[0046] In some preferred embodiments, the expert control model in the control module can optimize the control strategy in the following manner: In each operation, the expert control model predicts the next best operation based on the current image features of the cavity and historical operation records. At the same time, through the feedback mechanism, the expert control model adjusts the operation strategy in real time to ensure that the center of the field of view of the colonoscope coincides with the center of the cavity.
[0047] In some preferred embodiments of the present invention, the control module performs deviation calculation of the expert control model and cavity-based mirror control.
[0048] Specifically, the expert control model calculates the deviation between the cavity and the intestinal lens through visual analysis, including direction deviation and position deviation. Using computer vision technology, the endoscope image is analyzed in real time to calculate the deviation between the center line of the cavity and the center of the colonoscope field of view.
[0049] Specifically, based on the calculated deviation, the expert control model generates control signals to control the bending and rotation of the colonoscope. Through the control module, the angle and force of the colonoscope are adjusted in real time to ensure that the colonoscope always moves along the center of the cavity.
[0050] In the above embodiment of the present invention, the image of the intestine is obtained in real time by installing a camera on the tube and the top of the colonoscope. The cavity features such as edges, textures, directions, etc. are extracted by image processing algorithms. Furthermore, by using the operation videos and annotation data of excellent doctors, the machine learning model is trained, and the reinforcement learning algorithm is adopted to continuously optimize the control strategy of the model and improve the autonomous learning ability. The control module calculates the deviation between the cavity and the intestinal tube lens through visual analysis, and generates a control signal based on the deviation information to control the bending and rotation of the colonoscope.
[0051] Through the above-mentioned technology, colonoscopy can be made intelligent, autonomous and precise, which not only reduces the difficulty of operation for doctors, but also improves the inspection efficiency and the comfort of patients, thus overcoming the shortcomings of existing technologies.
[0052] Specifically, in a preferred embodiment, the control module for the autonomous intraluminal advancement of the colonoscope takes intraluminal advancement as the optimization control target and always keeps the advancing direction of the colonoscope consistent with the intestinal cavity; executes the first action of the action sequence obtained by open-loop control, and then receives feedback from the environment (the cavity image and force information obtained in real time by the camera and pressure sensor at the front end of the colonoscope), and then executes open-loop simulation on the transfer state of the expert control model, and then takes the first action until the decision is made; after the decision is made, the control instructions formed by the action sequence are output by the expert control model to control the colonoscope to complete the automatic intraluminal advancement.
[0053] In the embodiment of the present invention, the autonomous insertion of the colonoscope into the cavity is realized through the expert control model, which has a high degree of automation. The intelligent level of colonoscopy can be improved through the learning and optimization of the expert control model, and an efficient and accurate inspection process can be realized. The high degree of automation greatly reduces the workload of doctors and the requirements for operating experience, which is conducive to the promotion and popularization of colonoscopy. In the embodiment of the present invention, the expert control model adjusts the insertion angle and force of the colonoscope in real time through the force feedback module to minimize the patient's discomfort. Automated insertion reduces the workload of doctors, shortens the inspection time, and improves the inspection efficiency. By reducing the requirements for doctors' experience, it reduces training costs and labor costs, and improves the inspection effect and patient experience. Ultimately, a higher degree of automation, accuracy, and real-time performance can be achieved, and the intelligent level of colonoscopy and user experience can be improved.
[0054] In some embodiments of the present invention, in order to capture images in the intestinal cavity, the above-mentioned image acquisition module may include a camera arranged on the tube and the top of the colonoscope, and the camera obtains video images during the process of entering the colonoscope, and the cavity characteristics are identified through the image. The use of a camera arranged on the tube and the top of the colonoscope as an image acquisition module is only a preferred embodiment of the present invention, and the cavity characteristics are identified through the images obtained by the lens, thereby simplifying the operation control and improving the integration and practicality of the system. Of course, in other embodiments, other settings can also be used to realize the video image during the process of entering the colonoscope.
[0055] In some embodiments of the present invention, in order to reduce the patient's discomfort, the control module also includes a force feedback module, which obtains the pressure during the colonoscope insertion process through the pressure sensors set in the colonoscope pipeline and the top, and provides feedback. The force feedback module obtains pressure information in real time, optimizes the colonoscope insertion angle and force, and reduces the patient's discomfort. At the same time, the role of pressure feedback is to transmit real-time pressure data to the control module. By analyzing these feedback data, the control module can adjust the colonoscope insertion angle and force in real time according to the feedback pressure information to avoid discomfort caused by excessive pressure, thereby improving the patient's tolerance and ensuring the stability and accuracy of the inspection process.
[0056] In some embodiments of the present invention, the control module is connected to the colonoscope controller by signals, and the control module outputs a control signal to the colonoscope controller, which controls the advancement, withdrawal and steering of the colonoscope tube through the colonoscope controller, thereby achieving stable and precise control of the colonoscope and ultimately achieving autonomous advancement of the colonoscope.
[0057] Exemplarily, the pressure sensors on the colonoscope tube and top collect pressure data in real time during the process of inserting the scope, and transmit the collected pressure data to the control module in real time. The control module analyzes the current force conditions according to the pressure data, determines whether it is necessary to adjust the angle or force of the scope, and generates new control signals through the expert control model to adjust the bending angle and thrust of the colonoscope, ensuring that the pressure on the cavity wall is reduced while the scope is inserted into the cavity, thereby avoiding patient discomfort. In addition, the colonoscope controller may also include a torque sensor for the motor, which obtains the torque data of the motor during the process of controlling the insertion, withdrawal and steering of the colonoscope. Combined with the data from the torque sensor, the control module can control the movement of the colonoscope more accurately, further optimize the patient's examination experience, and ensure the safety and comfort of the colonoscopy.
[0058] Based on the same technical concept, in another embodiment of the present invention, a control method for autonomously advancing a colonoscope into a lumen is provided, including S100-S300:
[0059] S100, collecting images transmitted back from the cavity;
[0060] In this step, the image of the colonoscope in the process of entering the colonoscope can be obtained by using the camera at the top of the colonoscope tube. Of course, other image acquisition devices can also be used to achieve this in other embodiments.
[0061] S200, identifying the characteristics and direction of the cavity according to the image;
[0062] According to the acquired image, the characteristics and direction of the cavity in the image are identified. In this embodiment, the image is first preprocessed. The preprocessing may include grayscale processing, noise filtering, contrast enhancement, etc., wherein the grayscale processing may convert the color image into a grayscale image to simplify the calculation. Noise filtering uses a filter (such as a Gaussian filter or a median filter) to remove noise in the image, which can improve the image quality. Contrast enhancement uses methods such as histogram equalization to enhance the image contrast and highlight the cavity features. The acquired intestinal image is preprocessed to improve the effect of feature extraction.
[0063] After the above preprocessing, image processing technology is further used to extract the features of the cavity from the preprocessed image. Specifically, in one embodiment, feature extraction may include: edge detection, morphological processing and feature point detection, wherein edge detection uses edge detection algorithms (such as Canny edge detection) to identify the edges of the cavity and obtain the cavity contour; morphological processing fills the gaps in the cavity contour through morphological operations such as expansion and corrosion to obtain the complete cavity shape; feature point detection uses feature point detection algorithms (such as Harris corner point detection, SIFT, SURF, etc.) to identify key points in the cavity and extract feature point descriptors. Thus, the features of the cavity are extracted.
[0064] After obtaining the above-mentioned cavity features, further feature matching and direction identification are performed. In one embodiment, feature matching and direction identification may include:
[0065] Feature matching: Match the features of the current frame image with the features in the previous frames, and use the nearest neighbor search or FLANN matching algorithm to find matching feature point pairs;
[0066] Direction calculation: Calculate the direction of the cavity based on the matched feature point pairs. Estimate the direction and position change of the cavity through geometric transformation (such as affine transformation or perspective transformation);
[0067] Motion estimation: Use optical flow methods (such as the Lucas-Kanade optical flow method) or SLAM (simultaneous localization and mapping) technology to estimate the motion trajectory of the colonoscope relative to the cavity and further determine the direction of the cavity.
[0068] Based on the above feature extraction and direction recognition results, the characteristics and direction of the cavity are comprehensively judged, which can be determined by the centerline and curvature of the cavity. For example, the centerline of the cavity is calculated by fitting the edge or feature points of the cavity; the curvature of the cavity is calculated based on the centerline to determine the curvature and direction of the cavity. Furthermore, if there are multiple cameras, the depth information of the cavity can be calculated using stereo vision technology to further improve the recognition accuracy.
[0069] S300 uses an expert control model to automatically control the colonoscope to adjust its position according to the characteristics and direction of the cavity, thus achieving autonomous insertion of the colonoscope. The expert control model calculates the deviation between the cavity and the intestinal tube lens through visual analysis, and uses the deviation information as a control signal to control the bending of the intestinal tube lens and the rotation of the tube body, so that the center of the field of view of the intestinal tube lens coincides with the center of the cavity.
[0070] In this embodiment, the colonoscope includes a colonoscope controller. The expert control model transmits control signals of advancing, withdrawing and turning the colonoscope tube to the colonoscope controller, and the colonoscope controller controls the movement of the colonoscope lens.
[0071] In some specific embodiments, the colonoscope and its control can be completed by the following basic components:
[0072] Intestinal tube lens: located at the front end of the colonoscope, contains a camera and a light source, used to collect real-time images of the intestine and illuminate the cavity;
[0073] Colonoscope controller: used to receive control signals and perform corresponding operations to control the entry, withdrawal and steering of the colonoscope;
[0074] Enteroscope tube: a flexible tube that connects the enteroscope lens and the controller, contains a signal transmission line and a power transmission mechanism, and can bend and rotate to adapt to the shape of the cavity;
[0075] Image acquisition module: usually integrated into the intestinal lens, including a camera, used to collect images of the intestine in real time and transmit them to the control module for processing;
[0076] Sensing system: including pressure sensors and torque sensors, etc., which are used to monitor the force applied to the colonoscope in real time during the insertion process and feed this information back to the control module;
[0077] Light source system: provides lighting to ensure that the camera can clearly capture images of the intestines;
[0078] Power transmission system: including motor and mechanical transmission mechanism, used to drive the colonoscope to move forward, backward and turn;
[0079] Data processing and control module: integrates the expert control model, generates control signals and transmits them to the colonoscope controller by receiving and processing data from the sensor system and image acquisition module;
[0080] Communication module: used to realize data transmission and communication between the colonoscope and external equipment (such as monitoring equipment, doctor's operating table, etc.).
[0081] Through the coordinated work of the above-mentioned basic components, the colonoscope can be inserted into the cavity autonomously, thus improving the intelligence of the examination and the comfort of the patient.
[0082] Based on the same technical concept, in another embodiment of the present invention, a colonoscopy system with autonomous cavity insertion is provided, including: a colonoscope and a control system, wherein the control system is connected to the colonoscope respectively, and the control system adopts the control system described in any of the above embodiments that can realize the autonomous cavity insertion of the colonoscope.
[0083] In an embodiment of the present invention, the control system includes an expert control model. An expert control model with autonomous learning ability is constructed by integrating the operating experience of excellent doctors, image and video data of relevant diagnosis and treatment processes, and corresponding measures. The expert control model recognizes the image information transmitted back by the cavity, automatically adjusts the position of the front end of the colonoscope according to the characteristics and direction of the cavity, adjusts the angle and force of the scope through the force information transmitted back by the force feedback sensor, and the colonoscope controller executes these control instructions to complete autonomous cavity insertion. The expert control model can simulate different operating actions and make predictions, select the optimal action sequence from many prediction results, and plan and execute them to ensure that the colonoscope always maintains the best state during the examination.
[0084] Specifically, for the control system, in order to realize the above functions of the expert control model, collect operation videos of excellent doctors, images of related diagnosis and treatment processes and corresponding operation data, including real-time images captured by the front-end camera of the colonoscope, doctors' operation actions (such as forward, backward, turning, bending angle) and force feedback data. Annotate the collected data, including cavity features, doctors' operation actions and corresponding feedback information, and perform data cleaning and standardization to ensure data consistency and quality. Use image processing algorithms (such as convolutional neural networks CNN) to extract cavity features from intestinal images, including edges, textures and shapes, and use machine learning algorithms (such as support vector machines SVM and random forest RF) to extract key features in doctors' operation data.
[0085] Based on the above data, a supervised learning method is adopted to train the expert control model using labeled training data, and a deep learning algorithm (such as long short-term memory network LSTM and reinforcement learning RL) is used to train the model so that it can simulate the doctor's operating behavior.
[0086] After obtaining the expert control model, the expert control model is used to predict the current cavity characteristics and operation status, simulate possible operation actions, and generate multiple possible action sequences (such as forward, backward, turning, and adjusting angles). Each simulated action sequence can also be predicted to evaluate its effect. For example, a reinforcement learning algorithm (such as deep Q learning DQN, policy gradient PG) is used to score each action sequence, and it is evaluated based on its matching degree to the cavity characteristics, the smoothness of the operation, and the rationality of the force feedback. According to the predicted score, the action sequence with the highest score is selected from a large number of action sequences as the current optimal action sequence. This ensures that the selected action sequence can achieve the best endoscope effect while minimizing the patient's discomfort. Then, the selected optimal action sequence is planned to generate specific control instructions, which include operating parameters such as the advance, retreat, turn, and bend of the colonoscope. The planned control instructions are sent to the colonoscope controller, and the colonoscope controller adjusts the movement of the colonoscope in real time to achieve autonomous endoscope entry. During the above process, force feedback sensors and image sensors are used to monitor the operation effect in real time. According to the real-time feedback data, the control strategy is dynamically adjusted to ensure that the colonoscope always moves along the center of the cavity and optimize the inspection effect.
[0087] Through the above steps, the expert control model of the embodiment of the present invention can simulate actions, make predictions, and select the optimal action sequence for planning and execution according to the prediction results, so as to realize the autonomous insertion of the colonoscope into the cavity. This method makes full use of the operation experience and intelligent algorithms of excellent doctors, improves the intelligence level and operation accuracy of colonoscopy, and reduces the discomfort of patients.
[0088] In the above-mentioned embodiments of the present invention, the control system has a clear goal and the characteristics of the goal are obvious. The circulation of the cavity can be used as the optimization control goal to always keep the advancing direction of the colonoscope consistent with the intestinal cavity. The first action of the action sequence obtained by executing the open-loop control is then received. Then, the feedback from the environment is received, and then the open-loop simulation is performed again on the transfer state of the expert control model, and then the first action is taken until the decision is made. After the decision is made, the expert system outputs the control instructions formed by the action sequence to control the colonoscope to complete the automatic circulation of the cavity. In this way, the random prediction error is corrected at each stage, and a more accurate prediction result can be obtained, thereby making it possible to realize the advance action planning and precise control of the colonoscope.
[0089] Reference Figure 3 As shown in the figure, the colonoscope sensing system mainly obtains video images and pressure feedback during the intestinal endoscope insertion process by deploying pressure sensors and cameras on the colonoscope tube and the top; the colonoscope controller is mainly used to control the entry and exit of the colonoscope tube and the steering, which can achieve stable and accurate control of the colonoscope. The expert control model conducts comprehensive analysis and judgment through the information returned by the sensor system and its own knowledge base, outputs control signals, and realizes autonomous endoscope insertion.
[0090] The above-mentioned self-knowledge base is used to store and manage expert experience, operating rules and learned knowledge. Its construction process includes data collection and storage, data labeling and classification, and knowledge extraction and storage.
[0091] The data collection and storage described herein include:
[0092] Operation data: Collect a large amount of real-time data of colonoscopy performed by excellent doctors, including image data, operation movements, force feedback information, etc.
[0093] Diagnosis and treatment data: collect and organize relevant diagnosis and treatment data, including the characteristics of different intestinal lesions, key points of operation and precautions;
[0094] Literature and Standards: Integrate professional literature, operating guidelines and industry standards as reference materials for the knowledge base.
[0095] The data labeling and classification include:
[0096] Data annotation: annotate the collected data, mark cavity features, operation actions, feedback information, etc.;
[0097] Data classification: Classify the labeled data into different categories such as operation scenarios, cavity characteristics, and lesion types.
[0098] The knowledge extraction and storage includes:
[0099] Feature extraction: Use image processing and machine learning techniques to extract key features from the data, such as cavity shape, texture, operation mode, etc.
[0100] Rule extraction: Extract operating rules and experience through data mining and analysis, such as the best operating strategy under a specific cavity shape;
[0101] Knowledge storage: The extracted features, rules and experiences are stored in a structured form in the knowledge base to facilitate quick query and application.
[0102] The above-mentioned expert system conducts comprehensive analysis and judgment through the information sent back by the sensor system and its own knowledge base, including image information analysis, force feedback analysis, operation rule application, analysis and decision-making, among which:
[0103] The image information analysis includes:
[0104] Cavity feature recognition: Based on the returned image, the shape, direction, texture and other features of the cavity are identified;
[0105] Lesion detection: Detect lesion areas in the image and determine whether special operations or adjustment strategies are required.
[0106] The force feedback analysis comprises:
[0107] Stress condition: According to the information fed back by the pressure sensor, the stress condition of the colonoscope in the cavity is determined;
[0108] Torque analysis: Use torque sensor data to analyze the bending and turning status of the colonoscope tube to ensure the smoothness and safety of the operation.
[0109] The operation rule application includes:
[0110] Optimal operation strategy: retrieve and apply the optimal operation strategy from the knowledge base based on the identified cavity characteristics and stress conditions;
[0111] Real-time adjustment: If abnormal situations occur during actual operation (such as excessive pressure, deviation of vision, etc.), adjust the operation strategy according to real-time feedback.
[0112] The above-mentioned assessment and decision-making include:
[0113] Multi-dimensional analysis: Combine image information, force feedback and operation rules to comprehensively judge the current operation status from multiple dimensions;
[0114] Action prediction: Use the expert control model to simulate multiple possible operation actions, make predictions, and evaluate the effect of each action;
[0115] Optimal decision-making: Based on the prediction results, the optimal action sequence is selected and a control signal is generated to control the advancement, withdrawal and steering of the colonoscope.
[0116] Through the above methods, the expert control model's own knowledge base is constructed, and in actual operation, through comprehensive judgment, multi-dimensional analysis using image information and force feedback, and application of the best operation strategy, the stability and accuracy of colonoscopy can be ensured, and autonomous insertion of the endoscope can be achieved. This not only improves the intelligence and operation efficiency of colonoscopy, but also significantly reduces the discomfort of patients.
[0117] The above-mentioned colonoscopy system can be used to perform colonoscopy using automatic cavity-following technology. When in use, the doctor debugs the colonoscopy equipment and inserts the colonoscopy tube into the patient's anus. At this time, the colonoscopy clearly projects the intestinal image onto the screen. The expert control model analyzes and locates the intestinal cavity through real-time video images. At this time, the doctor turns on the automatic cavity-following function. The expert control model calculates the deviation between the cavity and the intestinal tube lens through visual analysis, and sends the deviation information to the colonoscopy controller. The colonoscopy controller converts the deviation information into a control signal to control the bending of the intestinal tube lens and the rotation of the tube body, so that the field of view center of the colonoscopy tube lens coincides with the center of the cavity. At this time, the colonoscopy controller controls the precise advancement of the colonoscopy tube, and fine-tunes the direction and angle in real time, always keeping the cavity in the center of the field of view while keeping the scope in place. During this process, the doctor can suspend the autonomous cavity-following function at any time and manually take over the equipment for further exploration of the lesion or biopsy. In this embodiment, during the process of automatic insertion of the colonoscope, the pressure sensor on the head of the colonoscope tube and the torque sensor of the motor on the colonoscope controller will provide real-time feedback of force information. The expert system (expert control model) analyzes the force information to reasonably adjust the angle and force of the insertion of the colonoscope to ensure that the discomfort to the patient is minimized.
[0118] When conventional hospitals conduct colonoscopy, they need experienced doctors to perform the examination and operation. The whole process is long and seriously consumes the physical strength of the doctors, making it difficult to conduct continuous examinations. In addition, the experience and abilities of different doctors vary, and the examination results and patient tolerance are also different. In areas with underdeveloped medical conditions, colonoscopy is difficult to popularize and cannot achieve the desired results. By using the automatic cavity-entering mirror method and system in the above-mentioned embodiments of the present invention, the expert system (expert control model) can reach the operating level of a first-class colonoscopy doctor through continuous learning and iteration. The whole process can be automated and precise, which greatly reduces the discomfort of patients during colonoscopy, improves the efficiency of the examination, and reduces the training threshold and difficulty of doctors. Colonoscopy can be quickly promoted and popularized in areas with underdeveloped medical conditions.
[0119] Based on the same technical concept, in another embodiment of the present invention, an electronic terminal is also provided, including: a memory for storing a computer program; a processor for implementing the control method for autonomous colonoscopy in any of the above embodiments when executing the computer program, or running the above control system for autonomous colonoscopy.
[0120] Optionally, the memory is used to store programs; the memory may include volatile memory (English: volatile memory), such as random-access memory (English: random-access memory, abbreviated: RAM), such as static random-access memory (English: static random-access memory, abbreviated: SRAM), double data rate synchronous dynamic random access memory (English: Double Data Rate Synchronous Dynamic Random Access Memory, abbreviated: DDR SDRAM), etc.; the memory may also include non-volatile memory (English: non-volatile memory), such as flash memory (English: flash memory). The memory is used to store computer programs (such as applications, functional modules, etc. that implement the above method), computer instructions, etc., and the above computer programs, computer instructions, etc. can be partitioned and stored in one or more memories. And the above computer programs, computer instructions, data, etc. can be called by the processor.
[0121] The above-mentioned computer programs, computer instructions, etc. may be stored in one or more memories in partitions, and the above-mentioned computer programs, computer instructions, data, etc. may be called by a processor.
[0122] The processor is used to execute the computer program stored in the memory to implement the various steps in the method involved in the above embodiment. For details, please refer to the relevant description in the above method embodiment.
[0123] The processor and the memory may be independent structures or integrated structures. When the processor and the memory are independent structures, the memory and the processor may be coupled and connected via a bus.
[0124] Based on the same technical concept, in another embodiment of the present invention, a computer-readable storage medium is further provided, on which a computer program is stored. When the computer program is executed by a processor, the control method for autonomously advancing the colonoscope into the cavity in any of the above embodiments is implemented, or the control system for autonomously advancing the colonoscope into the cavity is run.
[0125] Among them, computer-readable media include computer storage media and communication media, wherein the communication media include any media that facilitates the transmission of computer programs from one place to another. The storage medium can be any available medium that can be accessed by a general or special-purpose computer. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and the storage medium can be located in an ASIC. In addition, the ASIC can be located in a user device. Of course, the processor and the storage medium can also be present in a communication device as discrete components.
[0126] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0127] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0128] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0129] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0130] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various modifications or variations within the scope of the claims, which does not affect the essence of the present invention. The above preferred features can be used in any combination without conflicting with each other.
Claims
1. A control system for autonomously advancing a colonoscope through a lumen, characterized in that: include: An image acquisition module, which acquires images transmitted back from the cavity; A feature recognition module, which recognizes the features and directions of the cavity according to the image; The control module uses an expert control model to automatically control the enteroscope to adjust the position of the enteroscope according to the characteristics and direction of the cavity, thereby realizing autonomous enteroscope. Among them, the expert control model calculates the deviation between the cavity and the intestinal tube lens through visual analysis, and uses the deviation information as a control signal to control the bending of the intestinal tube lens and the rotation of the tube body, so that the field of view center of the intestinal tube lens coincides with the center of the cavity.
2. The control system for autonomous enteroscopic endoscope according to claim 1, characterized in that: The control system: Taking the cavity as the optimization control target, the advancing direction of the colonoscope is always kept consistent with the intestinal cavity; Execute the first action of the action sequence obtained by open-loop control, then receive feedback from the environment, perform open-loop simulation again on the transition state of the expert control model, and then take the first action until the decision is completed; After the decision is made, the expert control model outputs control instructions in the form of an action sequence to control the colonoscope to automatically advance into the cavity.
3. The control system for autonomous enteroscopic endoscope according to claim 1, characterized in that: The image acquisition module includes a camera arranged on the colonoscope tube and the top end, and the camera obtains video images during the process of entering the colonoscope.
4. The control system for autonomous enteroscopic endoscope according to claim 1, characterized in that: The control system also includes a force feedback module, which is used to obtain the pressure during the insertion of the colonoscope and provide feedback through pressure sensors arranged on the colonoscope pipeline and the top end.
5. The control system for autonomous enteroscopic endoscope according to any one of claims 1 to 4, characterized in that: The control module is connected to the colonoscope controller by signals, and the control module outputs a control signal to the colonoscope controller, which controls the advancement, withdrawal and steering of the colonoscope tube, thereby achieving stable and precise control of the colonoscope and ultimately achieving autonomous advancement of the colonoscope.
6. A control method for autonomously advancing a colonoscope through a lumen, characterized in that: include: Collect images transmitted back from the cavity; Identifying the characteristics and direction of the cavity according to the image; According to the characteristics and direction of the cavity, an expert control model is used to automatically control the enteroscope to adjust the position of the enteroscope, thereby achieving autonomous enteroscope through the cavity. Among them, the expert control model calculates the deviation between the cavity and the intestinal tube lens through visual analysis, and uses the deviation information as a control signal to control the bending of the intestinal tube lens and the rotation of the tube body, so that the field of view center of the intestinal tube lens coincides with the center of the cavity.
7. The control method for autonomously advancing a colonoscope through a lumen according to claim 6, characterized in that: The expert control model: Taking the cavity as the optimization control target, the advancing direction of the colonoscope is always kept consistent with the intestinal cavity; Execute the first action of the action sequence obtained by open-loop control, then receive feedback from the environment, perform open-loop simulation again on the transition state of the expert control model, and then take the first action until the decision is completed; After the decision is made, the expert control model outputs control instructions in the form of an action sequence to control the colonoscope to automatically advance into the cavity.
8. An autonomous enteroscope system, characterized in that: include: Colonoscopy; A control system adopts the above-mentioned control system that can realize the autonomous insertion of the colonoscope into the cavity; the control system is connected to the colonoscope respectively.
9. An electronic terminal comprising: Memory for storing computer programs; A processor is used to implement the control method for autonomous enteroendoscopic endoscope according to any one of claims 6-7 when executing the computer program, or to run the control system for autonomous enteroendoendoscopic endoscope according to any one of claims 1-5.
10. A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the control method for autonomous colonoscopy according to any one of claims 6 to 7, or runs the control system for autonomous colonoscopy according to any one of claims 1 to 5.
Citation Information
Patent Citations
Image processing method, device and equipment, medium and endoscope system
CN112465763A
Cited By
Lens inlet control system, control method and computer equipment
CN120859412A
Endoscope control system, control method and computer device
CN120859412B
