Autonomous control method of cardiovascular interventional robot based on deep learning and its application

By combining ultrasound sensors, posture sensors and deep learning models, the vascular interventional robot system is solved by solving the problem of doctors relying on experience in PCI surgery, achieving autonomous control and accurate catheter movement, improving surgical effectiveness and safety.

CN116021512BActive Publication Date: 2025-08-08TONGJI UNIV
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202211563325.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-12-07
Publication Date
2025-08-08
Estimated Expiration
2042-12-07

AI Technical Summary

Technical Problem

The existing PCI surgery depends on the experience of doctors, has the risk of operational errors, has a long learning curve, doctors' health is threatened by radiation, medical imaging results are poor, and the existing technology has failed to effectively realize the autonomous control and posture adjustment of cardiovascular interventional surgery robots.

Method used

The cardiovascular interventional robot system based on deep learning is adopted, combined with ultrasonic sensors and posture sensors, and through the U-Net deep learning network model and PnP algorithm, catheter state judgment and motion control are realized, intubated intravascular ultrasonic technology is integrated, and the data set is expanded to adapt to individual differences and reduce X-ray dependence.

Benefits of technology

It realizes the autonomous control of cardiovascular interventional surgery robots, reduces the burden on doctors, improves surgical results, avoids X-ray health hazards, and improves system robustness and accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116021512B_ABST
    Figure CN116021512B_ABST
Patent Text Reader

Abstract

The present invention discloses a deep learning-based autonomous control method for a cardiovascular interventional robot and its application. The method comprises the following steps: obtaining the ultrasonic signal and posture signal currently acquired by a cardiovascular interventional catheter through an ultrasonic sensor and a posture sensor; inputting the ultrasonic signal into an image segmentation model based on deep learning, and then inputting its output into a binary classifier to obtain the current state of the cardiovascular interventional catheter; simultaneously, performing a kinematic solution on the posture signal to obtain the current posture of the cardiovascular interventional catheter head, and then using a motion model to solve the current posture of the ultrasonic sensor; determining the control scheme for the cardiovascular interventional catheter at the next moment based on the current state of the cardiovascular interventional catheter and the current posture of the ultrasonic sensor, and sending the control scheme for the cardiovascular interventional catheter at the next moment to a catheter drive device to complete the action at the next moment, and then returning to the first step until all instructions are completed. The present invention can realize automatic control of a cardiovascular interventional surgical robot with high accuracy.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of medical robot technology, and in particular to a deep learning-based autonomous control method for a cardiovascular interventional robot and its application, and in particular to a deep learning-based autonomous control method, device, and storage medium for a cardiovascular interventional robot. Background Art

[0002] Percutaneous coronary intervention (PCI) is a technique in which a catheter is inserted from peripheral blood vessels and delivered to the heart and large blood vessels. This type of surgery has the advantages of precise operation, short operation time, minimal surgical trauma, short postoperative recovery time, and less pain for patients. It has gradually become one of the main means of treating cardiovascular diseases.

[0003] However, current PCI surgeries still face numerous challenges, including the following: 1) Traditional PCI procedures rely on the physician's experience, resulting in a high probability of error; 2) The learning curve is steep for less experienced physicians; 3) Physicians are exposed to radiation and must wear protective clothing, which affects their performance and harms the health of medical staff; and 4) Traditional medical imaging techniques are suboptimal, making it difficult for physicians to read and interpret images.

[0004] To improve the working environment and outcomes of interventional surgery, and with the rapid development of artificial intelligence (AI), engineers have developed cardiovascular interventional surgery robots to assist doctors in minimally invasive procedures. These robots can be remotely controlled in a safe, X-ray-free environment to perform cardiovascular interventional procedures. However, these robots can take a long time to perform, requiring doctors to maintain their focus and prone to fatigue, which can lead to errors.

[0005] CN 202010767888.7 discloses a deep learning-based automated vascular interventional robot surgery method. This method is based on the Alexnet deep learning network model and first collects terminal medical image information and doctor operation data during the vascular interventional surgery robot surgery. After processing the collected data, it is input into the deep learning network for network training. The trained network model can be applied to the automated surgery process of the vascular interventional surgery robot. However, this method does not specify the source of the dataset used and the type of medical image, and the method does not involve spatial registration-related operations or the posture adjustment of the interventional surgery robot.

[0006] With the continuous development of technology, the auxiliary role of intravascular ultrasound (IVUS) in PCI surgery is increasing day by day. The standard IVUS image is a 360-degree cross-sectional tomographic image of the intravascular structure. At present, in the field of medical image segmentation, the best performance is the U-Net deep learning network model. U-Net can use effective labeled data more effectively from very few training images by relying on data enhancement. Menghua Xia et al. proposed an algorithm that uses multi-scale feature aggregation U-Net to extract membrane boundaries in IVUS images. However, this method cannot be applied to the autonomous control of cardiovascular interventional robots. In addition, the system uses an existing public dataset, and its robustness is not enough to meet the individual differences of patients in actual surgery. Its algorithm and dataset need to be further improved to achieve the robustness required for autonomous control.

[0007] Furthermore, from a computer graphics perspective, the medical imaging data involved in this invention has a common characteristic: the sample size is generally small. When there are too few training samples, the training effect is likely to be poor. Therefore, we need to expand the dataset as much as possible to improve the effectiveness of model training.

[0008] Therefore, it is of great practical significance to develop a control method that can realize automatic control of cardiovascular interventional surgery robots with good accuracy. Summary of the Invention

[0009] Due to the above-mentioned defects in the prior art, the present invention provides a control method that can realize automatic control of a cardiovascular interventional surgical robot with good accuracy. This method can assist the automated surgical process of the cardiovascular interventional surgical robot, improve the doctor's operating experience, reduce the doctor's physical burden, and improve the surgical effect.

[0010] In order to achieve the above object, the present invention provides the following technical solutions:

[0011] A deep learning-based autonomous control method for a cardiovascular interventional robot. The cardiovascular interventional robot system includes an ultrasonic sensor, a posture sensor, and a catheter drive mounted on a cardiovascular interventional catheter, and a processing control unit connected to the ultrasonic sensor, posture sensor, and catheter drive. The processing control unit executes the following autonomous control method:

[0012] (1) The processing control unit obtains the ultrasound signal and posture signal currently obtained by the cardiovascular intervention catheter through the ultrasound sensor and posture sensor;

[0013] (2) The processing control unit preprocesses the ultrasound signal, inputs the preprocessed ultrasound signal into an image segmentation model based on deep learning, and then inputs the output of the image segmentation model into a binary classifier, which outputs the current state of the cardiovascular intervention catheter (the binary classifier classifies according to the position of the blood vessel wall in the image and estimates the position of the catheter in the blood vessel, so that the position can be judged based on the position to determine whether the posture is safe or not);

[0014] At the same time, the processing control unit performs kinematic calculations on the posture signal to obtain the current posture of the cardiovascular intervention catheter head, and then uses the motion model to calculate the current posture of the ultrasound sensor;

[0015] (3) the processing control unit determines a control scheme for the cardiovascular interventional catheter at a next moment according to the current state of the cardiovascular interventional catheter and the current position of the ultrasonic sensor obtained in step (2), and sends the control scheme for the cardiovascular interventional catheter at the next moment to the catheter driving device;

[0016] (4) The catheter driving device completes the action at the next moment according to the control scheme and returns to step (1) until the cardiovascular intervention catheter reaches the designated position.

[0017] The deep learning-based autonomous control method for cardiovascular interventional robots proposed in this paper further deepens and improves existing achievements. It combines the U-Net deep learning model with intravascular ultrasound technology and proposes the requirements for achieving the accuracy of autonomous control and the robustness of the system caused by individual differences in patients.

[0018] As the preferred technical solution:

[0019] The deep learning-based autonomous control method for a cardiovascular interventional robot as described above, wherein the current state of the cardiovascular interventional catheter is a safe posture or an unsafe posture;

[0020] In step (3), if the current state of the cardiovascular interventional catheter is a safe posture, the control plan of the cardiovascular interventional catheter at the next moment is that the cardiovascular interventional catheter continues to move forward. If the current state of the cardiovascular interventional catheter is an unsafe posture, the control plan of the cardiovascular interventional catheter at the next moment is that the cardiovascular interventional catheter returns to the state of the previous moment.

[0021] As described above, in a method for autonomous control of a cardiovascular interventional robot based on deep learning, the preprocessing steps are: after reading the image input frame, performing grayscale conversion operations (to reduce the amount of data and improve the computing speed and efficiency), Frost filtering operations, and Sobel operator filtering operations (to assist in subsequent edge detection).

[0022] As described above, a deep learning-based autonomous control method for a cardiovascular interventional robot randomly samples the preprocessed input frames after preprocessing, takes a portion (30%) of them for labeling and adds them to the training data set of the deep learning-based image segmentation model in real time. This can more effectively meet the individual differences of patients and improve the robustness of the system.

[0023] As described above, a method for autonomous control of a cardiovascular interventional robot based on deep learning, the image segmentation model based on deep learning is a U-Net deep learning network model, the input of which is the ultrasound image information within the blood vessel, and the output is a mask of the blood vessel contour in the ultrasound image within the blood vessel. The U-Net deep learning network model specifically includes an Encoder (contraction path) and a Decoder (expansion path), wherein the Encoder includes a convolution operation and a downsampling operation, and the Decoder consists of a convolution, upsampling, and skip-level structure. Specifically, the depth of U-Net can be 4 layers, the size of the convolution kernel can be 3×3, the loss function used is a weighted cross entropy loss function, and the Softmax function is used as the activation function. Of course, only one feasible construction scheme for the U-Net deep learning network model is given here, and those skilled in the art can reasonably adjust the model parameters according to actual conditions.

[0024] The autonomous control method of a cardiovascular intervention robot based on deep learning as described above, wherein the motion model is a PnP algorithm;

[0025] After solving the motion model, the relationship between the spatial point position and the pixel position can be established according to the camera model to obtain the current position of the ultrasonic sensor.

[0026] In the above-mentioned deep learning-based autonomous control method for a cardiovascular interventional robot, the cardiovascular interventional catheter, ultrasonic sensor, posture sensor, and catheter drive device are the lower computer part of the cardiovascular interventional robot system;

[0027] The processing control unit is the host computer part of the cardiovascular interventional robot system, which includes an image processing module (corresponding to image preprocessing, image segmentation based on deep learning, and catheter state judgment) and a kinematic solution module (corresponding to kinematic solution and motion model solution).

[0028] The ultrasonic sensor and posture sensor installed on the head of the cardiovascular interventional catheter obtain relevant information of the catheter head and transmit it to the upper computer for solution and processing. Through the combination of deep learning and posture information, the status information of the catheter in the blood vessel can be accurately obtained. This status information includes: the binary classification of the current position of the catheter (safe and unsafe), and whether the current posture of the catheter can continue to maintain operation; based on this, the lower computer controls the movement of the servo motor of the cardiovascular interventional surgical robot to achieve corresponding motion control, including: controlling the forward, backward and rotation of the interventional catheter.

[0029] The present invention further provides a computer device, comprising:

[0030] at least one processor; and,

[0031] a memory communicatively connected to the at least one processor; wherein,

[0032] The memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the deep learning-based autonomous control method of the cardiovascular intervention robot as described above is implemented.

[0033] In addition, the present invention also provides a computer-readable storage medium, which stores computer-readable instructions. When the computer-readable instructions are executed by a processor, the deep learning-based autonomous control method of the cardiovascular interventional robot as described above is implemented.

[0034] The above technical solution is only a feasible technical solution of the present invention. The protection scope of the present invention is not limited thereto. Those skilled in the art can reasonably adjust the specific design according to actual needs.

[0035] The above invention has the following advantages or beneficial effects:

[0036] (1) The deep learning-based autonomous control method for cardiovascular interventional robots of the present invention realizes autonomous control of cardiovascular interventional surgical robots, which can effectively reduce the burden on surgeons and effectively improve the effectiveness of cardiovascular interventional surgery.

[0037] (2) The deep learning-based autonomous control method for cardiovascular interventional robots of the present invention uses intravascular ultrasound as the source of intravascular image information, fundamentally avoiding the health hazards of X-ray angiography to both doctors and patients, and is more conducive to the integration of cardiovascular interventional surgical robot equipment;

[0038] (3) The deep learning-based autonomous control method for the cardiovascular interventional robot of the present invention uses a dataset derived from pre-calibrated datasets and real-time image information acquired during surgery, which improves the robustness of the system to individual differences among patients.

[0039] (4) The deep learning-based autonomous control method for cardiovascular interventional robots of the present invention combines the U-Net deep learning network based on image information with the kinematic information of the catheter head posture for prediction and judgment. The system can more accurately grasp the real-time information of the catheter part of the interventional surgical robot, including spatial position, posture, etc., and thus make more accurate judgments, which has great application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The present invention and its features, configurations, and advantages will become more apparent upon reading the detailed description of non-limiting embodiments with reference to the following drawings. Like reference numerals indicate like parts throughout the drawings. The drawings are not necessarily drawn to scale, emphasis being placed on illustrating the subject matter of the present invention.

[0041] Figure 1 Schematic diagram of the data flow of the deep learning-based autonomous control method for cardiovascular interventional robots of the present invention;

[0042] Figure 2 Schematic diagram of the image preprocessing steps of the present invention;

[0043] Figure 3 Schematic diagram of the system architecture and control principle of the deep learning-based autonomous control method for cardiovascular interventional robots of the present invention;

[0044] Figure 4 is a schematic diagram of the deep learning process of the present invention;

[0045] Figure 5 This is a schematic diagram of the structure of the computer device of Example 2. DETAILED DESCRIPTION

[0046] The structure of the present invention is further described below with reference to the accompanying drawings and specific embodiments, but is not intended to limit the present invention.

[0047] Example 1

[0048] A deep learning-based autonomous control method for cardiovascular interventional robot. Figure 3As shown, it includes an ultrasonic sensor, a posture sensor, and a catheter drive device installed on a cardiovascular intervention catheter, and a processing control unit connected to the ultrasonic sensor, posture sensor, and catheter drive device. The cardiovascular intervention catheter, ultrasonic sensor, posture sensor, and catheter drive device are the lower computer part of the cardiovascular intervention robot system; the processing control unit is the upper computer part of the cardiovascular intervention robot system, which includes an image processing module and a kinematic solution module;

[0049] The processing control unit operates as Figure 1 Autonomous control method shown:

[0050] (1) The processing control unit obtains the ultrasound signal and posture signal currently obtained by the cardiovascular intervention catheter through the ultrasound sensor and posture sensor;

[0051] (2) The processing control unit pre-processes the ultrasonic signal (step sequence as follows Figure 2 As shown, specifically, after reading the image input frame, grayscale conversion operation, Frost filtering operation and Sobel operator filtering operation are sequentially performed on it. After preprocessing, the preprocessed input frame is randomly sampled, 30% of which are marked and added to the training data set of the deep learning-based image segmentation model in real time). The preprocessed ultrasound signal is input into the deep learning-based image segmentation model (U-Net deep learning network model, whose input is the ultrasound image information in the blood vessel and the output is the mask of the blood vessel contour in the intravascular ultrasound image). The output of the image segmentation model is then input into a binary classifier, and the binary classifier outputs the current state of the cardiovascular interventional catheter, which is either a safe posture or an unsafe posture.

[0052] The formula of Frost filtering in the above preprocessing process is as follows:

[0053]

[0054] Where (x, y) is the coordinate of the pixel to be denoised, i and j represent the offset of (x, y) within a certain size window. m(x+i, y+j) is the weighted value of the pixel value, and its value decreases with increasing distance. Therefore, the estimated value of a pixel in the image is the weighted average of all pixel values within a certain window in the noisy image.

[0055] Among them, the calculation method of weight m(x+i,y+j) is:

[0056] m(x+i, y+j)=K2αexp[-α|t|];

[0057] in, K is a constant, and It is called the coefficient of variation of the window in the image domain, K2 is the normalization constant,

[0058] We can get the filtering formula of Frost filter:

[0059]

[0060] Where K1 is the filtering parameter and K is a normalization constant. We can adjust the filter's effect on the image by changing the value of K1 in the expression. The larger K1 is, the better the image smoothing effect is; conversely, the smaller the K1, the more edge information is preserved.

[0061] The Sobel operator used in the Sobel filter operation is of size 3×3, and the values of GX and GY are as follows:

[0062]

[0063] By performing a planar convolution with the image, we can obtain the approximate horizontal and vertical brightness difference values respectively;

[0064] The deep learning framework involved in the above U-Net deep learning network model is as follows Figure 4 As shown;

[0065] U-Net employs a network structure that combines downsampling and upsampling. The U-Net network can be simply thought of as first downsampling, then undergoing varying degrees of convolution to learn deep features, and then upsampling to restore the original image size. Upsampling is achieved through deconvolution. Downsampling gradually reveals environmental information, while upsampling combines information from each downsampled layer with the upsampled input to restore detailed information and gradually restore image accuracy.

[0066] The upsampling and downsampling stages of the U-Net network use the same number of convolution operations, and use a skip connection chain structure to connect the downsampling layer to the upsampling layer, so that the features extracted by the downsampling layer can be directly passed to the upsampling layer, which makes the pixel positioning of the U-Net network more accurate and the segmentation accuracy higher.

[0067] This embodiment involves building an image segmentation model based on the U-Net structure, including an encoder (contraction path) and a decoder (expansion path), wherein the encoder includes convolution operations and downsampling operations, and the decoder consists of convolution, upsampling and skip structures.

[0068] The depth of U-Net is 4 layers, the size of the convolution kernel is 3×3, the loss function used is the weighted cross entropy loss function, and the Softmax function is used as the activation function.

[0069] The softmax activation function nonlinearly superimposes the input features and weights for each pixel. The number of output values generated by each pixel after softmax processing is equal to the number of categories in the label. Softmax transforms the output value of each pixel into a probability distribution with positive values summing to 1, thereby obtaining the confidence level for each class at each pixel.

[0070] The acquired image enters a binary classifier, which classifies it according to the position of the blood vessel wall in the image, and estimates the position of the catheter in the blood vessel, so that the safety of the posture can be judged based on the position.

[0071] This embodiment uses a cross entropy loss function with boundary weights.

[0072]

[0073] p is the output value after softmax processing;

[0074] l:Ω→{1,...,K}, is the true label of each pixel;

[0075] pl(x)(x): the activation value of the output of the category given by the corresponding label at point x;

[0076] w:Ω→R is the weight added to each pixel during training.

[0077] The network is built using a deep learning platform. Network training includes data set preparation and parameter adjustment. The number of training iterations and batch size can be adjusted based on the performance of the training loss function, taking training speed into consideration. The learning rate should also be adjusted. Excessively high rates can cause the network to enter local optimization prematurely, resulting in unsatisfactory results. Adjustments can be made based on actual conditions.

[0078] The present invention adopts the existing relatively mature deep learning network for detection, eliminating the need for the design and implementation of complex algorithms, and can achieve the purpose more conveniently and directly.

[0079] At the same time, the processing control unit performs kinematic calculations on the posture signal to obtain the current posture of the cardiovascular intervention catheter head. After calculation using the motion model (PnP algorithm), the relationship between the spatial point position and the pixel position can be established based on the camera model to obtain the current posture of the ultrasound sensor.

[0080] When establishing the relationship between the spatial point position and the pixel position, consider n three-dimensional spatial points P and their projections p, and calculate the camera's posture R,t. Its Lie group is represented by T. Assume that the coordinates of a spatial point P are i =[X i , Y i , Z i ] TThe pixel coordinate of its projection is u i =[u i ,u i ] T , we can obtain the following formula:

[0081]

[0082] Through the three-dimensional coordinate transformation and posture estimation of the catheter head, the safety of the catheter can be judged. Based on the judgment results, the upper computer can control the lower computer, which controls the cardiovascular interventional surgery robot to drive the end servo motor to achieve corresponding forward, backward and rotation movements;

[0083] (3) the processing control unit determines a control scheme for the cardiovascular interventional catheter at a next moment based on the current state of the cardiovascular interventional catheter and the current posture of the ultrasonic sensor obtained in step (2), and sends the control scheme for the cardiovascular interventional catheter at a next moment to the catheter driving device. If the current state of the cardiovascular interventional catheter is that the posture is safe, the control scheme for the cardiovascular interventional catheter at a next moment is that the cardiovascular interventional catheter continues to move forward. If the current state of the cardiovascular interventional catheter is that the posture is unsafe, the control scheme for the cardiovascular interventional catheter at a next moment is that the cardiovascular interventional catheter returns to the state of the previous moment.

[0084] (4) The catheter driving device completes the action at the next moment according to the control scheme and returns to step (1) until the cardiovascular intervention catheter reaches the designated position.

[0085] Example 2

[0086] A computer device such as Figure 5 As shown, it includes at least one processor and a memory communicatively connected to the at least one processor;

[0087] The memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, it implements the deep learning-based autonomous control method of the cardiovascular intervention robot as described in Example 1.

[0088] Example 3

[0089] A computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the deep learning-based autonomous control method for a cardiovascular interventional robot as described in Example 1.

[0090] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0091] Those skilled in the art should understand that they can implement variations by combining the prior art with the above embodiments, which will not be described in detail here. Such variations do not affect the essence of the present invention and will not be described in detail here.

[0092] The above describes the preferred embodiments of the present invention. It should be understood that the present invention is not limited to the above-mentioned specific embodiments, and the devices and structures that are not described in detail should be understood to be implemented in a common manner in the art; any technician familiar with the art can use the above-mentioned disclosed methods and technical contents to make many possible changes and modifications to the technical solutions of the present invention without departing from the scope of the technical solutions of the present invention, or modify them into equivalent embodiments of equivalent changes, which does not affect the essential content of the present invention. Therefore, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention that do not depart from the content of the technical solutions of the present invention are still within the scope of protection of the technical solutions of the present invention.

Claims

1. A deep learning-based autonomous control method for a cardiovascular interventional robot. The cardiovascular interventional robot system includes an ultrasonic sensor, a posture sensor, and a catheter drive mounted on a cardiovascular interventional catheter, and a processing control unit connected to the ultrasonic sensor, posture sensor, and catheter drive. The method is characterized by: The processing control unit runs the following autonomous control method: (1) The processing control unit obtains the ultrasound signal and posture signal currently obtained by the cardiovascular intervention catheter through the ultrasound sensor and posture sensor; (2) The processing control unit preprocesses the ultrasonic signal, inputs the preprocessed ultrasonic signal into an image segmentation model based on deep learning, and then inputs the output of the image segmentation model into a binary classifier. The binary classifier outputs the current state of the cardiovascular intervention catheter. The binary classifier classifies the catheter according to the position of the blood vessel wall in the image, estimates the position of the catheter in the blood vessel, and judges whether the posture is safe or not based on the position. The current state of the cardiovascular intervention catheter is a safe posture or an unsafe posture; At the same time, the processing control unit performs kinematic calculations on the posture signal to obtain the current posture of the cardiovascular intervention catheter head, and then uses the motion model to calculate the current posture of the ultrasound sensor; (3) the processing control unit determines a control scheme for the cardiovascular interventional catheter at a next moment according to the current state of the cardiovascular interventional catheter and the current position of the ultrasonic sensor obtained in step (2), and sends the control scheme for the cardiovascular interventional catheter at the next moment to the catheter driving device; (4) The catheter driving device completes the action at the next moment according to the control scheme and returns to step (1) until the cardiovascular intervention catheter reaches the designated position.

2. The deep learning-based autonomous control method for a cardiovascular interventional robot according to claim 1, characterized in that: In step (3), if the current state of the cardiovascular interventional catheter is a safe posture, the control plan of the cardiovascular interventional catheter at the next moment is that the cardiovascular interventional catheter continues to move forward. If the current state of the cardiovascular interventional catheter is an unsafe posture, the control plan of the cardiovascular interventional catheter at the next moment is that the cardiovascular interventional catheter returns to the state of the previous moment.

3. The deep learning-based autonomous control method for a cardiovascular interventional robot according to claim 1, characterized in that: The pre-processing steps are: after reading the image input frame, performing grayscale conversion operation, frost filtering operation and sobel operator filtering operation on it in sequence.

4. The deep learning-based autonomous control method for a cardiovascular interventional robot according to claim 3, characterized in that: After preprocessing, the preprocessed input frames are randomly sampled, a portion of which is marked and added to the training dataset of the deep learning-based image segmentation model in real time.

5. The deep learning-based autonomous control method for a cardiovascular interventional robot according to claim 1, characterized in that: The image segmentation model based on deep learning is a U-Net deep learning network model, whose input is the ultrasonic image information in the blood vessel, and the output is the mask of the blood vessel contour in the intravascular ultrasonic image.

6. The deep learning-based autonomous control method for a cardiovascular interventional robot according to claim 1, characterized in that: The motion model is a PnP algorithm; After solving the motion model, the relationship between the spatial point position and the pixel position is established according to the camera model to obtain the current position of the ultrasonic sensor.

7. The deep learning-based autonomous control method for a cardiovascular interventional robot according to claim 1, characterized in that: The cardiovascular intervention catheter, ultrasonic sensor, posture sensor and catheter drive device are the lower computer part of the cardiovascular intervention robot system; The processing control unit is the host computer part of the cardiovascular intervention robot system, which includes an image processing module and a kinematic solution module.

8. A computer device, characterized in that: The computer device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, it implements the deep learning-based autonomous control method of the cardiovascular intervention robot according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the deep learning-based autonomous control method of the cardiovascular intervention robot according to any one of claims 1 to 7 is implemented.

Citation Information

Patent Citations

  • Vascular intervention robot automatic operation method based on deep learning

    CN111931626A

  • Catheter robot system for minimally invasive interventional operation in blood vessel

    CN102028549A

  • Arterial intervention implant implanting system capable of fusing real-time ultrasonic information based on magnetic navigation

    CN102319117A