Medical device control methods, systems and storage media

Abstract

This application relates to a medical device control method, system, and storage medium. The method includes: acquiring real-time medical images during surgery; registering the real-time medical images with a pre-generated standard medical model to obtain a first registration relationship; mapping a pre-generated first target path and first target blood vessel information to the real-time medical images according to the first registration relationship; determining control information for the medical device based on the position of the medical device in the real-time medical images, the second target blood vessel information mapped to the real-time medical images, and the second target path; and controlling the medical device based on the control information. This method can improve the degree of automation.

Description

Technical Field

[0001] This application relates to the field of medical device control technology, and in particular to a medical device control method, system and storage medium. Background Technology

[0002] With the development of minimally invasive technology, more and more surgeries are using minimally invasive methods to achieve their surgical goals. For example, in interventional minimally invasive surgery, it is necessary to deliver implants or replacements to the target location in the human body. Generally, a small hole is opened in the femoral artery or other arteries, and the implants or replacements are delivered to the target location in the human body through a delivery device.

[0003] To facilitate understanding, let's take heart valve replacement surgery as an example. Before the surgery, the surgeon analyzes and models the patient's CT angiography images to identify the type and location of vascular lesions. Based on this, the surgeon plans the surgical path and uses necessary consumables. During the surgery, the surgeon, wearing a lead apron under X-ray guidance, performs the procedure. Digital subtraction angiography is obtained using contrast agents. The surgeon relies on visual observation, experience, and tactile feedback to determine if the consumables are being pushed in the correct position and to proceed with the next steps in the surgical procedure.

[0004] However, the current method relies heavily on the doctor's personal experience and operational intuition, and has a very low degree of automation. Summary of the Invention

[0005] This application provides a medical device control method, system, and storage medium that can improve the level of automation.

[0006] In a first aspect, this application provides a medical device control method, the method comprising:

[0007] Acquire real-time medical images during surgery;

[0008] The real-time medical image is registered with the standard medical model generated before surgery to obtain the first registration relationship;

[0009] Based on the first registration relationship, the first target path and the first target blood vessel information generated before the operation are mapped to the real-time medical image;

[0010] Based on the position of the medical device in the real-time medical image, the second target blood vessel information mapped to the real-time medical image, and the second target path, the control information of the medical device is determined.

[0011] The medical device is controlled based on the control information.

[0012] In one embodiment, determining the control information of the medical device based on the position of the medical device in the real-time medical image, the second target blood vessel information mapped to the real-time medical image, and the second target path includes:

[0013] Based on the position of the medical device in the real-time medical image, the third target blood vessel information corresponding to the previous control cycle is selected from the second target blood vessel information mapped to the real-time medical image;

[0014] The control speed and movement distance of the medical device are determined based on the third target blood vessel information.

[0015] The target location of the medical device is determined based on the movement distance and the target path;

[0016] The control speed and the target position are used as the control information of the medical device.

[0017] In one embodiment, determining the control speed and movement distance of the medical device based on the third target blood vessel information includes:

[0018] Obtain the blood vessel information threshold corresponding to the velocity threshold;

[0019] The control speed of the medical device is determined based on the third target vascular information and the threshold value of the vascular information.

[0020] The movement distance of the medical device is determined based on the position of the medical device in the real-time medical image, the control speed, and the control cycle.

[0021] In one embodiment, before registering the real-time medical image with the preoperatively generated standard medical model to obtain a first registration relationship, the method further includes:

[0022] Acquire all preoperative medical images;

[0023] Semantic segmentation was performed on each of the aforementioned preoperative medical images to obtain segmentation results;

[0024] A standard medical model is obtained by spatial location fusion based on the segmentation results described above.

[0025] In one embodiment, the method further includes:

[0026] Regional vascular boundaries were extracted from the standard medical model.

[0027] The first target blood vessel information is determined by traversing the blood vessel boundary of the region. The first target blood vessel information includes at least one of the blood vessel centerline, blood vessel diameter, and blood vessel curvature.

[0028] In one embodiment, the method further includes:

[0029] Target detection is performed on the standard medical model to obtain target information;

[0030] Based on the target location in the target information, determine the initial path to the target location;

[0031] The first target path is obtained by filtering each of the initial paths based on the first target blood vessel information.

[0032] In one embodiment, the step of filtering each of the initial paths based on the first target blood vessel information to obtain the first target path includes:

[0033] Based on the first target blood vessel information, the path factor of each initial path is calculated, and the path factor includes path length, path diameter information and path curvature information.

[0034] Obtain the weight coefficients of each path factor;

[0035] Based on the path factors of each initial path and the corresponding weight coefficients, the initial paths are filtered to obtain the first target path.

[0036] In one embodiment, after filtering the initial paths based on the path factors of each initial path and the corresponding weight coefficients to obtain the first target path, the method further includes:

[0037] The medical device to be used for operation is determined based on the path length and diameter information of the first target path.

[0038] In one embodiment, registering the real-time medical image with a preoperatively generated standard medical model to obtain a first registration relationship includes:

[0039] Feature extraction is performed on the real-time medical image and the standard medical model to obtain the features to be processed;

[0040] The real-time medical image and the standard medical model are used to perform feature matching for the features to be processed.

[0041] The first registration relationship is obtained based on the feature matching results.

[0042] In one embodiment, the medical device is mounted on a delivery device; controlling the medical device based on the control information includes:

[0043] Obtain the second registration relationship between the push device and the image coordinate system corresponding to the real-time medical image;

[0044] Based on the second registration relationship, the control information of the medical device is converted into the control information of the push device;

[0045] The push device is controlled based on the control information of the push device.

[0046] In one embodiment, obtaining the second registration relationship between the push device and the image coordinate system corresponding to the real-time medical image includes:

[0047] At least two push positions are obtained on the calibration paper of the push device;

[0048] Obtain the positions to be processed in the image coordinate system corresponding to the at least two push positions in the real-time medical image;

[0049] A second registration relationship is generated based on the push location and the location to be processed.

[0050] In one embodiment, the method further includes:

[0051] Real-time detection of whether the medical device collides with the blood vessel;

[0052] When the medical device collides with the blood vessel, control the medical device to stop moving and / or output an alarm message;

[0053] When the medical device does not collide with the blood vessel, the system continues to monitor whether the medical device collides with the blood vessel in real time until the operation is completed.

[0054] In one embodiment, the real-time detection of whether the medical device collides with the blood vessel includes:

[0055] Based on the position of the medical device in the real-time medical image and the second target blood vessel information mapped to the real-time medical image, determine the interference between the medical device and the real-time medical image and / or obtain the force situation of the medical device;

[0056] The medical device is judged to have collided with the blood vessel based on the interference and / or the force conditions described.

[0057] In one embodiment, determining whether the medical device collides with the blood vessel based on the interference and / or the force conditions includes:

[0058] When the medical device interferes with the medical image, it is determined that the medical device has collided with the blood vessel;

[0059] When the force applied to the medical device exceeds the force threshold, it is determined that the medical device has collided with the blood vessel.

[0060] When the force on the medical device is less than or equal to the force threshold, and vascular damage is detected based on the real-time medical image, it is determined that the medical device has collided with the blood vessel.

[0061] When the force on the medical device is less than or equal to the force threshold, and the blood vessel is not damaged based on the real-time medical image, the system continues to detect whether the medical device interferes with the real-time medical image and / or detect whether the force on the medical device is greater than the force threshold, based on the position of the medical device in the real-time medical image and the second target blood vessel information mapped to the real-time medical image.

[0062] In one embodiment, the method further includes:

[0063] Receive mode switching command;

[0064] The current control mode is determined according to the mode switching command;

[0065] When the current control mode is automatic mode, the control information of the medical device is determined based on the position of the medical device in the real-time medical image, the second target blood vessel information mapped to the real-time medical image, and the second target path.

[0066] When the current control mode is manual control mode, the medical device control information sent by the main operating device is received.

[0067] In one embodiment, after controlling the medical device based on the control information, the process includes:

[0068] When the medical device reaches the target position corresponding to the target path, a release mode of the medical device is determined, and the medical device is released based on the release mode. In one embodiment, releasing the medical device based on the release mode includes:

[0069] When the medical device can be automatically released, the push device is controlled to release the medical device.

[0070] When the medical device cannot be automatically released, a manual release prompt message is output.

[0071] Secondly, this application also provides a medical device control system, the control system comprising:

[0072] Image acquisition equipment used to acquire real-time medical images;

[0073] Medical devices;

[0074] A pushing device, wherein the medical device is mounted on the pushing device, and the pushing device is used to push the medical device to a target location;

[0075] The processor is configured to execute the medical device control method described above based on the real-time medical image, so as to control the pushing device to push the medical device to the target location.

[0076] In one embodiment, the control system further includes:

[0077] A sensing device, wherein the sensing device is used to collect force information of the medical device and / or position information of the medical device;

[0078] The processor is also used to detect whether the medical device collides with the blood vessel based on the force information and / or position information of the medical device collected by the sensing device.

[0079] In one embodiment, the control system further includes:

[0080] The main operating device is used to control the pushing device to push the medical device to the target position when the control mode is manual control mode and / or to manually release the medical device when the medical device reaches the target position.

[0081] Thirdly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the methods in any of the above embodiments.

[0082] The aforementioned medical device control method, system, and storage medium involve registering a real-time medical image with a pre-operatively generated standard medical model to obtain a first registration relationship. Based on the first registration relationship, the pre-operatively generated first target path and first target blood vessel information are mapped onto the real-time medical image, thereby combining the pre-operative standard medical model with the intra-operative real-time medical image. Furthermore, based on the position of the medical device in the real-time medical image, the second target blood vessel information mapped onto the real-time medical image, and the second target path, the control information of the medical device is determined, automatically controlling the medical device without the need for doctor intervention, thus improving the degree of automation. Attached Figure Description

[0083] Figure 1 This is a schematic diagram of a medical device control system in one embodiment;

[0084] Figure 2 This is a schematic diagram of a push device in one embodiment;

[0085] Figure 3 This is a schematic diagram of the main operating device in one embodiment;

[0086] Figure 4 This is a schematic diagram of a medical device control system in one embodiment;

[0087] Figure 5 This is a schematic diagram of the modules of a medical device control system in one embodiment;

[0088] Figure 6 This is a flowchart illustrating a medical device control method in one embodiment;

[0089] Figure 7 A flowchart of the control information generation steps in one embodiment;

[0090] Figure 8 A flowchart of the standard medical model generation steps in one embodiment;

[0091] Figure 9 This is a schematic diagram of a standard medical model in one embodiment;

[0092] Figure 10 This is a schematic diagram of the blood vessel centerline extraction step in one embodiment;

[0093] Figure 11 This is a schematic diagram illustrating the extraction of the blood vessel centerline in one embodiment.

[0094] Figure 12 Here are some network architecture diagrams of a machine learning model in one embodiment;

[0095] Figure 13 A flowchart of the target path selection steps in one embodiment;

[0096] Figure 14 A flowchart of the first registration relationship generation step in one embodiment;

[0097] Figure 15 This is a schematic diagram of the steps for generating the second registration relationship in one embodiment;

[0098] Figure 16 This is a schematic diagram of the security protection process in one embodiment;

[0099] Figure 17 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0100] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0101] See Figure 1 As shown, Figure 1 This is a schematic diagram of a medical device 200 control system in one embodiment. The medical device 200 control system includes an image acquisition device 100, a medical device 200, a push device 400, and a processor 300.

[0102] The medical device 200 is installed on the pushing device 400, which can push the medical device 200 to the target location. The image acquisition device 100 is used to acquire real-time medical images and send the acquired real-time medical images to the processor 300. The processor 300 can then generate control information for the pushing device 400 based on the real-time medical images and control the pushing device 400 to push the medical device 200 to the target location based on the control information, thereby achieving automatic control.

[0103] Combination Figure 2 As shown, Figure 2 This is a schematic diagram of a pushing device 400 in one embodiment. The pushing device 400 includes a linear motion mechanism, a rotation mechanism 401, and a clamping mechanism, wherein the clamping mechanism may include a friction wheel 402, such as... Figure 2 The device includes two pairs of friction wheels 402, which are used to clamp and push the medical device 200. A rotating mechanism 401 is used to reciprocate the clamping of the medical device 200. During automatic control, the pushing device 400 is controlled by the processor 300 to control the movement of the medical device 200, including linear and rotary motion. When switched to manual mode, the pushing device 400 is controlled by the main operating device 500, which sends control information to the processor 300. The processor 300 then controls the pushing device 400 based on the control information sent by the main operating device 500 to push the medical device 200 to the target position.

[0104] Combination Figure 3 As shown, Figure 3 This is a schematic diagram of the main operating device 500 in one embodiment. In this embodiment, the main operating device 500 includes a moving pushing unit 502 and a rotating unit 501 corresponding to the pushing device 400, that is, it has the same two degrees of freedom as the pushing device 400, thereby controlling the linear movement and rotation of the medical device 200. During operation, the doctor observes the information displayed on the monitor 600 to determine whether doctor intervention is required. If intervention is required, the doctor can switch to manual mode through the main operating device 500 to control the pushing device 400.

[0105] The medical device 200 can be a medical consumable that needs to be implanted in the human body during interventional surgery. The pushing device 400 controls the catheter to push the medical device 200 to the target location. For example, in heart valve transplantation surgery, the pushing device 400 can control the catheter to deliver the artificial heart valve to the target part of the heart.

[0106] Optionally, sensors can be installed at the medical device 200 or at the end of the catheter to collect force information and / or position information of the medical device 200. For example, force sensors can be used to collect force information, and electromagnetic sensors can be used to collect position information. The collected force information and / or position information of the medical device 200 are then sent to the processor 300. The processor 300 can then detect whether the medical device 200 has collided with a blood vessel based on the force information and / or the position information of the medical device 200, thereby ensuring operational safety.

[0107] For ease of understanding, combined with Figure 4 As shown, Figure 4 This is a schematic diagram of a scenario for the control system of the medical device 200 in one embodiment. This scenario could be a diagram of a vascular interventional surgery, where a doctor monitors the surgical process outside the operating room, for example, by observing it on a monitor, and switches the surgical control mode (from automatic control mode to manual control mode) via the main operating device 500. The operating room includes an image acquisition device 100, which can acquire real-time medical images and send them to a processor 300. The processor 300 processes these images to obtain control information from the push module, thereby controlling the medical device 200 based on this control information.

[0108] One point to note is that the processor 300 can be a separate controller, such as the controller of the push device 400. In other embodiments, the processor 300 may include the controller of the push device 400 and the image processing unit in the image acquisition device 100, for example, in combination. Figure 5As shown, the image acquisition device 100 includes an image acquisition unit, an image storage unit, and an image processing unit. This image acquisition device 100 can be an automatic navigation device. The image acquisition unit of the automatic navigation device is used to acquire real-time medical images, the image storage unit is used to store the acquired real-time medical images, and the image processing unit is used to obtain real-time medical images from the image storage unit and process them to obtain control information, i.e., generate automatic navigation information. This control information is then sent to the controller of the pushing device 400. The controller, based on this control information, controls the pushing device 400 to push the medical device 200 to the target location. The main operating device 500 is controlled by the doctor. When it is determined to take over the surgery, it switches from automatic control mode to manual control mode and sends the control information corresponding to the doctor's operation to the controller. The controller, based on this control information, controls the pushing device 400 to push the medical device 200 to the target location.

[0109] The sensors at the medical device 200 can collect force information and feed it back to the controller, enabling the controller to perform safety control and also to feed the force information back to the main operating device 500 to provide guidance to the doctor. The sensors can also collect the position information of the medical device 200 and send it to the image processing unit, so that the image processing unit can generate control information for the pushing device 400 in the next cycle based on this position information.

[0110] The image processing unit is also used to process the preoperative medical images acquired before the operation to generate a standard medical model, and to extract the first target blood vessel information, generate the first target path, and select the medical device 200 from the standard medical model. The standard medical model, the first target blood vessel information, the generated first target path, and the selected medical device 200 are then stored in the image storage unit so that the real-time medical images can be registered with the standard medical model during the operation. The control information of the push device 400 can be generated based on the registered target path, etc., as detailed below.

[0111] In one embodiment, such as Figure 6 As shown, a medical device control method is provided, which is applied to... Figure 1 Taking the processor in the example, the following steps are included:

[0112] S602: Acquire real-time medical images during surgery.

[0113] Specifically, real-time medical images are acquired during surgery, such as medical images captured by an image acquisition device during the operation, which can be directly focused on the area being operated on. This image acquisition device can be a medical imaging device, such as a CT scanner or other X-ray imaging equipment.

[0114] S604: Register the real-time medical image with the standard medical model generated before surgery to obtain the first registration relationship.

[0115] S606: Based on the first registration relationship, map the first target path and the first target blood vessel information generated before the operation to the real-time medical image.

[0116] The standard medical model is generated preoperatively. For example, preoperative medical images are acquired using CT angiography equipment, and these images are reconstructed to obtain the standard medical model. This model may include information about a first target blood vessel and a first target path. In other words, the standard medical model is obtained by reconstructing the preoperative medical images, and then surgical planning is performed on this model. For example, the first target location, such as the lesion location, is identified on the standard medical model. Surgical planning is then performed based on this first target location and the first target blood vessel information to obtain the first target path. Therefore, both the first target blood vessel information and the first target path have a positional relationship with the standard medical model.

[0117] The processor can register real-time medical images with preoperatively generated standard medical models using feature-based registration. For example, it can identify features of tissues, etc., in both the real-time medical image and the standard medical model, and then register them using these features to obtain a first registration relationship. For instance, it can identify at least one feature of the spine, blood vessels, or organs in both the real-time medical image and the standard medical model, and then match these features to achieve registration. It's important to note that the identification of these features in the standard medical model can be done preoperatively, so that during surgery, only the real-time medical image needs to be identified, improving processing efficiency and reducing the computational load. In other embodiments, the standard medical model and the real-time medical image can be processed in parallel during surgery. However, it should be noted that the processing of the standard medical model only needs to be done once, and the data should be stored locally after identification, thus reducing subsequent redundant calculations.

[0118] Specifically, since the first target path and the first target blood vessel information have a positional relationship with the standard medical model, and the standard medical model has a first registration relationship with the real-time medical image, the first target path and the first target blood vessel information can be mapped onto the real-time medical image based on the first registration relationship to obtain the second target path and the second target blood vessel information. This allows the real-time medical image to be displayed to the doctor for review. Optionally, in other embodiments, the position of the medical device in the real-time medical image can also be mapped onto the standard medical model, allowing the standard medical model to be displayed to the doctor for review.

[0119] The first and second target blood vessel information may include at least one of the vessel's diameter, length, and curvature. It should be noted that the distinction between "first" and "second" here is only formal; the first target blood vessel information and the first target path are generated in a standard medical model, i.e., preoperatively. The second target blood vessel information and the second target path are generated by mapping the preoperative first target blood vessel information and the first target path onto the real-time medical image, i.e., displayed in the real-time medical image.

[0120] S608: Based on the position of the medical device in the real-time medical image, the second target blood vessel information mapped to the real-time medical image, and the second target path, determine the control information of the medical device.

[0121] Specifically, in the above steps, the first target blood vessel information and the first target path are mapped onto the real-time medical image. Thus, the next position of the medical device can be determined based on the second target blood vessel information and the second target path mapped onto the real-time medical image. Then, the motion information of the medical device is generated according to the position of the medical device and the next position of the medical device. Based on the motion information, the control information of the medical device can be determined.

[0122] The motion information of the medical device may include speed and position, which are not specifically limited here. The speed can be generated based on the information of the second target blood vessel, and the position can be generated based on the second target path and control cycle.

[0123] S610: Controlling medical devices based on control information.

[0124] Specifically, the processor can control the movement of the medical device based on control information, such as by controlling a pushing device. Therefore, in an optional embodiment, the processor can determine the control information of the pushing device based on the control information of the medical device, and then control the movement of the medical device by controlling the pushing device.

[0125] The aforementioned medical device control method registers the intraoperative real-time medical images with the preoperative standard medical model to obtain a first registration relationship. Based on the first registration relationship, the first target path and the first target blood vessel information are mapped onto the real-time medical images, thereby combining the preoperative standard medical model with the intraoperative real-time medical images. Furthermore, based on the position of the medical device in the real-time medical images, the second target blood vessel information mapped onto the real-time medical images, and the second target path, the control information of the medical device is determined, and the medical device is automatically controlled without the need for doctor intervention, thus improving the degree of automation.

[0126] In one embodiment, the control information of the medical device is determined based on the position of the medical device in the real-time medical image, the second target blood vessel information mapped to the real-time medical image, and the second target path. This includes: selecting third target blood vessel information corresponding to the previous control cycle from the second target blood vessel information mapped to the real-time medical image based on the position of the medical device in the real-time medical image; determining the control speed and movement distance of the medical device based on the third target blood vessel information; determining the target position of the medical device based on the movement distance and the target path; and using the control speed and target position as the control information of the medical device.

[0127] Specifically, the position of the medical device in the real-time medical image can be obtained through sensors or determined by target recognition of the real-time medical image, which will not be elaborated here. This allows for the acquisition of second target blood vessel information within a certain range based on the position of the medical device, which can then be used as third target blood vessel information. The speed of the medical device can then be determined based on this third target blood vessel information. The third target blood vessel information within this range can refer to the blood vessel information from the previous control cycle, specifically the third target blood vessel information of the blood vessels traversed by the medical device during the previous control cycle.

[0128] Specifically, determining the control speed and movement distance of the medical device based on the third target vascular information may include: acquiring a vascular information threshold corresponding to a speed threshold; determining the control speed of the medical device based on the third target vascular information and the vascular information threshold; and determining the movement distance of the medical device based on the position, control speed, and control cycle of the medical device in a real-time medical image.

[0129] Specifically, the speed threshold is the maximum push speed set, which is usually set by the doctor based on the location of the blood vessel or is the default setting of the push device. The control cycle can be the cycle of motion, or it can be preset by the doctor, such as 1 second or 5 seconds, etc., without specific limitations here.

[0130] Specifically, in combination Figure 7 As shown, Figure 7 This is a flowchart of the control information generation steps in one embodiment. In this embodiment, since the target path is predetermined, the third target blood vessel information corresponding to the previous control cycle is first obtained. The third target blood vessel information may include the diameter and curvature of the blood vessel corresponding to the previous control cycle. The third target blood vessel information may also include mathematical statistical values ​​of the diameter and curvature of the blood vessel corresponding to the previous control cycle, such as the average and maximum values, etc., which will not be elaborated here.

[0131] The control speed can be calculated using the following formula:

[0132]

[0133] Where Vmax is the set maximum push speed, i.e. the speed threshold, d(n-1) is the diameter of the blood vessel in the previous control cycle, dmax is the set blood vessel diameter that can meet the maximum push speed, Cmin is the set blood vessel curvature (detour degree) that can meet the maximum push speed, and c(n-1) is the curvature of the blood vessel in the previous control cycle.

[0134] The control position can be calculated using the following formula:

[0135] P(n)=P(n-1)+V(n)×dt

[0136] Where P(n-1) is the medical device push position in the previous control cycle, dt is the control cycle, and P(n) is the medical device push position in the current control cycle.

[0137] The narrower the diameter and the greater the curvature of the blood vessel, the slower the medical device is pushed. The calculated push position is sent to the processor, which controls the pushing device to push the medical device according to the calculated speed and position until it reaches the lesion. If the medical device can be released automatically, the pushing device will release it automatically; if it cannot be released automatically, the doctor will be alerted to take over the surgery and manually release the medical device.

[0138] In the above embodiments, the control information for this control cycle is generated based on the third target blood vessel information and the target path from the previous control cycle, thereby controlling the medical device. The pushing distance of the medical device in each control cycle is calculated in real time according to the target path and the lesion location, and the pushing device pushes automatically.

[0139] In one embodiment, before registering the real-time medical images with the preoperatively generated standard medical model to obtain the first registration relationship, the method further includes: acquiring each preoperative medical image; performing semantic segmentation on each preoperative medical image to obtain the segmentation result; and performing spatial location fusion based on each segmentation result to obtain the standard medical model.

[0140] Specifically, this embodiment mainly explains the generation process of the standard medical model, wherein, in combination with Figure 8 As shown, when there are CT image data with multiple FOVs (fields of view), i.e., multiple preoperative medical images, a specific semantic segmentation task is performed on the CT image data of each FOV. For example, a corresponding trained deep learning semantic segmentation algorithm is used to generate segmentation results. After obtaining the segmentation results for multiple FOVs, the images are fused and 3D reconstructed based on their spatial positions to finally obtain a standard medical model. Specifically, combined with Figure 9 As shown, Figure 9This is a schematic diagram of a standard medical model in one embodiment, in which a standard medical model including blood vessels and various tissues and organs is obtained by segmentation.

[0141] In the above embodiments, preoperative medical images are segmented using machine learning algorithms, and blood vessels are reconstructed in three dimensions to obtain a standard medical model, laying the foundation for subsequent operational control.

[0142] In one embodiment, the method further includes: extracting regional vascular boundaries from a standard medical model; traversing the regional vascular boundaries to determine first target vascular information, the first target vascular information including at least one of vascular centerline, vascular diameter, and vascular curvature.

[0143] Specifically, the first target blood vessel information includes at least one of the following: blood vessel centerline, blood vessel diameter, and blood vessel curvature. The blood vessel centerline refers to the midline of the blood vessel, which can be obtained through the boundary of the blood vessel. The blood vessel diameter can also be obtained through the boundary of the blood vessel, i.e., the distance from the blood vessel boundary. The blood vessel curvature is obtained by calculating the angle between the tangent vector of the blood vessel boundary and the blood vessel centerline.

[0144] Combination Figure 10 As shown, Figure 10 This is a schematic diagram of the steps for extracting the vascular centerline in one embodiment. In this embodiment, a standard medical model is obtained, or the segmentation results are obtained by semantic segmentation of various preoperative medical images. Based on the segmentation results, the vascular boundaries of the traversed regions are extracted. For example, the vascular boundaries are traversed in a certain order, and the Euclidean distance of the vascular boundary in each traversed region is calculated. The Euclidean distance of the current boundary is calculated to obtain the maximum inscribed sphere of the boundary, thereby obtaining the center of the inscribed sphere as the centerline. After all traversed regions are completed, the centers of all the obtained inscribed spheres are connected to form a line to obtain the vascular centerline. Specifically, combined with Figure 11 As shown, Figure 11 This is a schematic diagram illustrating the extraction of the vessel centerline in one embodiment, where the vessel centerline is obtained by connecting the centers of all the inscribed spheres. Optionally, the diameter of the inscribed sphere is the diameter of the vessel in the traversed region, and the angle between the tangent of the inscribed sphere and the centerline is the vessel curvature.

[0145] In the above embodiments, vascular information is automatically identified, laying the foundation for subsequent control.

[0146] In one embodiment, the method further includes: performing target detection on a standard medical model to obtain first target blood vessel information; determining an initial path to the first target location based on the first target location in the first target blood vessel information; and filtering each initial path based on the first target blood vessel information to obtain a first target path.

[0147] Specifically, the target location can refer to the location of the lesion. Target information can be obtained by performing target detection on a standard medical model, or by obtaining the segmentation results from semantic segmentation of various preoperative medical images, and then obtaining the target information based on the segmentation results.

[0148] The detection of target information can be performed through machine learning models, which can obtain the location of the target, as well as the type and severity of the lesion. In practical applications, machine learning target detection models are used to match and evaluate different lesion types, obtain the type and severity of vascular lesions, and finally mark the location of the lesion on the center line of the blood vessel.

[0149] Combination Figure 12 As shown, Figure 12 This diagram illustrates several network architectures for a machine learning model in one embodiment, including FAST R-CNN, SSD, and YOLO. Other network architectures may be used in other embodiments, such as... Figure 9 The input image segmentation results are used by the model, which extracts and segments the data through a multi-layer deep learning network. This allows for automatic identification and labeling of lesion types and locations within the segmentation results. The machine learning object detection model is trained on a large number of expert-annotated lesion types to ensure accuracy.

[0150] After determining the first target information, the initial path to the target location can be determined based on the target location, the surgical entry point, and the branches of the blood vessels. By filtering the initial path, the first target path can be obtained.

[0151] Optionally, the first target path is obtained by filtering each initial path based on the first target blood vessel information, including: calculating the path factor of each initial path based on the first target blood vessel information, the path factor including path length, path diameter information and path curvature information; obtaining the weight coefficient of each path factor; and filtering the initial paths based on the path factor of each initial path and the corresponding weight coefficient to obtain the first target path.

[0152] The weight coefficients of each path factor can be preset, such as by the doctor based on the location of the blood vessels, or by default; no specific restrictions are made here.

[0153] The processor iterates through all possible initial paths, extracting geometric information such as the length, diameter, and curvature of each initial path, and calculates the access evaluation value for each initial path based on this geometric information. Finally, the initial path with the smallest access evaluation value is selected as the first target path. The selection of the first target path prioritizes shorter access length, larger vessel diameter, and smaller vessel curvature. The calculation method for the access evaluation value is not limited to the method shown in this embodiment.

[0154] Specifically, in combination Figure 13 As shown, the approach evaluation value can be calculated using the following formula:

[0155] EP=a×L+b×dmin+c×dave+d×cmax+e×cave

[0156] Where a, b, c, d, and e are weighting coefficients, L is the length of the initial path, dmin is the minimum diameter of the blood vessel, dave is the average diameter of the blood vessel, cmax is the maximum curvature of the blood vessel, and cave is the average curvature of the blood vessel.

[0157] The processor selects the initial path with the smallest ingress evaluation value as the first target path.

[0158] Optionally, after filtering the initial paths based on the path factors and corresponding weight coefficients of each initial path to obtain the first target path, the method further includes: determining the medical device to be operated based on the path length and diameter information of the first target path.

[0159] Specifically, after the first target path is determined, the processor determines the medical device to be operated based on the path length and diameter information of the first target path. For example, the length of the catheter is determined based on the path length of the first target path, and the diameter of the catheter and the diameter of the medical device are determined based on the minimum diameter of the first target path.

[0160] In the above embodiments, different initial paths are analyzed based on the location of the lesion, the optimal path is automatically planned, and the corresponding medical device is selected.

[0161] In one embodiment, registering a real-time medical image with a preoperatively generated standard medical model to obtain a first registration relationship includes: extracting features from the real-time medical image and the standard medical model to obtain features to be processed; performing feature matching between the real-time medical image and the standard medical model and the features to be processed; and obtaining the first registration relationship based on the feature matching result.

[0162] Specifically, the features to be processed can refer to at least one feature from the spine, blood vessels, or organs. Registration is achieved by matching these features. It's important to note that the identification of these features in the standard medical model can be done preoperatively, so that during surgery, only real-time identification of the real-time medical image is needed, improving processing efficiency and reducing the computational load during surgery. In other embodiments, the standard medical model and the real-time medical image can be processed separately in parallel during surgery. However, it should be noted that the processing of the standard medical model only needs to be done once, and the identified features are stored locally, thus reducing subsequent redundant calculations.

[0163] Combination Figure 14 As shown, Figure 14 The flowchart shows the first registration relationship generation step in one embodiment. In this embodiment, the processor performs feature recognition on the standard medical model and the real-time medical image, such as recognizing features of at least one of the spine, blood vessels, or organs. Then, these features are matched, and after matching, a linear matrix transformation is performed on the real-time medical image to achieve the fusion of the image and the three-dimensional model.

[0164] In the above embodiments, intraoperative two-dimensional X-ray images are registered and fused with preoperative standard medical models for display, thereby applying the preoperative images and models to the surgical process. The push position of medical devices is automatically calculated, and the push device pushes automatically without the need for doctor intervention, reducing radiation damage to doctors and shortening the operation time.

[0165] In one embodiment, a medical device is installed on a push device; controlling the medical device based on control information includes: obtaining a second registration relationship between the image coordinate system corresponding to the push device and the real-time medical image; converting the control information of the medical device into control information of the push device based on the second registration relationship; and controlling the push device based on the control information of the push device.

[0166] Optionally, obtaining the second registration relationship between the push device and the image coordinate system corresponding to the real-time medical image includes: obtaining at least two push positions on the calibration paper of the push device; obtaining the positions to be processed corresponding to the at least two push positions in the image coordinate system corresponding to the real-time medical image; and generating the second registration relationship based on the push positions and the positions to be processed.

[0167] Specifically, in combination Figure 15 As shown, Figure 15 This is a schematic diagram of the steps for generating the second registration relationship in one embodiment. The calibration paper is fixed to the pushing device, and the coordinates (X, Y, F, G) of the calibration paper at two positions on the pushing device are obtained. 1B X 2B ) and image coordinates (X 1C X 2C After obtaining the coordinates, the transformation relationship H between the image coordinate system and the push device coordinate system can be obtained according to the coordinate system transformation relationship. BC This establishes the coordinate relationship between the image coordinate system corresponding to the real-time medical image and the push device.

[0168] In this way, the processor can base its operation on the control information of the medical device and the conversion relationship H. BC The control information of the push device is determined, and then the movement of the medical device is controlled by controlling the push device.

[0169] In the above embodiments, real-time medical images are registered and fused with preoperative 3D models for display. Based on the second registration relationship between the coordinate system of the real-time medical images and the coordinate system of the push device, the control information for the medical device is converted into control information for the push device, thereby enabling control of the medical device through the push device.

[0170] In one embodiment, the method further includes: real-time detection of whether the medical device collides with the blood vessel; when the medical device collides with the blood vessel, controlling the medical device to stop moving and / or outputting alarm information; when the medical device does not collide with the blood vessel, continuing to detect whether the medical device collides with the blood vessel in real time until the operation ends.

[0171] Specifically, collisions between medical devices and blood vessels can be determined using real-time medical images and / or data collected by sensors. For example, real-time medical images can be used to determine whether the medical device is in contact with the blood vessel, whether the blood vessel is damaged, and data collected by sensors can be used to determine whether the force applied to the medical device meets the requirements.

[0172] Optionally, real-time detection of whether a medical device collides with a blood vessel includes: determining the interference between the medical device and the real-time medical image and / or obtaining the force on the medical device based on the position of the medical device in the real-time medical image and the target blood vessel information mapped to the real-time medical image; and determining whether the medical device collides with the blood vessel based on the interference and / or force. Specifically, determining whether the medical device collides with the blood vessel based on interference and / or force conditions may include: detecting whether the medical device interferes with the real-time medical image and / or detecting whether the force on the medical device exceeds a force threshold; when the medical device interferes with the medical image, determining that the medical device has collided with the blood vessel; when the force on the medical device exceeds the force threshold, determining that the medical device has collided with the blood vessel; when the force on the medical device is less than or equal to the force threshold, and blood vessel damage is detected based on the real-time medical image, determining that the medical device has collided with the blood vessel; when the force on the medical device is less than or equal to the force threshold, and blood vessel damage is detected based on the real-time medical image, continuing to detect whether the medical device interferes with the real-time medical image and / or detect whether the force on the medical device exceeds the force threshold based on the position of the medical device in the real-time medical image and the target blood vessel information mapped to the real-time medical image.

[0173] Specifically, in combination Figure 16 As shown, Figure 16This is a schematic diagram of the safety protection process in one embodiment. In this embodiment, dual protection is provided by real-time medical images and force sensors. First, the processor detects whether the medical device interferes with the blood vessel based on the real-time medical images. If interference occurs, i.e., the medical device comes into contact with the blood vessel, an alarm message is generated and output to alert the doctor. If the medical device does not interfere with the blood vessel, the processor continues to detect the force on the medical device. When the force on the medical device exceeds the force threshold, the operation is terminated. Otherwise, the processor continues to detect whether the blood vessel is damaged based on the real-time medical images. If damage occurs, the operation is terminated. Otherwise, the process of detecting whether the medical device interferes with the blood vessel based on the real-time medical images continues until the surgery is completed.

[0174] In the above embodiments, by analyzing real-time medical images and combining them with force sensor feedback values, the system detects whether medical devices interfere with or collide with the blood vessel wall, thus ensuring surgical safety.

[0175] In one embodiment, the method further includes: receiving a mode switching instruction; determining the current control mode according to the mode switching instruction; when the current control mode is automatic mode, continuing to determine the control information of the medical device based on the position of the medical device in the real-time medical image, the second target blood vessel information mapped to the real-time medical image, and the second target path; when the current control mode is manual control mode, receiving the control information of the medical device sent by the master operating device.

[0176] In one embodiment, after controlling the medical device based on the control information, the process includes: when the medical device reaches the target position corresponding to the target path, determining the release mode of the medical device, and releasing the medical device based on the release mode. Specifically, it can detect whether the medical device can be released automatically; when the medical device can be released automatically, controlling the pushing device to release the medical device; when the medical device cannot be released automatically, outputting a manual release prompt message.

[0177] Specifically, this embodiment mainly introduces the control modes, which include automatic control mode and manual control mode. During operation, it can be set to automatic control mode, allowing control of the medical device and its transport to the corresponding target location. The doctor can observe the transport of the medical device in automatic control mode. When the doctor determines that intervention is necessary, the control mode is switched to manual control mode. In this mode, the processor receives control information from the main operating device, allowing for timely intervention by the doctor to ensure the safety of the surgical procedure.

[0178] In addition, if the doctor does not intervene during the entire transportation process and the medical device is transported to the target location, it is necessary to assess whether surgical consumables such as balloons and valves can be released automatically. If they can be released automatically, they will be released automatically; otherwise, a prompt message will be output so that the doctor can intervene in a timely manner and release the medical device accurately.

[0179] In the above embodiments, doctors can intervene in and take over the automated surgery at any time through the main operating device, further ensuring the safety of the surgery.

[0180] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0181] Based on the same inventive concept, this application also provides a medical device control apparatus for implementing the medical device control method described above. The solution provided by this apparatus is similar to the solution described in the above method; therefore, the specific limitations in one or more embodiments of the medical device control apparatus provided below can be found in the limitations of the medical device control method described above, and will not be repeated here.

[0182] In one embodiment, a medical device control device is provided, comprising:

[0183] Intraoperative real-time medical image acquisition module, used to acquire real-time medical images during surgery;

[0184] The first registration module is used to register the real-time medical images during the operation with the standard medical model generated before the operation to obtain the first registration relationship;

[0185] The mapping module is used to map the first target path and the first target blood vessel information generated before surgery to the real-time medical image according to the first registration relationship;

[0186] The control information generation module is used to determine the control information of the medical device based on the position of the medical device in the real-time medical image, the second target blood vessel information mapped to the real-time medical image, and the second target path.

[0187] The control module is used to control medical devices based on control information.

[0188] In one embodiment, the control information generation module is further configured to, based on the position of the medical device in the real-time medical image, select the third target blood vessel information corresponding to the previous control cycle from the target blood vessel information mapped to the real-time medical image; determine the control speed and movement distance of the medical device based on the third target blood vessel information; determine the target position of the medical device based on the movement distance and the target path; and use the control speed and target position as the control information of the medical device.

[0189] In one embodiment, the above-mentioned device further includes a standard medical model generation module, which is used to acquire the collected preoperative medical images; perform semantic segmentation on the preoperative medical images to obtain segmentation results; and perform spatial location fusion based on the segmentation results to obtain a standard medical model.

[0190] In one embodiment, the above-mentioned device further includes a feature extraction module for extracting regional blood vessel boundaries from a standard medical model; traversing the regional blood vessel boundaries to determine first target blood vessel information, the first target blood vessel information including at least one of blood vessel centerline, blood vessel diameter, and blood vessel curvature.

[0191] In one embodiment, the above-mentioned device further includes a target path generation module, which is used to perform target detection on a standard medical model to obtain target information; determine an initial path to the target location based on the target location in the target information; and filter each initial path based on the first target blood vessel information to obtain a first target path.

[0192] In one embodiment, the target path generation module is further configured to calculate the path factors of each initial path based on the first target blood vessel information, the path factors including path length, path diameter information and path curvature information; obtain the weight coefficient of each path factor; and filter the initial paths based on the path factors of each initial path and the corresponding weight coefficients to obtain the first target path.

[0193] In one embodiment, the target path generation module is further configured to determine the medical device to be operated based on the path length and diameter information of the first target path.

[0194] In one embodiment, the above-mentioned apparatus further includes a first registration relationship generation module, which is used to extract features from real-time medical images and standard medical models to obtain features to be processed; to perform feature matching between real-time medical images and standard medical models and the features to be processed; and to obtain a first registration relationship based on the feature matching result.

[0195] In one embodiment, the medical device is mounted on a push device; the control module is further configured to obtain a second registration relationship between the image coordinate system of the push device and the real-time medical image; convert the control information of the medical device into the control information of the push device based on the second registration relationship; and control the push device based on the control information of the push device.

[0196] In one embodiment, the registration module is further configured to acquire at least two push positions on the calibration paper of the push device; acquire the positions to be processed corresponding to the at least two push positions in the image coordinate system corresponding to the real-time medical image; and generate a second registration relationship based on the push positions and the positions to be processed.

[0197] In one embodiment, the device further includes a safety detection module, which is used to detect in real time whether the medical device collides with the blood vessel; when the medical device collides with the blood vessel, it controls the medical device to stop moving and / or outputs an alarm message; when the medical device does not collide with the blood vessel, it continues to detect in real time whether the medical device collides with the blood vessel until the operation ends.

[0198] In one embodiment, the safety detection module is further configured to determine the interference between the medical device and the real-time medical image and / or obtain the force condition of the medical device based on the position of the medical device in the real-time medical image and the target blood vessel information mapped to the real-time medical image; and to determine whether the medical device collides with the blood vessel based on the interference and / or force condition.

[0199] In one embodiment, the device further includes a mode switching module, which is used to receive a mode switching instruction; determine the current control mode according to the mode switching instruction; when the current control mode is automatic mode, continue to determine the control information of the medical device based on the position of the medical device in the real-time medical image, the second target blood vessel information mapped to the real-time medical image, and the second target path; when the current control mode is manual control mode, receive the control information of the medical device sent by the main operating device.

[0200] In one embodiment, the device further includes a release module for determining the release mode of the medical device when the medical device reaches the target position corresponding to the target path, and releasing the medical device based on the release mode.

[0201] Each module in the aforementioned medical device control device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0202] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 17 As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a medical device control method. The display unit is used to form a visually visible image and can be a display screen, projection device, or virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0203] Those skilled in the art will understand that Figure 17 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0204] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0205] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0206] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0207] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0208] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0209] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A medical device control system, characterized in that, The control system includes: Image acquisition equipment used to acquire real-time medical images; Medical devices; A pushing device, wherein the medical device is mounted on the pushing device, and the pushing device is used to push the medical device to a target location; The processor is configured to perform the following steps based on the real-time medical images: Acquire real-time medical images during surgery; The real-time medical image is registered with the standard medical model generated before surgery to obtain the first registration relationship; Based on the first registration relationship, the first target path and the first target blood vessel information generated before the operation are mapped to the real-time medical image to obtain the second target blood vessel information and the second target path. The first target blood vessel information and the second target blood vessel information include at least one of the diameter, length and curvature of the blood vessel. Based on the location of the medical device in the real-time medical image, the information of the second target blood vessel, and the second target path, the control information of the medical device is determined; The medical device is controlled based on the control information; The step of determining the control information of the medical device based on the position of the medical device in the real-time medical image, the second target blood vessel information, and the second target path includes: Based on the position of the medical device in the real-time medical image, the third target blood vessel information corresponding to the previous control cycle is selected from the second target blood vessel information; The control speed and movement distance of the medical device are determined based on the third target blood vessel information. The target location of the medical device is determined based on the movement distance and the target path; The control speed and the target position are used as the control information of the medical device.

2. The system according to claim 1, characterized in that, Before registering the real-time medical image with the preoperatively generated standard medical model to obtain the first registration relationship, the method further includes: Acquire all preoperative medical images; Semantic segmentation was performed on each of the aforementioned preoperative medical images to obtain segmentation results; A standard medical model is obtained by spatial location fusion based on the segmentation results described above.

3. The system according to claim 2, characterized in that, The processor is also used to perform the following steps: Regional vascular boundaries were extracted from the standard medical model. The first target blood vessel information is determined by traversing the blood vessel boundaries of the region.

4. The system according to claim 3, characterized in that, The processor is also used to perform the following steps: Target detection is performed on the standard medical model to obtain target information; Based on the target location in the target information, determine the initial path to the target location; The first target path is obtained by filtering each of the initial paths based on the first target blood vessel information.

5. The system according to claim 4, characterized in that, The step of filtering each of the initial paths based on the first target blood vessel information to obtain the first target path includes: Based on the first target blood vessel information, the path factor of each initial path is calculated, and the path factor includes path length, path diameter information and path curvature information. Obtain the weight coefficients of each path factor; Based on the path factors of each initial path and the corresponding weight coefficients, the initial paths are filtered to obtain the first target path.

6. The system according to claim 5, characterized in that, After obtaining the first target path by filtering the initial paths based on the path factors and corresponding weight coefficients of each initial path, the method further includes: The medical device to be used for operation is determined based on the path length and diameter information of the first target path.

7. The system according to claim 1, characterized in that, The step of registering the real-time medical image with the preoperatively generated standard medical model to obtain a first registration relationship includes: Feature extraction is performed on the real-time medical image and the standard medical model to obtain the features to be processed; The real-time medical image and the standard medical model are used to perform feature matching for the features to be processed. The first registration relationship is obtained based on the feature matching results.

8. The system according to claim 1, characterized in that, The medical device is installed in the delivery device; controlling the medical device based on the control information includes: Obtain the second registration relationship between the push device and the image coordinate system corresponding to the real-time medical image; Based on the second registration relationship, the control information of the medical device is converted into the control information of the push device; The push device is controlled based on the control information of the push device.

9. The system according to claim 8, characterized in that, The step of obtaining the second registration relationship between the push device and the image coordinate system corresponding to the real-time medical image includes: At least two push positions are obtained on the calibration paper of the push device; Obtain the positions to be processed in the image coordinate system corresponding to the at least two push positions in the real-time medical image; A second registration relationship is generated based on the push location and the location to be processed.

10. The system according to claim 1, characterized in that, The control system further includes: A sensing device, wherein the sensing device is used to collect force information of the medical device and / or position information of the medical device; The processor is also used to perform the following steps: Based on the force information and / or position information of the medical device collected by the sensing device, the system can detect in real time whether the medical device collides with the blood vessel. When the medical device collides with the blood vessel, control the medical device to stop moving and / or output an alarm message; When the medical device does not collide with the blood vessel, the system continues to monitor whether the medical device collides with the blood vessel in real time until the operation is completed.

11. The system according to claim 10, characterized in that, The real-time detection of whether the medical device collides with the blood vessel includes: Based on the position of the medical device in the real-time medical image and the second target blood vessel information mapped to the real-time medical image, determine the interference between the medical device and the real-time medical image and / or obtain the force situation of the medical device; The medical device is judged to have collided with the blood vessel based on the interference and / or the force conditions described.

12. The system according to claim 1, characterized in that, The processor is also used to perform the following steps: Receive mode switching command; The current control mode is determined according to the mode switching command; When the current control mode is automatic mode, the control information of the medical device is determined based on the position of the medical device in the real-time medical image, the second target blood vessel information mapped to the real-time medical image, and the second target path. When the current control mode is manual control mode, the medical device control information sent by the main operating device is received.

13. The system according to claim 1, characterized in that, After controlling the medical device based on the control information, the following steps are included: When the medical device reaches the target position corresponding to the target path, the release mode of the medical device is determined, and the medical device is released based on the release mode.

14. The system according to claim 13, characterized in that, The control system further includes: The main operating device is used to control the pushing device to push the medical device to the target position when the control mode is manual control mode and / or to manually release the medical device when the medical device reaches the target position.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it performs the following steps: Acquire real-time medical images during surgery; The real-time medical image is registered with the standard medical model generated before surgery to obtain the first registration relationship; Based on the first registration relationship, the first target path and the first target blood vessel information generated before the operation are mapped to the real-time medical image to obtain the second target blood vessel information and the second target path. The first target blood vessel information and the second target blood vessel information include at least one of the diameter, length and curvature of the blood vessel. Based on the location of the medical device in the real-time medical image, the information of the second target blood vessel, and the second target path, the control information of the medical device is determined; The medical device is controlled based on the control information; The step of determining the control information of the medical device based on the position of the medical device in the real-time medical image, the second target blood vessel information, and the second target path includes: Based on the position of the medical device in the real-time medical image, the third target blood vessel information corresponding to the previous control cycle is selected from the second target blood vessel information; The control speed and movement distance of the medical device are determined based on the third target blood vessel information. The target location of the medical device is determined based on the movement distance and the target path; The control speed and the target position are used as the control information of the medical device.

16. The computer-readable storage medium according to claim 15, characterized in that, Before the process of registering the real-time medical image with the preoperatively generated standard medical model to obtain the first registration relationship, as implemented by the computer program executed by the processor, the program further includes: Acquire all preoperative medical images; Semantic segmentation was performed on each of the aforementioned preoperative medical images to obtain segmentation results; A standard medical model is obtained by spatial location fusion based on the segmentation results described above.

17. The computer-readable storage medium according to claim 16, characterized in that, When the computer program is executed by the processor, it also performs the following steps: Regional vascular boundaries were extracted from the standard medical model. The first target blood vessel information is determined by traversing the blood vessel boundaries of the region.

18. The computer-readable storage medium according to claim 17, characterized in that, When the computer program is executed by the processor, it also performs the following steps: Target detection is performed on the standard medical model to obtain target information; Based on the target location in the target information, determine the initial path to the target location; The first target path is obtained by filtering each of the initial paths based on the first target blood vessel information.

19. The computer-readable storage medium according to claim 18, characterized in that, When the computer program is executed by the processor, the process of filtering each of the initial paths based on the first target blood vessel information to obtain the first target path includes: Based on the first target blood vessel information, the path factor of each initial path is calculated, and the path factor includes path length, path diameter information and path curvature information. Obtain the weight coefficients of each path factor; Based on the path factors of each initial path and the corresponding weight coefficients, the initial paths are filtered to obtain the first target path.

20. The computer-readable storage medium according to claim 19, characterized in that, After the computer program, when executed by the processor, filters the initial paths to obtain the first target path based on the path factors and corresponding weight coefficients of each initial path, it further includes: The medical device to be used for operation is determined based on the path length and diameter information of the first target path.

21. The computer-readable storage medium according to claim 15, characterized in that, When the computer program is executed by the processor, the process of registering the real-time medical image with a pre-operatively generated standard medical model to obtain a first registration relationship includes: Feature extraction is performed on the real-time medical image and the standard medical model to obtain the features to be processed; The real-time medical image and the standard medical model are used to perform feature matching for the features to be processed. The first registration relationship is obtained based on the feature matching results.

22. The computer-readable storage medium according to claim 15, characterized in that, The medical device is installed in the delivery device; the control of the medical device based on the control information, implemented by the computer program when executed by the processor, includes: Obtain the second registration relationship between the push device and the image coordinate system corresponding to the real-time medical image; Based on the second registration relationship, the control information of the medical device is converted into the control information of the push device; The push device is controlled based on the control information of the push device.

23. The computer-readable storage medium according to claim 22, characterized in that, When the computer program is executed by the processor, the acquisition of the second registration relationship between the image coordinate system corresponding to the push device and the real-time medical image includes: At least two push positions are obtained on the calibration paper of the push device; Obtain the positions to be processed in the image coordinate system corresponding to the at least two push positions in the real-time medical image; A second registration relationship is generated based on the push location and the location to be processed.

24. The computer-readable storage medium according to claim 15, characterized in that, When the computer program is executed by the processor, it also performs the following steps: Based on the force information and / or position information of the medical device collected by the sensing device, the system can detect in real time whether the medical device collides with the blood vessel. When the medical device collides with the blood vessel, control the medical device to stop moving and / or output an alarm message; When the medical device does not collide with the blood vessel, the system continues to monitor whether the medical device collides with the blood vessel in real time until the operation is completed.

25. The computer-readable storage medium according to claim 24, characterized in that, The real-time detection of whether the medical device collides with the blood vessel, implemented by the computer program when executed by the processor, includes: Based on the position of the medical device in the real-time medical image and the second target blood vessel information mapped to the real-time medical image, determine the interference between the medical device and the real-time medical image and / or obtain the force situation of the medical device; The medical device is judged to have collided with the blood vessel based on the interference and / or the force conditions described.

26. The computer-readable storage medium according to claim 15, characterized in that, When the computer program is executed by the processor, it also performs the following steps: Receive mode switching command; The current control mode is determined according to the mode switching command; When the current control mode is automatic mode, the control information of the medical device is determined based on the position of the medical device in the real-time medical image, the second target blood vessel information mapped to the real-time medical image, and the second target path. When the current control mode is manual control mode, the medical device control information sent by the main operating device is received.

27. The computer-readable storage medium according to claim 15, characterized in that, After the computer program is executed by the processor, the control of the medical device based on the control information includes: When the medical device reaches the target position corresponding to the target path, the release mode of the medical device is determined, and the medical device is released based on the release mode.

Images