A surgical robot speed limiting method and system

Abstract

The embodiment of the specification provides a surgical robot speed limiting method and system, which can be applied to the technical field of medical instruments. A three-dimensional model corresponding to a surgical area is constructed according to a surgical area image; the surgical area image comprises an image collected in a surgical execution process; a surgical instrument and patient biological tissue are distinguished in the three-dimensional model; the distance between the surgical instrument and the patient biological tissue is determined in combination with the three-dimensional model; the movement speed and / or movement speed threshold of a surgical robot is limited based on the distance between the surgical instrument and the patient biological tissue; and the surgical robot is used to drive the surgical instrument to move. The above embodiment ensures that the currently set movement speed threshold meets the needs of the current surgical state, and ensures the execution effect of the surgery. In addition, by identifying the surgical instrument and the patient biological tissue in the three-dimensional model, the accuracy of distance measurement is improved, and the actual application effect is ensured.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of medical device technology, and in particular to a method and system for limiting the speed of a surgical robot. Background Technology

[0002] With its advantages of less damage, less bleeding, and faster recovery, minimally invasive surgery has experienced rapid development and widespread application. During minimally invasive surgery, only a few small incisions are made on the surface of the patient's body at the surgical site. Through these incisions, an endoscope and surgical instruments are inserted into the patient's body. The endoscope allows for monitoring of the patient's internal condition, and surgical robots can be used to operate the instruments and perform the corresponding surgical procedures.

[0003] Unlike traditional surgery where doctors hold surgical instruments, minimally invasive surgery involves doctors controlling a surgical robot via a control terminal. The surgical robot indirectly uses surgical instruments to perform corresponding operations. To ensure the safety of the surgical process, the movement of the surgical instruments by the surgical robot must be at its maximum speed to avoid increasing the difficulty of the operation due to the surgical instruments moving too fast and causing unexpected trauma to the patient.

[0004] Currently, the movement speed threshold is generally set based on the surgeon's practical experience. However, different surgical scenarios and different stages of the same surgery may have different requirements for the movement speed of the surgical robot, making the experience-based threshold unsuitable for all situations. Setting the threshold too high for the current needs increases the risk of trauma to the patient, while setting it too low reduces surgical efficiency and prolongs operation time. Therefore, there is an urgent need for a method that allows setting the surgical robot's movement speed threshold according to actual surgical requirements. Summary of the Invention

[0005] The purpose of the embodiments in this specification is to provide a method and system for limiting the speed of a surgical robot, so as to solve the problem of how to set the threshold of the movement speed of a surgical robot based on actual surgical needs.

[0006] To address the aforementioned technical problems, embodiments of this specification propose a method for limiting the speed of a surgical robot, comprising: constructing a three-dimensional model corresponding to the surgical area based on an image of the surgical area; the surgical area image including images acquired during the surgical procedure; distinguishing surgical instruments and patient biological tissue in the three-dimensional model; determining the distance between the surgical instruments and the patient biological tissue based on the three-dimensional model; limiting a threshold for the movement speed of the surgical robot based on the distance between the surgical instruments and the patient biological tissue; and the surgical robot being used to move the surgical instruments.

[0007] In some embodiments, constructing a three-dimensional model corresponding to the surgical area based on the surgical area image includes: acquiring depth data corresponding to the surgical area image; the depth data being used to describe the distance between the acquisition device of the surgical area image and each point in the surgical area image; constructing a three-dimensional point cloud map corresponding to the surgical area based on the depth data; and constructing a three-dimensional model corresponding to the surgical area by combining the three-dimensional point cloud map and the surgical area image.

[0008] Based on the above implementation method, the step of constructing a three-dimensional model corresponding to the surgical area by combining the three-dimensional point cloud map and the surgical area image includes: determining a continuous spatial range based on continuously distributed points in the three-dimensional point cloud map; and constructing a three-dimensional model based on the continuous spatial range.

[0009] Based on the above embodiments, the step of obtaining depth data corresponding to the surgical area image includes: when the surgical area image is a parallax image captured by a binocular camera, calculating depth data for the parallax image using the binocular parallax principle, or obtaining depth data obtained by a distance sensor, wherein the distance sensor includes at least one of lidar, infrared sensor, and acoustic rangefinder.

[0010] In some embodiments, distinguishing surgical instruments and patient biological tissue in the three-dimensional model includes: using a classification model to distinguish surgical instrument images and patient biological tissue images in the surgical area image; and determining the surgical instruments and patient biological tissues corresponding to the surgical instrument images and patient biological tissue images in the three-dimensional model according to the correspondence between the surgical area image and the three-dimensional model.

[0011] Based on the above implementation, before using the classification model to distinguish between surgical instrument images and patient biological tissue images in the surgical area image, the method further includes: preprocessing the surgical area image; the preprocessing includes at least one of image filtering, image denoising, and image dimensionality reduction.

[0012] Based on the aforementioned implementation method, the classification model is obtained through the following means: acquiring sample image data; labeling surgical instruments and patient biological tissues in the sample image data; extracting image features from the sample image data; training an initial classification model using the image features until the trained model meets the application conditions; the initial classification model includes a neural network model.

[0013] Based on the above embodiments, before extracting image features from the sample image data, the method further includes: preprocessing the sample image data; the preprocessing includes at least one of image filtering, image denoising, and image dimensionality reduction.

[0014] In some embodiments, the movement speed threshold and the distance between the surgical instrument and the patient's biological tissue are linearly related, or the movement speed threshold is set based on the interval distance range corresponding to the distance between the surgical instrument and the patient's biological tissue; the interval distance range is a region determined according to at least one pre-set division distance.

[0015] In some embodiments, the surgical robot includes a motor and a motor drive unit; the motor drive unit is used to output a power output signal for driving the motor; the motor is used to drive the surgical robot to move; limiting the movement speed threshold of the surgical robot includes: sending the movement speed threshold to the motor drive unit so that the motor drive unit calculates a power threshold based on the movement speed threshold, and limits the power output of the drive motor based on the power threshold.

[0016] In some embodiments, after distinguishing surgical instruments and patient biological tissue in the three-dimensional model, the method further includes: if at least two surgical instruments are distinguished in the three-dimensional model, determining the distance between each surgical instrument; and limiting the movement speed threshold of the surgical robot based on the distance between each surgical instrument.

[0017] In some implementations, after limiting the movement speed threshold of the surgical robot based on the distance between the surgical instrument and the patient's biological tissue, the method further includes: after detecting that the movement speed of the surgical robot has reached the movement speed threshold, displaying a prompt message on the doctor's control terminal to inform the doctor that the movement speed of the surgical robot is currently restricted.

[0018] In some implementations, after determining the distance between the surgical instrument and the patient's biological tissue using the three-dimensional model, the method further includes adding corresponding markers to the surgical instrument and the patient's biological tissue on the doctor's control terminal based on the different distance ranges corresponding to the distance between the surgical instrument and the patient's biological tissue.

[0019] This specification also proposes a surgical robot speed limiting system, including a surgical robot, an image sensing device, surgical instruments, and a processor. The surgical robot is used to hold the image sensing device and the surgical instruments and drive them to move. The image sensing device is used to acquire a surgical area image corresponding to the surgical area. The processor is used to receive the surgical area image and perform the following steps: constructing a three-dimensional model corresponding to the surgical area based on the surgical area image; the surgical area image includes images acquired during the surgical procedure; distinguishing between surgical instruments and patient biological tissue in the three-dimensional model; determining the distance between the surgical instruments and patient biological tissue based on the three-dimensional model; limiting the moving speed threshold of the surgical robot based on the distance between the surgical instruments and patient biological tissue; the surgical robot is used to drive the surgical instruments to move.

[0020] This specification also proposes a computer-readable storage medium storing a computer program / instructions that, when executed, implement the above-described surgical robot speed limiting method.

[0021] As can be seen from the technical solutions provided in the embodiments of this specification above, the above-described surgical robot speed limiting method acquires images of the surgical area during surgical execution, constructs a three-dimensional model corresponding to the surgical area using these images, distinguishes surgical instruments and patient biological tissue within the three-dimensional model, and determines the distance between the surgical instruments and patient biological tissue based on the display effect in the three-dimensional model. This allows for the limitation of the surgical robot's movement speed threshold based on the distance between the surgical instruments and patient biological tissue. The above method adjusts the surgical robot's movement speed threshold according to the real-time execution status during surgery, thereby ensuring that the currently set movement speed threshold meets the needs of the current surgical state. This avoids damage to patient tissue caused by excessively fast movement of the surgical instruments, and also avoids reduced surgical efficiency due to excessively slow movement of the surgical instruments, thus ensuring the effectiveness of the surgical execution. Furthermore, by identifying surgical instruments and patient biological tissue in the three-dimensional model, the accuracy of distance measurement is improved, ensuring the effectiveness of practical applications. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a schematic diagram of a minimally invasive surgical environment as described in an embodiment of this specification;

[0024] Figure 2 This is a schematic diagram of the structure of a surgical robot speed limiting system according to an embodiment of this specification;

[0025] Figure 3 This is a schematic diagram of the structure of a surgical robot according to an embodiment of this specification;

[0026] Figure 4 This is a schematic diagram of a binocular endoscope as an embodiment of this specification;

[0027] Figure 5 This is a flowchart illustrating a method for limiting the speed of a surgical robot according to an embodiment of this specification;

[0028] Figure 6 This is a schematic diagram illustrating the principle of binocular vision in an embodiment of this specification;

[0029] Figure 7A This is a schematic diagram of a point cloud diagram according to an embodiment of this specification;

[0030] Figure 7B This is a schematic diagram of a point cloud diagram according to an embodiment of this specification;

[0031] Figure 7C This is a schematic diagram of a point cloud diagram according to an embodiment of this specification;

[0032] Figure 8A This is a schematic diagram of a three-dimensional point cloud diagram according to an embodiment of this specification;

[0033] Figure 8B This is a schematic diagram of a three-dimensional model as an embodiment of this specification;

[0034] Figure 9 This is a schematic diagram illustrating surgical instrument identification based on an image of a surgical area, as described in this specification.

[0035] Figure 10 This is a schematic diagram of the structure of a motor drive unit and a motor according to an embodiment of this specification;

[0036] Figure 11 This is a schematic diagram illustrating the boundary delineation of biological tissues according to an embodiment of this specification;

[0037] Figure 12 This is a schematic diagram illustrating the distance division between a surgical instrument and a tissue / organ according to an embodiment of this specification;

[0038] Figure 13 This is a schematic diagram illustrating the relationship between distance and speed threshold in an embodiment of this specification;

[0039] Figure 14 This is a schematic diagram illustrating a display screen showing prompt information according to an embodiment of this specification;

[0040] Figure 15 This is a schematic diagram illustrating the marking of surgical instruments according to an embodiment of this specification. Detailed Implementation

[0041] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0042] To better understand the technical solution of this application, we will first introduce the minimally invasive surgical scenarios in which the surgical robot speed limiting system of this application is applied.

[0043] like Figure 1 The diagram illustrates a real-world minimally invasive surgical scenario. This execution environment includes an imaging cart, a patient operating console, and a surgeon's operating console. The patient operating console corresponds to the surgical robot, which may contain multiple robotic arms. These robotic arms can be used to hold surgical instruments or endoscopes, among other devices. The robotic arms can move the surgical instruments or endoscopes, allowing surgeons to view the patient's internal condition from different perspectives during the procedure or perform corresponding surgical operations.

[0044] The image cart can be connected to an endoscope to display images captured by the endoscope on a screen for other medical staff to view. The image cart also has some image processing capabilities, allowing it to display relevant information after image processing.

[0045] The doctor's control unit is the terminal device operated by the doctor. By receiving image signals transmitted from the image carriage, the doctor can observe the images captured by the endoscope and understand the current surgical status inside the patient's body. The doctor's control unit can also control the robotic arm on the patient's control unit to adjust the endoscopic observation angle and perform specific operations using surgical instruments.

[0046] Currently, in minimally invasive surgery, the surgeon observes images of the patient's body obtained through the endoscope at the surgeon's end, and operates the control arm on the surgeon's end. The surgeon's end then generates corresponding operation signals based on the surgeon's actions and transmits them to the patient's end, so that the robotic arm on the patient's end moves the surgical instruments or endoscope according to the operation signals. Since the surgeon does not directly control the surgical instruments during this process, it is necessary to limit the movement speed of the surgical robot corresponding to the patient's end to ensure surgical safety. However, the currently set movement speed threshold is merely a fixed value set based on past operational experience, making it difficult to dynamically change the movement speed and / or movement speed threshold according to the actual needs of the surgery.

[0047] To address the aforementioned problems, this specification provides an embodiment of a surgical robot speed limiting system. For example... Figure 2 As shown, the surgical robot speed limiting system 200 includes a surgical robot 210, an image sensing device 220, surgical instruments 230, and a processor 240.

[0048] Surgical robot 210 is a robot installed at the patient's operating end to perform surgical procedures on the patient. The surgical robot 210 can hold surgical instruments 230 and an image sensor 220. By moving the image sensor 220, it can acquire images from different perspectives, and by moving the surgical instruments 230, it can perform corresponding surgical operations. Figure 3 The diagram shows a surgical robot 210. Different joints on the surgical robot 210 can move in different directions under the drive of motors, such as moving up and down as a whole, rotating as a whole, and manipulating the robotic arm to make precise movements.

[0049] Surgical instruments 230 are instruments used to achieve different surgical effects during the surgical procedure, such as instruments for cutting or sampling patient tissue. The specific type of surgical instrument 230 can be configured according to actual application needs, and will not be elaborated upon here.

[0050] The image sensing device 220 is used to acquire images. Preferably, the images acquired by the image sensing device 220 can be used to construct corresponding three-dimensional models. For example, Figure 4 As shown, the image sensing device 220 can be an endoscope equipped with a binocular camera to determine the spatial position of different points in the surgical area based on the principle of binocular vision in subsequent processes. The image sensing device 220 may also additionally include sensors capable of distance measurement, such as lidar, infrared ranging sensors, acoustic ranging sensors, etc., so as to simultaneously determine the spatial distance of the surgical area and the corresponding different positions in the image based on the captured image.

[0051] The processor 240 can receive surgical area images acquired by the image sensing device 220, analyze and process the surgical area images, and ultimately determine the movement speed and / or movement speed threshold for the surgical robot 210. The specific analysis and processing procedure can be found in the description of the method for limiting the speed of the surgical robot 210. After determining the movement speed threshold, the processor 240 can send a corresponding signal to the surgical robot 210 to inform it of the currently limited movement speed threshold, ensuring that the speed at which the surgical robot 210 moves the surgical instrument 230 does not exceed the movement speed threshold.

[0052] The communication relationships between different components in the system can be referred to Figure 2 The description in the text, but not limited to Figure 2 direction of communication.

[0053] Based on the above-described surgical robot speed limiting system, this specification introduces a method for limiting the speed of a surgical robot according to an embodiment. The processor can be the entity executing this method. Figure 5 As shown, the method for limiting the speed of the surgical robot includes the following specific implementation steps.

[0054] S510: Construct a three-dimensional model corresponding to the surgical area based on the surgical area image; the surgical area image includes images acquired during the surgical procedure.

[0055] The surgical area image is the image acquired by the image sensing device. To align with the purpose of this invention, the surgical area image is an image acquired in real-time by the image sensing device of the surgical area during the surgical procedure.

[0056] The surgical area refers to the environment in which the surgery is currently being performed. For example, in the case of laparoscopic surgery, the surgical area is the patient's intra-abdominal environment. Accordingly, the surgical area image can be an image acquired after an image sensing device is inserted into the patient's body.

[0057] When the image sensing device is a binocular endoscope, the surgical area image can be multiple sets of images captured by different cameras of the binocular endoscope.

[0058] Preferably, the surgical area image needs to include surgical instruments and patient biological tissue so that the distance between the surgical instruments and patient biological tissue can be determined in subsequent execution steps.

[0059] In some implementations, the three-dimensional model can be constructed based on a three-dimensional point cloud. A point cloud can refer to a dataset of points in a specific coordinate system, thus representing the position of each sampling point in the three-dimensional coordinate system, as well as the spatial distance between each sampling point, etc.

[0060] Specifically, depth data corresponding to the surgical area image can be obtained first, then a three-dimensional point cloud map corresponding to the surgical area can be constructed based on the depth data, and a three-dimensional model corresponding to the surgical area can be constructed by combining the three-dimensional point cloud map and the surgical area image.

[0061] Depth data is used to describe the distance between points in the surgical area image and the image sensing device. Since this involves the construction of a 3D model, while the captured surgical area image is generally a 2D image, depth data needs to be determined first to build the model.

[0062] In some examples, where the surgical area image is a parallax image captured using a binocular camera, depth data can be directly calculated based on the binocular parallax principle using the parallax image.

[0063] like Figure 6 The diagram illustrates a scene captured by a binocular camera, where P, PL, and PR represent the object point and the object's pixels on the normalized planes of the left and right eyes, respectively. Due to displacement of the left and right eyes along the baseline b axis, the image distance of object P differs between the left and right cameras. Based on the geometric relationships shown in the diagram, ΔPP... L P R and ΔPO L O R Similarity allows for the construction of formulas. In the formula, z is the depth distance, f is the focal length, i.e., the distance from the normalized plane to the optical center of the camera, and u L u is the distance from the left pixel to the left eye camera. R is the distance from the right pixel to the right eye camera, and b is the length of the baseline between the left eye camera and the right eye camera.

[0064] The distance between each pixel and the stereo camera can be calculated using the above method, thereby obtaining depth data.

[0065] In other examples, the image sensing device may also include a distance sensor, which includes at least one of lidar, infrared sensors, and acoustic rangefinders, thereby directly acquiring depth data. Correspondingly, the image sensing device may consist of only a monocular camera, allowing a correspondence between the images captured by the monocular camera and the depth data.

[0066] Preferably, after obtaining the surgical area image and depth data, a surgical area image-depth information database can be constructed, and a mapping relationship between the surgical area image and depth data can be established.

[0067] After acquiring the depth data, a 3D point cloud map can be constructed based on it. For example... Figure 7AThe diagram shown is a schematic of a point cloud map corresponding to the direction of a binocular camera lens. The sampling points in the point cloud map are uniformly distributed in the plane. Each sampling point contains the x-coordinate, y-coordinate, and depth information z of each point, which comprehensively reflects the coordinates of the sampling point in space.

[0068] For each sampling point, the depth distance is quantized on the corresponding coordinate axis to represent the vertical depth of the point cloud. For example, ... Figure 7B As shown, planes A and B represent the quantized depth distance intervals. By using the depth information from each sampling point, a 3D dense point cloud map can be constructed. Figure 7C The image shown is a schematic diagram of a three-dimensional dense point cloud map, in which different parts can be distinguished at different depth distances. Figure 7C The location of the surgical instruments can be determined from this.

[0069] After constructing a 3D point cloud map, since each point in the 3D point cloud map reflects the position of each measured point on the object's surface, a continuous spatial range can be determined based on the continuously distributed points in the 3D point cloud map, and then a 3D model can be constructed based on this continuous spatial range. The continuous spatial range generally corresponds to the object's surface; therefore, it can reflect the spatial distribution of the object's surface, thus completing the model reconstruction. For example... Figure 8A The image shown is a 3D point cloud containing surgical instruments. After reconstructing the point cloud of the surgical instrument portion, the desired result can be obtained. Figure 8B The model reconstruction diagram shown is used to complete the construction of a three-dimensional model corresponding to the surgical instruments.

[0070] The above steps primarily involve constructing a 3D model of the surgical area using point clouds. In practical applications, other methods can also be used to construct the 3D model, such as analyzing and processing the disparity image to directly convert it into a 3D model. Specific application methods can be configured according to the actual application requirements, and will not be elaborated upon here.

[0071] In some implementations, after acquiring the surgical area image, in order to ensure the effective execution of subsequent steps, the surgical area image can be preprocessed. The preprocessing operation may include at least one of image filtering processing and image dimensionality reduction processing.

[0072] Image filtering primarily aims to remove images that are unsuitable for the current application requirements, including blurry images and invalid images that do not contain surgical instruments or biological tissues for identification, ensuring the proper execution of subsequent operations. The filtering method can pre-identify surgical area images to determine if surgical instruments or biological tissues can be identified, thus confirming whether the surgical area image meets the processing criteria.

[0073] Image dimensionality reduction can be achieved by using image filtering algorithms such as median filtering and Gaussian filtering to denoise and enhance images. Furthermore, dimensionality reduction reduces the amount of data in the image, preventing overfitting and ensuring better subsequent processing results.

[0074] In practical applications, other preprocessing methods can be used to optimize the quality of surgical area images, and these are not limited to the examples mentioned above, which will not be elaborated here.

[0075] S520: Distinguish between surgical instruments and patient biological tissues in the three-dimensional model.

[0076] After the 3D model is constructed, surgical instruments and patient biological tissues can be distinguished within it. Specifically, this can be achieved by identifying surgical instrument images corresponding to surgical instruments and patient biological tissue images corresponding to patient biological tissues in the surgical area image, and then distinguishing between surgical instruments and patient biological tissues in the 3D model based on the correspondence between the surgical area image and the 3D model.

[0077] Because surgical instruments and biological tissues typically differ in color, outline, and other image features, color spaces can be used directly within the image to identify surgical instruments and biological tissues. For example, ... Figure 9 As shown, images can be binarized, color contrast in the image can be analyzed to determine different regions, and the outlines of surgical instruments or biological tissues can be identified through edge detection and judgment to complete image segmentation, thereby distinguishing surgical instrument images from patient biological tissue images.

[0078] In some implementations, a classification model can be used to distinguish between surgical instrument images and patient biological tissue images within the surgical area image. The classification model can be a neural network model, which identifies different modules within the image through image analysis and processing.

[0079] Specifically, before applying the classification model, the model can be trained using sample image data to ensure the accuracy of the classification model.

[0080] The sample image data can be pre-labeled for surgical instruments and patient biological tissues to train the classification model through supervised learning. Preferably, the surgical region images can be preprocessed before model training. Preprocessing operations can include at least one of image filtering, image dimensionality reduction, and image annotation. Image filtering mainly removes images that are not suitable for the current application requirements, including blurry images and invalid images that do not contain surgical instruments or biological tissues, ensuring the normal execution of subsequent operations. The filtering method can pre-identify the surgical region images to determine whether surgical instruments or biological tissues can be identified from them, thus determining whether the surgical region images meet the processing conditions. Image dimensionality reduction can use image filtering algorithms such as median filtering and Gaussian filtering to denoise and enhance the images. In addition, dimensionality reduction can reduce the amount of data in the image, avoid overfitting, and ensure the subsequent execution effect. Image annotation mainly labels the surgical instruments and patient biological tissues in the images. Specifically, it can be done manually or automatically based on the characteristics of the instruments themselves or other image analysis methods, which will not be elaborated here.

[0081] Feature extraction can be performed on sample image data to obtain image features, which can then be converted into vector features, facilitating their use in subsequent processes.

[0082] The initial classification model, pre-built, is trained using image features until it meets the application requirements. This initial classification model may include a neural network model. For example, a loss function can be set, a loss value can be calculated using this function, and the initial classification model can be optimized based on the loss value. This iterative process is repeated until the loss value meets certain conditions. In practical applications, the specific training process of the model can be customized according to requirements, which will not be elaborated here.

[0083] After distinguishing between surgical instrument images and patient biological tissue images in the surgical area image, the corresponding surgical instruments and patient biological tissues can be directly determined in the 3D model based on the correspondence between the surgical area image and the 3D model. This allows for direct analysis of the 3D model in subsequent steps to determine the distance between the surgical instruments and patient biological tissues.

[0084] S530: Based on the three-dimensional model, determine the distance between the surgical instruments and the patient's biological tissue.

[0085] After determining the modules corresponding to surgical instruments and patient biological tissues in the 3D model, the distance between the surgical instruments and patient biological tissues can be determined based on the proportional relationships in the 3D model. For example, the distance between the modules corresponding to surgical instruments and patient biological tissues in the 3D model can be predetermined, and then this distance can be converted into the actual distance between the surgical instruments and patient biological tissues based on a fixed proportional relationship.

[0086] When the surgical instrument and the patient's biological tissue are three-dimensional modules, the determined distance can be the minimum distance between the sampling point corresponding to the surgical instrument and the patient's biological tissue, or it can be the distance between a specific part of the surgical instrument and the patient's biological tissue, without any limitation.

[0087] In some implementations, if the constructed three-dimensional model contains at least two surgical instruments, in order to avoid interference between the surgical instruments, the distance between each surgical instrument can be determined separately, so as to combine the distance between the surgical instruments and the patient's biological tissue, and the distance between each surgical instrument to limit the movement speed and / or movement speed threshold of the surgical robot in subsequent steps.

[0088] S540: Based on the distance between the surgical instrument and the patient's biological tissue, a moving speed and / or a moving speed threshold of the surgical robot are defined; the surgical robot is used to move the surgical instrument.

[0089] Determining the distance between the surgical instruments and the patient's biological tissue allows for the limitation of the surgical robot's movement speed and / or speed threshold. Limiting the movement speed can be achieved by directly controlling the robot's current speed; for example, maintaining a constant speed and preventing further increases, or by gradually decreasing the speed, with the reduction magnitude and rate set based on requirements. The speed threshold represents the surgical robot's maximum movement speed. This threshold can be greater than or equal to the current speed, preventing further increases beyond it. Conversely, it can be less than the current speed, requiring the robot to reduce its speed below the threshold. The choice between limiting the movement speed, limiting the speed threshold, or limiting both can be made based on specific application requirements and corresponding judgment conditions.

[0090] Specifically, the surgical robot includes motors and a motor drive unit. The motors can be located at various joints of the surgical robot to drive its movement. The motor drive unit outputs a power output signal to the motors, i.e., to limit the motor power. After the motor drive unit obtains a specific movement speed and / or movement speed threshold, it can calculate the power and / or power threshold based on the movement speed and / or movement speed threshold, thereby limiting the power output of the drive motor according to the corresponding power and / or power threshold.

[0091] like Figure 10 As shown, the motor drive unit may include a controller and a driver. The controller can communicate with a processor to receive real-time changes in the movement speed and / or a movement speed threshold. Accordingly, it outputs a corresponding control signal based on the movement speed and / or the movement speed threshold. The driver adjusts the motor's power output according to the received control signal, thereby controlling and / or limiting the movement speed of the surgical robot.

[0092] In some embodiments, the movement speed and / or movement speed threshold are limited based on an interval range corresponding to the distance between the surgical instrument and the patient's biological tissue. The interval range is a region determined according to at least one pre-defined division distance.

[0093] Based on this implementation method, the spatial location of the patient's biological tissue can be determined first, then different spatial regions can be determined based on the distance to the patient's biological tissue, and different movement speeds and / or movement speed thresholds can be determined according to the spatial region where the surgical instrument is located. For example, Figure 11 As shown, for biological tissue, the boundaries of Region 1 and Region 2 can be determined to divide Region 1 and Region 2. Region 1 and Region 2 correspond to a pre-set movement speed and / or movement speed threshold. Based on the identified distance between the surgical instrument and the biological tissue, the region where the surgical instrument is located can be directly determined, and thus the corresponding movement speed and / or movement speed threshold can be determined.

[0094] Based on this implementation method, different distance interval values ​​can also be pre-defined according to the distance between the surgical instrument and the biological tissue, and a corresponding moving speed and / or moving speed threshold can be assigned to each distance interval value. For example, Figure 12 As shown, based on the distance between the surgical instrument and the tissue / organ, five positions A, B, C, D, and E are defined. Based on the actual distance between the surgical instrument and the biological tissue and the proximity relationship between these five positions, the corresponding movement speed and / or movement speed threshold can be directly determined.

[0095] In other embodiments, the movement speed and / or movement speed threshold decreases as the distance between the surgical instrument and the patient's biological tissue decreases. In this embodiment, based on the movement of the surgical instrument, the movement speed threshold needs to be changed in real time to prevent the surgical instrument from "running away" in the surgical environment. For example, as... Figure 13 As shown, different movement speeds and / or movement speed thresholds are set for different distances, where the movement speed and / or movement speed thresholds increase continuously with increasing distance. Accordingly, the movement speed and / or movement speed thresholds can vary differently within different distance ranges.

[0096] In the above embodiments, the specific correspondence between the movement speed and / or the movement speed threshold and the distance can be determined in advance through experiments and stored in a corresponding database so that the movement speed threshold can be directly determined according to the correspondence in practical applications, thus ensuring the application effect of actual surgery.

[0097] Based on the example in step S530, when two surgical instruments are identified, the movement speed and / or movement speed threshold of the surgical robot can also be limited based on the distance between the surgical instruments. The specific limiting method can be referred to the above description and will not be repeated here. Furthermore, the standard for limiting the movement speed and / or movement speed threshold based on the distance between the surgical instruments can differ from the standard for limiting the movement speed and / or movement speed threshold based on the distance between the surgical instruments and the patient's biological tissue; it can be set according to actual needs.

[0098] In some implementations, in order to effectively ensure the surgical outcome, if the movement speed of the surgical robot is detected to have reached the currently limited movement speed threshold during the operation, even though the movement speed of the surgical robot is within the limited range, it may be too fast. In this case, a prompt message can be displayed on the doctor's control terminal to inform the doctor that the movement speed of the surgical robot is currently restricted, thereby improving the doctor's control over the surgical instruments.

[0099] For example, such as Figure 14 As shown, a prompt information area can be set above the operation screen on the doctor's operating terminal. When the movement speed of the surgical robot is detected to have reached the movement speed threshold, the prompt information area can display "The current robot movement speed has reached the threshold" to remind the doctor of the current operation speed.

[0100] In some implementations, when displaying an image of the surgical area on the doctor's control screen, corresponding markers can be added to the surgical instruments and patient tissue on the doctor's control terminal based on different distance ranges corresponding to the distance between the surgical instruments and the patient's biological tissue. For example, when the distance between the surgical instruments and the patient's biological tissue is less than a certain value, other colors can be overlaid on the surgical instruments, and the vibrancy of the color can increase as the distance decreases; in addition, different texture formats can also be set, without limitation.

[0101] like Figure 15 As shown, different textures can be added to surgical instruments to indicate the distance between different surgical instruments and biological tissues, thus helping doctors to better perform minimally invasive surgery.

[0102] As can be seen from the above embodiments, the surgical robot speed limiting method acquires images of the surgical area during surgery and constructs a three-dimensional model corresponding to the surgical area using these images. After distinguishing between surgical instruments and patient tissues within the three-dimensional model, the distance between the surgical instruments and patient tissues is determined based on the display effect in the three-dimensional model. This allows for the limitation of the surgical robot's movement speed and / or movement speed threshold based on the distance between the surgical instruments and patient tissues. This method adjusts the surgical robot's movement speed and / or movement speed threshold according to the real-time execution status during surgery, ensuring that the currently set movement speed threshold meets the needs of the current surgical state. This avoids damage to patient tissues caused by excessively fast movement of the surgical instruments, and also avoids reduced surgical efficiency due to excessively slow movement of the surgical instruments, thus ensuring the effectiveness of the surgery. Furthermore, by identifying surgical instruments and patient tissues in the three-dimensional model, the accuracy of distance measurement is improved, ensuring the effectiveness of practical applications.

[0103] based on Figure 5 Regarding the corresponding surgical robot speed limiting method, this specification provides a computer-readable storage medium storing a computer program / instruction. The computer-readable storage medium can be read by a processor via the device's internal bus, and the processor can then implement the program instructions in the computer-readable storage medium.

[0104] In this embodiment, the computer-readable storage medium can be implemented in any suitable manner. The computer-readable storage medium includes, but is not limited to, random access memory (RAM), read-only memory (ROM), cache, hard disk drive (HDD), memory card, etc. The computer storage medium stores computer program instructions. When the computer program instructions are executed, this specification is implemented. Figure 1 The program instructions or modules corresponding to the embodiments.

[0105] In this embodiment, the processor can be implemented in any suitable manner. For example, the processor can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers, etc. Specifically, the processor can execute when it is set on a surgical robot speed limiting system. Figure 5 The method steps in the corresponding embodiments.

[0106] Although the process described above includes multiple operations that occur in a specific order, it should be clearly understood that these processes may include more or fewer operations, which may be executed sequentially or in parallel (e.g., using parallel processors or a multithreaded environment).

[0107] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0108] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0109] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0110] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0111] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0112] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0113] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, the embodiments of this specification can take the form of computer program products implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0114] The embodiments described in this specification can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. The embodiments of this specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0115] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, system embodiments are basically similar to method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. In the description of this specification, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the embodiments in this specification. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0116] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed, a method for limiting the speed of a surgical robot is implemented, the method comprising: Constructing a 3D model corresponding to the surgical area based on a surgical area image; including: acquiring depth data corresponding to the surgical area image; the depth data being used to describe the distance between the image sensing device of the surgical area image and each point in the surgical area image, the surgical area image containing surgical instruments; constructing a 3D point cloud map corresponding to the surgical area based on the depth data; constructing a 3D model corresponding to the surgical area based on the 3D point cloud map and the surgical area image; the surgical area image including images acquired during the surgical procedure; and establishing a mapping relationship between the surgical area image and the depth data; Distinguishing surgical instruments and patient biological tissue in the three-dimensional model includes: using a classification model to distinguish surgical instrument images and patient biological tissue images in the surgical area image; and determining the surgical instruments and patient biological tissues corresponding to the surgical instrument images and patient biological tissue images respectively in the three-dimensional model according to the correspondence between the surgical area image and the three-dimensional model. Based on the three-dimensional model, the distance between the surgical instruments and the patient's biological tissues is determined; The movement speed and / or movement speed threshold of the surgical robot are defined based on the distance between the surgical instrument and the patient's biological tissue; the surgical robot is used to move the surgical instrument. Based on the different distance ranges corresponding to the distance between the surgical instruments and the patient's biological tissue, corresponding markers are added to the surgical instruments and the patient's biological tissue on the doctor's control terminal.

2. The computer-readable storage medium as claimed in claim 1, characterized in that, The step of constructing a three-dimensional model corresponding to the surgical area by combining the three-dimensional point cloud map and the surgical area image includes: Determine the continuous spatial range based on continuously distributed points in a 3D point cloud map; A three-dimensional model is constructed based on the continuous spatial range.

3. The computer-readable storage medium as claimed in claim 1, characterized in that, The step of acquiring depth data corresponding to the surgical area image includes: In the case where the surgical area image is a parallax image captured by a binocular camera, depth data is calculated for the parallax image using the binocular parallax principle, or... The system acquires depth data from a distance sensor, which includes at least one of a lidar, an infrared sensor, and an acoustic rangefinder.

4. The computer-readable storage medium as claimed in claim 1, characterized in that, The classification model is obtained through the following method: Acquire sample image data; the sample image data is labeled with surgical instruments and patient biological tissues; Image features are obtained by extracting features from the sample image data; The initial classification model is trained using the image features until the trained model meets the application conditions. The initial classification model includes a neural network model.

5. The computer-readable storage medium as claimed in claim 1, characterized in that, The movement speed and / or movement speed threshold decrease as the distance between the surgical instrument and the patient's biological tissue decreases, or, Based on the interval range corresponding to the distance between the surgical instrument and the patient's biological tissue, a corresponding moving speed and / or moving speed threshold are set; the interval range is a region determined according to at least one pre-set division distance.

6. The computer-readable storage medium as claimed in claim 1, characterized in that, After distinguishing surgical instruments and patient biological tissues in the three-dimensional model, the process further includes: If at least two surgical instruments are distinguished in the three-dimensional model, the distance between each surgical instrument is determined. The threshold for the movement speed of the surgical robot is limited based on the distance between each surgical instrument.

7. The computer-readable storage medium as claimed in claim 1, characterized in that, After determining the movement speed threshold of the surgical robot based on the distance between the surgical instrument and the patient's biological tissue, the method further includes: Once the movement speed of the surgical robot is detected to have reached the threshold, a prompt message is displayed on the doctor's control panel to inform the doctor that the movement speed of the surgical robot is currently restricted.

8. A speed limiting system for a surgical robot, characterized in that, This includes surgical robots, image sensing devices, surgical instruments, and processors; The surgical robot is used to hold the image sensing device and surgical instruments, and to move the image sensing device and surgical instruments. The image sensing device is used to acquire images of the surgical area corresponding to the surgical area; The processor is configured to receive the surgical area image and perform the following steps: constructing a three-dimensional model corresponding to the surgical area based on the surgical area image; This includes: acquiring depth data corresponding to the surgical area image; the depth data being used to describe the distance between the image sensing device of the surgical area image and each point in the surgical area image, the surgical area image containing surgical instruments; constructing a three-dimensional point cloud map corresponding to the surgical area based on the depth data; constructing a three-dimensional model corresponding to the surgical area based on the three-dimensional point cloud map and the surgical area image; the surgical area image including images acquired during the surgical procedure; constructing a mapping relationship between the surgical area image and the depth data; distinguishing surgical instruments and patient biological tissue in the three-dimensional model; including: using a classification model to distinguish surgical instrument images and patient biological tissue images in the surgical area image; determining the surgical instruments and patient biological tissue corresponding to the surgical instrument images and patient biological tissue images respectively in the three-dimensional model according to the correspondence between the surgical area image and the three-dimensional model; determining the distance between the surgical instruments and patient biological tissue based on the three-dimensional model; limiting the movement speed threshold of the surgical robot based on the distance between the surgical instruments and patient biological tissue; the surgical robot being used to move the surgical instruments; and adding corresponding markers for the surgical instruments and patient biological tissue on the doctor's control terminal based on different distance intervals corresponding to the distance between the surgical instruments and patient biological tissue.

Images