Physical medical element placement system

Abstract

An example system is configured to obtain anatomical property data representing a property associated with an anatomical surface to be covered by a physical medical element, the anatomical surface being within an interior space of a patient; and determine a set of placement guidance parameters based on the anatomical property data. The set of placement guidance parameters can include one or more parameters configured to guide placement of the physical medical element on the anatomical surface with one or more surgical instruments controlled by a computer-assisted surgical system.

Description

[0001] Related applications

[0002] This application claims priority to U.S. Provisional Patent Application No. 62 / 911,853, filed October 7, 2019, entitled “PHYSICAL MEDICAL ELEMENT PLACEMENT SYSTEMS AND METHODS”, the contents of which are incorporated herein by reference in their entirety. Background Technology

[0003] Computer-assisted surgical systems are commonly used to perform hernia repair procedures inside a patient's body. As part of the hernia repair procedure, a mesh patch can be placed on the hernia and attached (e.g., sutured) to the tissue surrounding the hernia. As the tissue heals, the mesh patch can provide support to the damaged tissue.

[0004] During a hernia repair procedure, the surgeon interacting with a computer-assisted surgical system must determine the appropriate size of the mesh patch. Once the mesh patch has been sized (e.g., by cutting the mesh patch from the mesh material), the surgeon must place it in the appropriate position within the patient's body. These procedures can be time-consuming and tedious. Summary of the Invention

[0005] The following description presents a simplified summary of one or more aspects of the systems and methods described herein. This summary is not a broad overview of all anticipated aspects and is neither intended to identify key or essential elements of all aspects nor to define the scope of any or all aspects. Its sole purpose is to demonstrate one or more aspects of the systems and methods described herein as a prelude to the specific embodiments shown below.

[0006] An exemplary system includes: a memory storing instructions; and a processor communicatively coupled to the memory and configured to execute the instructions to: obtain anatomical characteristic data representing characteristics associated with an anatomical surface to be covered by a physical medical element, the anatomical surface being within a patient's internal space; and determine a placement guidance parameter set based on the anatomical characteristic data, the placement guidance parameter set including one or more parameters configured to guide the placement of the physical medical element onto the anatomical surface using one or more surgical instruments controlled by a computer-assisted surgical system.

[0007] An exemplary system includes: a memory storing instructions; and a processor communicatively coupled to the memory and configured to execute the instructions to: obtain anatomical characteristic data representing characteristics associated with an anatomical surface to be covered by a physical medical element, the anatomical surface being within a patient's internal space; determine, based on the anatomical characteristic data, a suture plan for suturing the physical medical element to the anatomical surface when one or more surgical instruments controlled by a computer-assisted surgical system hold the physical medical element in proper position on the anatomical surface; and graphically indicate the suture plan within an image of the internal space.

[0008] An exemplary method includes: obtaining anatomical characteristic data via a medical component management system, the anatomical characteristic data representing characteristics associated with an anatomical surface to be covered by a physical medical component, the anatomical surface being within a patient's internal space; and determining a placement guidance parameter set based on the anatomical characteristic data via the medical component management system, the placement guidance parameter set including one or more parameters configured to guide the placement of the physical medical component onto the anatomical surface using one or more surgical instruments controlled by a computer-assisted surgical system.

[0009] An exemplary method includes: obtaining anatomical characteristic data, representing characteristics associated with an anatomical surface to be covered by a physical medical element, within a patient's internal space, via a medical element management system; determining, based on the anatomical characteristic data, a suture plan for suturing the physical medical element to the anatomical surface when one or more surgical instruments controlled by a computer-assisted surgical system hold the physical medical element in an appropriate position on the anatomical surface; and graphically indicating the suture plan within an image of the internal space via the medical element management system.

[0010] An exemplary non-transitory computer-readable medium storage instruction, which, when executed, directs a processor of a computing device to: obtain anatomical characteristic data representing characteristics associated with an anatomical surface to be covered by a physical medical element, the anatomical surface being within a patient's internal space; and determine a placement guidance parameter set based on the anatomical characteristic data, the placement guidance parameter set including one or more parameters configured to guide the placement of the physical medical element onto the anatomical surface using one or more surgical instruments controlled by a computer-assisted surgical system.

[0011] An exemplary non-transitory computer-readable medium stores instructions that, when executed, instruct a processor of a computing device to: obtain anatomical characteristic data representing characteristics associated with an anatomical surface to be covered by a physical medical element, the anatomical surface being within a patient's internal space; determine, based on the anatomical characteristic data, a suturing plan for suturing the physical medical element to the anatomical surface when one or more surgical instruments controlled by a computer-assisted surgical system hold the physical medical element in proper position on the anatomical surface; and graphically indicate the suturing plan within an image of the internal space. Attached Figure Description

[0012] The accompanying drawings illustrate various embodiments and are part of the specification. The illustrated embodiments are merely examples and do not limit the scope of this disclosure. Throughout the drawings, the same or similar reference numerals designate the same or similar elements.

[0013] Figure 1 An exemplary medical component management system based on the principles described herein is illustrated.

[0014] Figure 2 An exemplary configuration based on the principles described herein is illustrated, in which the system is configured to identify a target region within an image acquired by an imaging device, and the target region delineates an anatomical surface to be covered by a physical medical element.

[0015] Figure 3 An exemplary configuration based on the principles described herein is illustrated, in which the imaging device includes a visible light camera and a depth sensor.

[0016] Figure 4 An exemplary configuration based on the principles described herein is illustrated, in which the depth sensor is implemented by a visible light camera.

[0017] Figure 5 An exemplary configuration based on the principles described herein is shown, in which the system obtains depth data from a preoperative image source.

[0018] Figure 6 An exemplary image that can be captured by an imaging device according to the principles described herein is shown.

[0019] Figures 7-9 An exemplary virtual medical element is shown, presented on top of an identified target area within an image, in accordance with the principles described herein.

[0020] Figures 10-11 An exemplary configuration based on the principles described herein is illustrated, in which the system generates physical dimension data.

[0021] Figure 12 An exemplary physical material is shown, from which a physical medical component is cut, according to the principles described herein.

[0022] Figures 13-14 An exemplary configuration based on the principles described herein is illustrated, in which the system determines where to place the boot parameter set.

[0023] Figures 15-16 An exemplary scenario is shown, based on the principles described herein, in which a physical medical element is placed on an anatomical surface that includes tissue defects and a portion of defect-free tissue.

[0024] Figures 17-20 An exemplary method based on the principles described herein is illustrated.

[0025] Figure 21 An exemplary computer-assisted surgical system based on the principles described herein is illustrated.

[0026] Figure 22 An exemplary computing device based on the principles described herein is illustrated. Detailed Implementation

[0027] This document describes a physical medical device positioning system and method. As described herein, an exemplary medical device management system can obtain anatomical characteristic data representing characteristics associated with an anatomical surface to be covered by a physical medical device (e.g., a mesh patch configured to cover a hernia), and determine a set of placement guidance parameters based on the anatomical characteristic data. As described herein, the set of placement guidance parameters may include one or more parameters configured to guide the placement of the physical medical device on the anatomical surface using one or more surgical instruments controlled by a computer-assisted surgical system.

[0028] The systems and methods described herein advantageously provide guidance during medical procedures when placing a physical medical element over an anatomical surface within the body. Such guidance assists the user in positioning the physical medical element on the anatomical surface and / or attaching (e.g., suturing) the physical medical element to the anatomical surface. In some alternative examples, as described herein, guidance can facilitate automated placement of the physical medical element over the anatomical surface. Furthermore, the systems and methods described herein can minimize the amount of time required to place the physical medical element over the anatomical surface, which can benefit both the patient and the surgical team involved in placing the physical medical element over the anatomical surface. Additionally or alternatively, the systems and methods described herein can minimize variations in the spacing between sutures used to place the physical medical element over the anatomical surface. Additionally or alternatively, the systems and methods described herein can take into account the three-dimensional contour of the anatomical surface (e.g., by using depth data) when providing guidance during the placement of the physical medical element over the anatomical surface.

[0029] These and other advantages and benefits of the systems and methods described in this paper will become apparent in this paper.

[0030] As used herein, a physical therapy element refers to any element independent of the patient's body that is configured to be placed on and cover an anatomical surface within the patient's body. For example, a physical therapy element may be implemented by a patch (e.g., a mesh patch) configured to cover a tissue defect (e.g., a hernia, incision, or other type of injury) within the patient's body. Other examples of physical therapy elements that can be used in conjunction with the systems and methods described herein include, but are not limited to, gauze, bandages, plates, prostheses, etc. Physical therapy elements can be placed on an anatomical surface in any suitable manner. For example, a physical therapy element may be sutured, anchored, or otherwise secured to an anatomical surface.

[0031] Figure 1An exemplary medical component management system 100 (“System 100”) configured to perform the various operations described herein is illustrated. As shown, System 100 may include, but is not limited to, storage facility 102 and processing facility 104 selectively and communicatively coupled to each other. Facilities 102 and 104 may each include or be implemented with hardware and / or software components (e.g., processor, memory, communication interface, instructions stored in memory for execution by the processor, etc.). For example, facilities 102 and / or 104 may be implemented by any component of a computer-assisted surgical system configured to perform a medical procedure in which a physical medical component is introduced into and placed on an anatomical surface within the patient's body. As another example, facilities 102 and / or 104 may be implemented by a computing device separate from and communicatively coupled to the computer-assisted surgical system. In some examples, facilities 102 and 104 may be distributed among multiple devices and / or multiple locations, as may be for a particular implementation.

[0032] Storage facility 102 may maintain (e.g., store) executable data that is used by processing facility 104 to perform one or more of the operations described herein. For example, storage facility 102 may store instructions 106 that can be executed by processing facility 104 to perform one or more of the operations described herein. Instructions 106 may be implemented by any suitable application, software, code, and / or other instance of executable data. Storage facility 102 may also maintain any data received, generated, managed, used, and / or transmitted by processing facility 104.

[0033] Processing facility 104 can be configured to perform (e.g., execute instructions 106 stored in storage facility 102 to perform) the various operations described herein.

[0034] For example, processing facility 104 may be configured to: instruct a display device to present a virtual medical element representing a physical medical element over a target area within an image of the patient's internal space, the target area depicting the anatomical surface to be covered by the physical medical element; receive user input while the virtual medical element is presented over the target area, the user input setting at least one of the pose and size of the virtual medical element within the image; and determine the physical dimensions of the physical medical element based on the user input and depth data of a depth map representing the internal space. As described herein, physical dimensions may define the surface area of ​​the physical medical element.

[0035] As another example, processing facility 104 may be configured to: access image data representing an image acquired by an imaging device and depicting the patient's internal space; obtain depth data representing a depth map of the internal space depicted in the image acquired by the imaging device; identify a target area within the image based on the image data and the depth data, the target area depicting an anatomical surface to be covered by a physical medical element; and instruct a display device to present a virtual medical element representing the physical medical element over the identified target area within the image.

[0036] As another example, processing facility 104 may be configured to: acquire anatomical characteristic data representing characteristics associated with an anatomical surface to be covered by a physical medical element, the anatomical surface being within the patient's internal space; and determine a set of placement guidance parameters based on the anatomical characteristic data. As described herein, the set of placement guidance parameters may include one or more parameters configured to guide the placement of the physical medical element onto the anatomical surface using one or more surgical instruments controlled by a computer-assisted surgical system.

[0037] As another example, the treatment facility 104 may be configured to: acquire anatomical characteristic data representing characteristics associated with an anatomical surface to be covered by a physical medical element, the anatomical surface being within the patient's internal space; determine, based on the anatomical characteristic data, a suture plan (or any other type of fixation plan) for suturing the physical medical element to the anatomical surface when one or more surgical instruments controlled by a computer-assisted surgical system hold the physical medical element in the appropriate position on the anatomical surface; and graphically indicate the suture plan within an image of the internal space.

[0038] This document describes these and other operations that can be performed by system 100 (e.g., processing facility 104).

[0039] Figure 2 An exemplary configuration 200 is illustrated, in which system 100 is configured to identify a target region within an image acquired by an imaging device, and the target region depicts an anatomical surface to be covered by a physical medical element. As shown, system 100 can access image data 202, which represents an image acquired by the imaging device and depicting the internal space of a patient. System 100 can also obtain depth data 204, which represents a depth map of the internal space depicted in the image acquired by the imaging device. Based on image data 202 and depth data 204, system 100 can identify a target region within the image depicting the anatomical surface to be covered by a physical medical element, and output target region data 206 representing the identified target region.

[0040] The target area data 202 can be in any suitable format. For example, the target area data 202 can include two-dimensional or three-dimensional pixel coordinates that represent pixels depicting the anatomical surface to be covered by the physical medical element.

[0041] Exemplary methods for generating image data 202 and depth data 204 will now be described.

[0042] Figure 3 An exemplary configuration 300 is illustrated, in which an imaging device 302 includes a visible light camera 304 configured to generate and output image data 202 and a depth sensor 306 configured to generate and output depth data 204.

[0043] Imaging device 302 may be implemented by an endoscope or other camera device configured to capture images of the scene. In some examples, imaging device 302 may be configured to be attached to and controlled by a computer-assisted surgical system. In alternative examples, imaging device 302 may be handheld and manually operated by an operator (e.g., a surgeon).

[0044] In some examples, the scene captured by imaging device 302 may include a surgical area associated with the patient. In some examples, the surgical area may be entirely located within the patient and may include the area within or near where a surgical procedure is planned to be performed, is being performed, or has been performed within the patient. For example, for a minimally invasive surgical procedure being performed on tissue within the patient, the surgical area may include the tissue, the anatomical structures beneath the tissue, and the space surrounding the tissue, such as the location of surgical instruments used to perform the surgical procedure. In some example embodiments, a surgical area entirely located within the patient may be referred to as “internal space.” As described herein, any internal anatomical structures of the patient (e.g., blood vessels, organs, and / or tissues) and / or surgical instruments located within the internal space may be referred to as objects and / or structures.

[0045] Visible light camera 304 (“camera 304”) is configured to generate image data 202 representing a two-dimensional visible light image of a scene. Camera 304 can be implemented by any suitable image sensor, such as a charge-coupled device (“CCD”) image sensor, a complementary metal-oxide-semiconductor (“CMOS”) image sensor, a hyperspectral camera, a multispectral camera, etc.

[0046] Depth sensor 306 can be implemented by any suitable sensor configured to generate depth data 204. For example, depth sensor 306 can be implemented by a time-of-flight sensor, a structured light sensor, an interferometer, a hyperspectral camera, a multispectral camera, and / or any other suitable sensor configured to acquire depth data, as may be served in a particular implementation. In the case where depth sensor 306 is implemented by a time-of-flight sensor, the time-of-flight sensor can be implemented by one or more photodetectors (e.g., one or more single-photon avalanche diode (“SPAD”) detectors), a CCD sensor, a CMOS sensor, and / or any other suitable configuration. Figure 3 In the example, the depth sensor 306 is separate from the camera 304 (i.e., physically different).

[0047] In configuration 300, system 100 can obtain image data 202 by directing camera 304 to acquire image data 202 and receiving image data 202 from camera 304. Similarly, system 100 can obtain depth data 204 by directing depth sensor 306 to acquire depth data 204 and receiving depth data 204 from depth sensor 306.

[0048] Therefore, in configuration 300, system 100 is communicatively coupled to imaging device 302 via bidirectional communication link 308 and to illumination system 310 via communication link 312. Communication links 308 and 312 can each be implemented using any suitable wired and / or wireless communication medium, as may be suited to a particular implementation. As described herein, system 100 can use communication links 308 and 312 to direct camera 304 and depth sensor 306 to acquire and receive image data 202 and depth data 204.

[0049] The lighting system 310 can be configured to emit light 314 (e.g., in the direction of system 100) to illuminate the scene imaged by the imaging device 302. The light 314 emitted by the lighting system 310 may include visible and / or invisible light (e.g., infrared light). As shown, the light 314 can travel to the scene through the imaging device 302 (e.g., through an illumination channel within the imaging device 302, which may be implemented by one or more optical fibers, light guides, lenses, etc.).

[0050] As shown, light 314 emitted by illumination system 310 can be reflected from surface 316 within the scene imaged by imaging device 302. When imaging device 302 is aimed at the patient's internal space, surface 316 refers to a surface within the internal space (e.g., an anatomical surface).

[0051] Camera 304 and depth sensor 306 can each detect reflected light 314. Camera 304 can be configured to generate image data 202 based on the detected light, which represents a two-dimensional visible light image of the scene including surface 316. Depth sensor 306 can be configured to generate depth data 204 based on the detected light. Image data 202 and depth data 204 can each have any suitable format.

[0052] To generate a stereoscopic image of the scene, system 100 can direct lighting system 310 to emit light 314. System 100 can also activate (e.g., turn on) a visible light camera 304 and a depth sensor 306. Light 314 travels into the scene and is reflected from surface 316 (and, in some examples, one or more other surfaces in the scene). Both camera 304 and depth sensor 306 detect the reflected light 314.

[0053] Camera 304 (and / or other circuitry included in imaging device 302) can generate image data 202 representing a two-dimensional visible light image of the scene based on detected light 314. This can be performed in any suitable manner. Visible light camera 304 (and / or other circuitry included in imaging device 302) can transmit image data 202 to system 100. This can also be performed in any suitable manner.

[0054] Depth sensor 306 can generate depth data 204 representing a depth map of the scene (e.g., a depth map of surface 316) based on detected light 314. This can be performed in any suitable manner. For example, depth sensor 306 can measure the amount of time it takes for photons of light 314 to travel from illumination system 310 to depth sensor 306. Based on this amount of time, depth sensor 306 can determine the depth of surface 316 relative to the orientation of depth sensor 306. The data representing this depth can be represented in depth data 204 in any suitable manner. For example, the depth map represented by depth data 204 can include an array of depth values ​​(e.g., Z-buffer values) corresponding to each pixel in an image.

[0055] The depth sensor 306 (and / or other circuitry included in the imaging device 302) can transmit depth data 204 to the system 100. This can be done in any suitable manner.

[0056] System 100 can receive image data 202 and depth data 204 and perform one or more processing operations on the image data 202 and depth data 204. For example, as will be described in more detail below, system 100 can generate target area data 206 based on image data 202 and depth data 204.

[0057] As another example, system 100 can generate a right-side perspective image of the scene and a left-side perspective image representing the scene based on image data 202 and depth data 204. This can be performed in any suitable manner. System 100 can then instruct a display device to simultaneously (concurrently) display the right-side and left-side perspective images in a manner that forms a stereoscopic image of the scene. In some examples, the display device is included in and / or communicatively coupled to the computer-assisted surgical system 204.

[0058] Figure 4 An exemplary configuration 400 is illustrated, in which the depth sensor 402 is implemented by visible light cameras 304-1 and 304-2 included in the imaging device 302. In configuration 400, system 100 can obtain depth data 204 by acquiring a first image (e.g., a first two-dimensional image) of the patient's internal space via guide camera 304-1, acquiring a second image (e.g., a second two-dimensional image) of the patient's internal space via guide camera 304-2, and generating a depth map represented by depth data 204 based on the first and second images.

[0059] exist Figure 4 In the figure, the first image acquired by camera 304-1 is represented by image data 202-1, and the second image acquired by camera 304-2 is represented by image data 202-2. As shown, image data 202-1 and 202-2 are transmitted to a depth data generator 402 implemented by system 100. The depth data generator 402 can use any visible image-based technique to determine depth data 204 based on image data 202-1 and 202-2.

[0060] Other configurations of the imaging device 302 are possible based on the systems and methods described herein. For example, the imaging device 302 may include multiple cameras 304 and / or multiple depth sensors 306. For illustration purposes, the imaging device 302 may include two cameras 304 combined with a single depth sensor 306. In these embodiments, depth data may be generated based on images acquired by the two cameras 304. The depth data generated by the depth sensors 304 may be used to fine-tune or otherwise enhance the depth data generated based on the images acquired by the two cameras 304.

[0061] In some examples, system 100 can obtain depth data 204 by accessing preoperative images registered to images from sources different from imaging device 204. For example, Figure 5An exemplary configuration 500 is illustrated, in which system 100 obtains depth data 204 from a preoperative image source 502. Source 502 may be implemented by a computer-aided computed tomography (CT) scanner, magnetic resonance imaging (MRI) device, ultrasound device, three-dimensional scanning (LIDAR) device, and / or any other suitable alternative imaging device configured to generate preoperative images of the patient. The preoperative images may be registered with the images represented by image data 202 to provide depth data 204.

[0062] System 100 can identify target areas based on image data 202 and depth data 204 in any suitable manner. For example, based on image data 202 and depth data 204, system 100 can identify areas within an image represented by image data 20 that depict tissue that needs to be covered by a physical medical element, and then designate the identified areas as target areas.

[0063] System 100 can identify areas in an image that depict tissue that needs to be covered by physical medical components in any suitable manner. For example, based on image data 202 and depth data 204, system 100 can segment the image (e.g., by classifying different parts of the image into corresponding different types of tissue) and identify areas based on the segmentation.

[0064] Additionally or alternatively, system 100 can identify areas of tissue that need to be covered by physical medical components by inputting image data 202 and depth data 204 into a machine learning model configured to identify tissue abnormalities. The machine learning model can be trained and / or used in any suitable manner.

[0065] In some examples, system 100 can determine the stage within a surgical procedure being performed on a patient and further base the identification of areas depicting tissue that need to be covered by physical medical components on the determined stage. For example, system 100 can receive input from a surgeon attempting to locate tissue defects within a patient by scanning the patient's internal space with an imaging device. Based on this input, system 100 can initiate a region identification heuristic step that uses image data 202 and depth data 204 to automatically identify areas within the image acquired by the imaging device that depict tissue that needs to be covered by physical medical components.

[0066] Once the target area depicting the anatomical surface to be covered by the physical medical element is identified, the system 100 can instruct the display device to present a virtual medical element representing the physical medical element over the identified target area within the image. This can be done in any suitable manner.

[0067] Using diagrams, Figure 6An exemplary image 602 is shown, which can be captured by an imaging device targeting the patient's internal space and can be represented by image data 202. As shown, image 602 depicts a tissue defect 604 surrounded by defect-free tissue 606. Tissue defect 604 can be a hernia, incision, or other type of injury. Defect-free tissue 606 can include, for example, healthy tissue unaffected by tissue defect 604.

[0068] System 100 can identify a target region within image 602 based on image data 202 and / or depth data 204, which delineates the anatomical surface to be covered by a physical medical element. This can be performed in any of the manner described herein. In some examples, the anatomical surface to be covered by the physical medical element includes at least a portion of tissue defect 604. In some cases, the anatomical surface to be covered by the physical medical element may also include at least a portion of defect-free tissue 606. For example, the anatomical surface to be covered by the physical medical element may include the entire tissue defect 604 and an overlapping area consisting of defect-free tissue 606 surrounding the tissue defect 604. The overlapping area may have any suitable width (e.g., between one and five centimeters) and can be used to attach the physical medical element to the anatomical surface as described herein.

[0069] Tissue defect 604 and defect-free tissue 606 can have varying relative depths. For example, tissue defect 604 and the surrounding defect-free tissue can have various ridges, peaks, valleys, and / or other uneven surfaces. However, such depth variations may not be visually perceptible within image 602, especially if image 602 is two-dimensional. Therefore, as described herein, system 100 can take such depth variations into account when determining the physical dimensions of a physical medical element that will cover the overlapping area formed by tissue defect 604 and the surrounding defect-free tissue 606.

[0070] System 100 can instruct a display device to present a virtual medical element representing a physical medical element over an identified target area within image 602. For example, Figure 7 An exemplary virtual medical element 702 is shown presented above the identified target area within image 602. Figure 7 In the example, the virtual medical element 702 includes dashed lines and an area enclosed by the dashed lines. Furthermore, in... Figure 7 In the example, the identified target area directly corresponds to the virtual medical element 702 (i.e., is completely covered by the virtual medical element 702).

[0071] Although the virtual medical element 702 is illustrated as a dashed rectangle positioned over a portion of the tissue defect 604 and the defect-free tissue 606, it will be appreciated that the virtual medical element 702 may alternatively be presented in any other suitable manner. For example, the virtual medical element 702 may be at least partially transparent to allow the user to visualize the target area and / or the tissue defect 604 when the virtual medical element 702 is presented over the target area.

[0072] When the virtual medical element 702 is presented over the target area, the user can provide user input to set the posture and / or size of the virtual medical element 702. For example, if the user determines that the initially determined posture and size of the virtual medical element 702 are sufficient (e.g., if the virtual medical element 702 adequately covers the depiction of the tissue defect 604), the user can provide user input confirming that the posture and size of the virtual medical element 702 are correct. Such user input can be provided in any suitable manner. For example, such user input can be provided by the user selecting an option displayed in image 602, by the user selecting a user input button on a component of the computer-assisted surgical system, by the user providing a verbal command, and / or in any other manner.

[0073] In some cases, the user may determine that the initially determined pose and / or size of the virtual medical element 702 needs further refinement. For example, the user may expect to zoom in or out of the virtual medical element 702, reposition one or more edges of the virtual medical element 702, and / or otherwise adjust the pose and / or size of the virtual medical element 702. In these cases, user input for setting the pose and / or size of the virtual medical element 702 may include one or more user input commands for adjusting the pose and / or size of the virtual medical element 702. Such user input may be provided in any suitable manner. For example, user input may be provided through user interaction with a virtual handle displayed as part of the virtual medical element 702, through a graphical user interface including image 602, through one or more keyboard or other input device commands, and / or in any other suitable manner. In response to receiving user input adjusting the pose and / or size of the virtual medical element 702, system 100 may dynamically adjust the presentation of the virtual medical element 702 substantially in real time to depict at least one of the pose and size of the virtual medical element adjusted by the user input.

[0074] For illustration purposes, Figure 8 The image shows the virtual medical element 702 after the user has adjusted its shape (and therefore its size). As shown in the figure... Figure 8 The shape and size of the virtual medical element 702 shown are different from those of the virtual medical element 702 before it was adjusted by the user. Figure 7 It more closely matches the actual shape and size of tissue defect 604.

[0075] In some examples, the initial shape of the virtual medical element 702 is a standard size for a specific type of physical medical element. For example, for a physical medical element that is typically rectangular, the initial shape of the virtual medical element 702 could be rectangular, such as... Figure 7 As shown.

[0076] Alternatively, the initial shape of the virtual medical element 702 can more closely conform to the actual shape of the anatomical surface to be covered by the physical medical element. For example, Figure 9 An exemplary embodiment is shown, wherein the virtual medical element 702 has a contour shape that follows the outer edge of the tissue defect 604.

[0077] The initial shape of the virtual medical element 702 can be automatically determined based on one or more attributes of the physical medical element to be covered on the anatomical surface. Alternatively, the initial shape of the virtual medical element 702 can be specified by the user by providing one or more user input commands.

[0078] Based on user input regarding the posture and / or size of the virtual medical element 702, and depth data 204, system 100 can determine the physical dimensions of the physical medical element to cover the anatomical surface. For example, Figure 10 An exemplary configuration 1000 is illustrated, in which system 100 accesses virtual data 1002 and depth data 204 and generates physical dimension data 1004 based on the virtual data 1002 and depth data 204. Virtual data 1002 may represent the pose, size, and / or positioning of a virtual medical element 702 set by the user. As explained herein, depth data 204 represents a depth map of the interior space depicted in image 602. Physical dimension data 1004 represents the physical dimensions of the physical medical element determined by system 100. As described herein, physical dimensions may represent the surface area of ​​the physical medical element.

[0079] System 100 can generate physical size data 1004 based on virtual data 1002 and depth data 204 in any suitable manner. For example, system 100 can identify multiple pixels within image 602 covered by virtual medical element 702 based on the pose and size of the virtual medical element represented by virtual data 1002. System 100 can determine two-dimensional pixel regions of the multiple pixels. System 100 can determine the depth value of each of the multiple pixels covered by virtual medical element 702 based on depth data 204. Based on the two-dimensional pixel regions and depth values, system 100 can determine the surface area of ​​the anatomical surface to be covered by the physical medical element. System 100 can make the physical dimensions of the physical medical element based on the surface area of ​​the anatomical surface. Each of these operations can be performed in any suitable manner.

[0080] Due to the potential depth variations on an anatomical surface, the surface area of ​​a physical medical element, as defined by the physical dimensions of the virtual medical element 702, may in some cases be larger than the physical area represented by a two-dimensional pixel region. For example, the physical medical element may be made of a material configured to "shrink wrap" or otherwise adhere to all surface areas of the anatomical surface. In this example, if the anatomical surface has protruding ridges that need to be covered by the physical medical element, this depth variation may result in the surface area defined by the physical dimensions of the physical medical element determined by system 100 being larger than the physical area represented by a two-dimensional pixel region of the virtual medical element 702.

[0081] Figure 11 An exemplary configuration 1100 is illustrated in which, in addition to virtual data 1002 and depth data 204, system 100 also bases the generation of physical dimension data 1004 on deformation data 1102. Deformation data 1102 represents a deformation model of a physical medical component. The deformation model can indicate one or more physical properties of the physical medical component that affect its ability to deform (e.g., stretch, compress, etc.) in response to forces applied to it. For example, the deformation model can indicate the type of material used to manufacture the physical medical component, the tensile strength of the material, and / or any other measure representing deformation, as may be applicable to a particular implementation.

[0082] System 100 can determine the physical dimensions of a physical medical element based on deformation data 1102 in any suitable manner. For example, if deformation data 1102 indicates that a physical medical element is relatively tensile-resistant compared to different types of physical medical elements that exhibit high tensile strength, system 100 can specify that the physical dimensions should be relatively small.

[0083] System 100 may acquire deformation data 1102 in any suitable manner. For example, system 100 may maintain or access a database including deformation data 1102 for various types of physical medical components. System 100 may automatically determine which physical medical component to use by receiving user input indicating which physical medical component to use, by tracking trends based on a specific user and / or by determining which specific physical medical component to use to cover the anatomical surface in any other suitable manner.

[0084] System 100 can output data representing physical dimensions in any suitable manner. For example, system 100 can output data representing physical dimensions by displaying the physical dimensions within image 602 or within any other suitable user interface. Alternatively, system 100 can output data representing physical dimensions by outputting physical measurements of the physical medical element (e.g., width and length in centimeters). Alternatively or additionally, system 100 can output data representing physical dimensions by outputting data representing a pattern to be used to cut the physical medical element from the material.

[0085] In some examples, system 100 can project a virtual medical element 702 onto a physical material from which the physical medical element will be cut. In this way, the virtual medical element 702 can guide the user to cut the physical medical element from the physical material.

[0086] For illustration purposes, Figure 12 An exemplary physical material 1202 is shown, from which a physical medical component will be cut. Figure 12 In the example, physical material 1202 is a mesh material from which patches for hernias and / or other types of tissue defects can be cut. As shown, system 100 has projected virtual medical element 702 onto physical material 1202. Virtual medical element 702 can be projected onto physical material 1202 in any suitable manner.

[0087] Once ready to introduce a physical medical device into a patient, the system 100 can be configured to provide placement guidance, which is configured to guide and / or assist the user in placing the physical medical device on an anatomical surface. This placement guidance can be provided in a variety of different ways.

[0088] For example, Figure 13An exemplary configuration 1300 is illustrated, in which system 100: obtains anatomical characteristic data 1302, which represents characteristics associated with an anatomical surface to be covered by a physical medical element; and determines a placement guidance parameter set 1304 based on the anatomical characteristic data 1302. The placement guidance parameter set 1304 includes one or more parameters configured to guide the placement of the physical medical element on the anatomical surface using one or more surgical instruments controlled by a computer-assisted surgical system.

[0089] Anatomical characteristic data 1302 may represent one or more characteristics associated with the anatomical surface to be covered by the physical medical element. For example, anatomical characteristic data 1302 may represent the tissue type constituting the anatomical surface, the size of the anatomical surface, and / or the location of the anatomical surface within the patient's body. These characteristics can influence how the physical medical element is placed on the anatomical surface. For example, if the anatomical surface is near an organ, system 100 may generate one or more parameters included in placement guidance parameter set 1304 that indicate the proximity of the anatomical surface to the organ and can be used to identify a suture plan for attaching the physical medical element to the anatomical surface in a manner that does not damage or otherwise affect the organ. As another example, if the tissue constituting the anatomical surface is relatively weak, system 100 may generate one or more parameters included in placement guidance parameter set 1304 that can be used to increase the relative number of sutures used to attach the physical medical element to the anatomical surface.

[0090] System 100 can acquire anatomical characteristic data 1302 in any suitable manner. For example, system 100 can acquire anatomical characteristic data 1302 by accessing image data 202, acquiring depth data 204, and determining anatomical characteristic data 1302 based on image data 202 and depth data 204. For illustration purposes, image data 202 and depth data 204 can be used to determine the location, size, and / or any other characteristics of the anatomical surface.

[0091] In some examples, system 100 may use image data 202 and depth data 204 to segment an image represented by image data 202. Segmentation may include classifying different parts of the image into categories corresponding to different items (e.g., tissue type). Based on the segmentation, system 100 may determine one or more characteristics of an anatomical surface, such as tissue type.

[0092] Figure 14An exemplary configuration 1400 is illustrated, in which system 100 obtains component characteristic data 1402 in addition to anatomical characteristic data 1302. As shown, in configuration 1400, system 100 generates a placement guidance parameter set 1304 based on both anatomical characteristic data 1302 and component characteristic data 1402. In an alternative example, system 100 generates the placement guidance parameter set 1304 based only on component characteristic data 1402 (and not on anatomical characteristic data 1302).

[0093] Component characteristic data 1402 represents one or more characteristics associated with a physical medical component that will cover an anatomical surface. For example, component characteristic data 1402 may represent the physical dimensions of the physical medical component, the type of material used for the physical medical component, the tensile strength of the physical medical component, and / or the deformation characteristics of the physical medical component.

[0094] System 100 can access component characteristic data 1402 in any suitable manner. For example, system 100 can maintain or access a database that includes component characteristic data 1402. As another example, system 100 can access component characteristic data 1402 by receiving user input indicating access to component characteristic data 1402.

[0095] In some examples, the placement guidance parameter set 1304 includes one or more parameters configured to specify a preferred orientation of the physical medical element when it is being placed on an anatomical surface using one or more surgical instruments controlled by a computer-assisted surgical system. In these examples, the system 100 may indicate the preferred orientation of the physical medical element to the user in any suitable manner.

[0096] For example, system 100 can be configured to indicate the preferred orientation of a physical medical component to a user by graphically indicating the preferred orientation within an image of the interior space (e.g., image 602). For illustration purposes, Figure 15 An exemplary scenario is shown in which a physical medical element 1502 is placed on an anatomical surface comprising a portion of tissue defect 604 and defect-free tissue 606.

[0097] As shown in the figure, a physical medical element 1502 is placed on an anatomical surface by surgical instruments 1504-1 and 1504-2 (“surgical instruments 1504”). Surgical instruments 1504 may include any suitable grasping tool configured to hold and guide the physical medical element 1502 into place. In some examples, surgical instruments 1504 are controlled by a computer-aided surgical system (e.g., in response to user input commands provided by a surgeon or other user).

[0098] As shown in the figure, when a user is placing a physical medical element 1502 on an anatomical surface using a surgical instrument 1504, the system 100 can present a virtual medical element 702 within image 602. In this configuration, the virtual medical element 702 graphically indicates the preferred orientation of the physical medical element. Furthermore, graphical arrows 1506-1 and 1506-2 can be presented by the system 100 within image 602 to indicate the direction the user should rotate the physical medical element 1502 to achieve the preferred orientation. The system 100 may additionally or alternatively display any other suitable virtual, auditory, and / or graphical aids indicating the preferred orientation of the physical medical element 1502, as may be available for a particular implementation.

[0099] In some examples, the placement guide parameter set 1304 includes one or more parameters configured to specify a preferred insertion path within an internal space that the surgical instrument 1504 should follow when bringing the physical medical element 1502 from outside the patient's body into contact with an anatomical surface. In these examples, system 100 may indicate the preferred insertion path of the physical medical element 1502 to the user in any suitable manner. For example, system 100 may be configured to indicate the preferred insertion path of the physical medical element by graphically indicating the preferred insertion path within an image of the internal space (e.g., image 602). System 100 may additionally or alternatively display any other suitable virtual, auditory, and / or graphical aids indicating the preferred insertion path, as may be available for a particular implementation.

[0100] A preferred insertion path can be defined by one or more parameters in the placement guide parameter set 1304, such that when the physical medical element 1502 is brought from outside the patient to contact the anatomical surface, the surgical instrument 1504 and / or the physical medical element 1502 avoid collisions with tissues and / or other objects (e.g., other surgical instruments) in the internal space.

[0101] In some examples, the placement guide parameter set 1304 includes one or more parameters configured to specify a suture plan for suturing the physical medical element 1502 to the anatomical surface as the surgical instrument 1504 holds the physical medical element 1502 in proper position on the anatomical surface. The suture plan may include information specifying the suture pattern for suturing the physical medical element 1502 to the anatomical surface, the spacing to be used between the sutures used to suturing the physical medical element 1502 to the anatomical surface, the type of thread used to suturing the physical medical element 1502 to the anatomical surface, the required length of thread for suturing the physical medical element 1502 to the anatomical surface, and / or any other aspect of the suture plan, as may be applicable to a particular implementation. While exemplary suture plans have been described herein, it will be appreciated that the placement guide parameter set 1304 may alternatively include one or more parameters configured to specify any other type of fixation plan that may be used to secure the physical medical element 1502 to the anatomical surface.

[0102] For example, an exemplary suture plan that can be specified by one or more parameters in the placement guide parameter set 1304 can indicate a recommended suture pattern, suture spacing, suture type, and / or suture length for a specific type of tissue as indicated in anatomical characteristic data 1302, a specific type and / or size of tissue defect 604 as indicated in anatomical characteristic data 1302, the proximity of other objects (e.g., organs) to tissue defect 604 as indicated in anatomical characteristic data 1302, and / or any other characteristics of the anatomical surface as indicated in anatomical characteristic data 1302. Additionally or alternatively, an exemplary suture plan that can be specified by one or more parameters in the placement guide parameter set 1304 can indicate a recommended suture pattern, suture spacing, suture type, and / or suture length for one or more characteristics of the physical medical component 1502 as indicated in component characteristic data 1402.

[0103] In some examples, system 100 can be configured to graphically indicate the suture plan within image 602. For example, Figure 16A physical medical element 1502 is shown before suturing to the anatomical surface. System 100 can graphically indicate the suturing plan within image 602 by displaying virtual markers (e.g., virtual markers 1602-1 to 1602-5) that indicate the locations where sutures will be placed to attach the physical medical element 1502 to the anatomical surface. The orientation and spacing of each virtual marker 1602 can be determined based on anatomical characteristic data 1302 and / or element characteristic data 1402. For example, most virtual markers 1602 are positioned at approximately the same distance from the outer edge of the physical medical element 1502. However, in this example, virtual marker 1602-2 is offset relative to its adjacent virtual markers 1602-1 and 1602-3 (i.e., virtual marker 1602-2 is farther from the edge of the physical medical element 1502 than virtual markers 1602-1 and 1602-3). This may be because the anatomical characteristic data 1302 may indicate that the tissue near the location corresponding to the virtual mark 1602-2 on the physical medical element 1502 is relatively weak, thus requiring a larger gap between the suture location and the edge of the physical medical element 1502.

[0104] In some examples, a user may provide user input configured to modify a suture plan graphically indicated within image 602. For example, a user may provide input to adjust the orientation of one or more virtual markers 1602, remove a specific virtual marker 1602, add a new virtual marker, and / or otherwise modify the suture plan. In response to this user input, system 100 may dynamically adjust the suture plan based on the user's repositioning of a specific virtual marker 1602. For example, based on the user's repositioning of a specific virtual marker 1602, system 100 may update the amount of suture required to perform the suture, adjust the positioning of the suture device and / or one or more other surgical instruments, and / or otherwise adjust the suture plan.

[0105] Figure 16 Also shown is a suturing device 1604 positioned on the physical medical element 1502 at locations corresponding to specific virtual markers 1602-4. The suturing device 1604 can be configured to suture the physical medical element 1502 to an anatomical surface in any suitable manner. For example, the suturing device 1604 can apply running sutures (or any other type of suture) around the periphery of the physical medical element 1502 at various locations corresponding to the virtual markers 1602.

[0106] In some examples, the suture device 1604 is controlled by a computer-assisted surgical system (e.g., via a manipulator arm connected to the computer-assisted surgical system). In these examples, the suture device 1604 may be referred to as a type of surgical instrument coupled to and controlled by the computer-assisted surgical system. In alternative examples, the suture device 1604 is not controlled by a computer-assisted surgical system. In these alternative examples, the suture device 1604 may be manually held and / or otherwise controlled by a user.

[0107] When the suture device 1604 is controlled by a computer-assisted surgical system, the positioning and / or operation of the suture device 1604 can be set in response to user input (e.g., the user can provide input commands to move and / or operate the suture device 1604). For example, the user can provide input commands that instruct the computer-assisted surgical system to move the suture device 1604 from a suture position to a suture location, as guided by a suture plan indicated by virtual markers 1602. For example, after the suture device 1604 has been used to suture the physical medical element 1502 to the anatomical surface at a suture location corresponding to virtual markers 1602-4, the user can provide input commands (e.g., by manipulating the master controls, which are part of the computer-assisted surgical system) that cause the suture device 1604 to move to a suture location corresponding to virtual markers 1602-5. Once at that location, the suture device 1604 can be used to suture the physical medical element 1502 to the anatomical surface at the suture location corresponding to virtual markers 1602-5. This repositioning of the suture device 1604 can be performed automatically by a computer-aided surgical system, without requiring specific user input to control the positioning and / or operation of the suture device 1604.

[0108] In some examples, system 100 can guide a computer-assisted surgical system to automatically place a physical medical element (e.g., physical medical element 1502) onto an anatomical surface using one or more surgical instruments (e.g., surgical instrument 1504 and / or suture device 1604) based on a placement guidance parameter set 1304, without requiring user input to control the movement of one or more surgical instruments as they place the physical medical element onto the anatomical surface. For example, system 100 can guide the computer-assisted surgical system to use surgical instrument 1504 to guide the physical medical element 1502 to the correct orientation and positioning above the anatomical surface. System 100 can then guide the computer-assisted surgical system to automatically suture the physical medical element 1502 to the anatomical surface using suture device 1604.

[0109] In some examples, system 100 may track the relative posture of suturing device 1604 and / or surgical instrument 1504 with respect to physical medical element 1502. System 100 may use the tracked posture to guide suturing device 1604 to properly perform suturing of physical medical element 1502 to anatomical surfaces and / or to guide surgical instrument 1504 to properly grasp and hold physical medical element 1502.

[0110] Figure 17 An exemplary method 1700 that can be performed by a medical component management system (e.g., system 100 and / or any implementation thereof) is illustrated. Although Figure 17 The illustration shows an exemplary operation according to one embodiment, but other embodiments may omit, add, reorder, and / or modify it. Figure 17 Any of the operations shown.

[0111] In operation 1702, the medical component management system instructs the display device to present a virtual medical component representing a physical medical component over a target area within an image of the patient's internal space. This target area depicts the anatomical surface to be covered by the physical medical component. Operation 1702 can be performed in any of the manner described herein.

[0112] In operation 1704, the medical component management system receives user input when a virtual medical component is presented over a target area. This user input sets at least one of the pose and size of the virtual medical component within the image. Operation 1704 can be performed in any of the manner described herein.

[0113] In operation 1706, the medical component management system determines the physical dimensions of the physical medical component based on user input and depth data from a depth map representing the internal space. These physical dimensions define the surface area of ​​the physical medical component. Operation 1706 can be performed in any of the manner described herein.

[0114] Figure 18 Another exemplary method 1800 that can be performed by a medical component management system (e.g., system 100 and / or any implementation thereof) is illustrated. Although Figure 18 The illustration shows an exemplary operation according to one embodiment, but other embodiments may omit, add, reorder, and / or modify it. Figure 18 Any of the operations shown.

[0115] In operation 1802, the medical component management system accesses image data representing images acquired by the imaging device and depicting the patient's internal space. Operation 1802 can be performed in any of the manner described herein.

[0116] In operation 1804, the medical component management system acquires depth data, which represents a depth map of the interior space depicted in an image acquired by the imaging device. Operation 1804 can be performed in any of the manner described herein.

[0117] In operation 1806, the medical component management system identifies a target region within an image based on image data and depth data. This target region delineates the anatomical surface to be covered by the physical medical component. Operation 1806 can be performed in any of the manner described herein.

[0118] In operation 1808, the medical component management system instructs the display device to present a virtual medical component representing the physical medical component over the identified target area within the image. Operation 1808 can be performed in any of the manner described herein.

[0119] Figure 19 Another exemplary method 1900 that can be performed by a medical component management system (e.g., system 100 and / or any implementation thereof) is illustrated. Although Figure 19 The illustration shows an exemplary operation according to one embodiment, but other embodiments may omit, add, reorder, and / or modify it. Figure 19 Any of the operations shown.

[0120] In operation 1902, the medical device management system obtains anatomical characteristic data representing the characteristics associated with the anatomical surface to be covered by the physical medical device, which is located within the patient's internal space. Operation 1902 can be performed in any of the manner described herein.

[0121] In operation 1904, the medical component management system determines a placement guidance parameter set based on anatomical characteristic data. This placement guidance parameter set includes one or more parameters configured to guide the placement of a physical medical component onto an anatomical surface using one or more surgical instruments controlled by a computer-assisted surgical system. Operation 1904 can be performed in any of the manner described herein.

[0122] Figure 20 Another exemplary method 2000 that can be performed by a medical component management system (e.g., system 100 and / or any implementation thereof) is illustrated. Although Figure 20 The illustration shows an exemplary operation according to one embodiment, but other embodiments may omit, add, reorder, and / or modify it. Figure 20 Any of the operations shown.

[0123] In operation 2002, the medical device management system obtains anatomical characteristic data representing the characteristics associated with the anatomical surface to be covered by the physical medical device, which is located within the patient's internal space. Operation 2002 can be performed in any of the manner described herein.

[0124] In Operation 2004, the medical element management system determines a suturing plan based on anatomical characteristic data for suturing the physical medical element to the anatomical surface when one or more surgical instruments controlled by a computer-assisted surgical system hold the physical medical element in the appropriate position on the anatomical surface. Operation 2004 can be performed in any of the manner described herein.

[0125] In Operation 2006, the medical component management system graphically indicates the suture plan within an image of the interior space. Operation 2006 can be performed in any of the methods described herein.

[0126] The systems and methods described herein can be used and / or implemented by computer-aided surgical systems for performing surgical procedures on patients. Figure 21 An exemplary computer-assisted surgical system 2100 (“surgical system 2100”) is illustrated. As shown, the surgical system 2100 may include a control system 2102, a user control system 2104, and an auxiliary system 2106 that are communicatively coupled to each other. A surgical team may utilize the surgical system 2100 to perform computer-assisted surgical procedures on a patient 2108. As shown, the surgical team may include a surgeon 2110-1, an assistant 2110-2, a nurse 2110-3, and an anesthesiologist 2110-4, all of whom may be collectively referred to as “surgical team members 2110”. Additional or alternative surgical team members may be presented during a surgical session, as may be necessary for a particular implementation.

[0127] although Figure 21 The illustration depicts a minimally invasive surgical procedure in progress; however, it should be understood that the surgical system 2100 can be similarly used to perform open surgical procedures or other types of surgical procedures that can similarly benefit from the accuracy and convenience of the surgical system 2100. Additionally, it should be understood that a surgical session in which the surgical system 2100 can be used throughout can include not only the surgical phase of a surgical procedure, such as... Figure 21 As shown, it may also include other appropriate stages of the preoperative, postoperative, and / or surgical procedure. The surgical procedure may include any procedure involving the use of manual and / or instrumental techniques to investigate or treat the patient's physical condition.

[0128] like Figure 21As shown, the manipulation system 2102 may include a plurality of manipulator arms 2112 (e.g., manipulator arms 2112-1 to 2112-4), to which a plurality of surgical instruments may be coupled. Each surgical instrument may be implemented by any suitable surgical tool (e.g., a tool with tissue interaction capabilities), medical tool, imaging device (e.g., an endoscope), sensing device (e.g., a force-sensing surgical instrument), diagnostic instrument, and the like, which may be used to perform computer-assisted surgical procedures on patient 2108 (e.g., by at least partially inserting into patient 2108 and being manipulated to perform computer-assisted surgical procedures on patient 2108). Although the manipulation system 2102 is depicted and described herein as comprising four manipulator arms 2112, it will be appreciated that the manipulation system 2102 may include only a single manipulator arm 2112 or any other number of manipulator arms, as may be served in a particular implementation.

[0129] The manipulator arm 2112 and / or the surgical instruments attached to the manipulator arm 2112 may include one or more displacement transducers, orientation sensors, and / or position sensors for generating raw (i.e., uncorrected) kinematic information. One or more components of the surgical system 2100 may be configured to use kinematic information to track surgical instruments (e.g., determine the position of surgical instruments) and / or control surgical instruments.

[0130] User control system 2104 can be configured to facilitate surgeon 2110-1's control of manipulator arm 2112 and surgical instruments attached to manipulator arm 2112. For example, surgeon 2110-1 can interact with user control system 2104 to remotely move or manipulate manipulator arm 2112 and surgical instruments. To this end, user control system 2104 can provide surgeon 2110-1 with images (e.g., high-resolution 3D images) of the surgical area associated with patient 2108 captured by an imaging system (e.g., any medical imaging system described herein). In some examples, user control system 2104 may include a stereoscopic viewer with two displays, where surgeon 2110-1 can view a stereoscopic image of the surgical area associated with patient 2108 and generated by a stereoscopic imaging system. Surgeon 2110-1 can use the images to perform one or more procedures with one or more surgical instruments attached to manipulator arm 2112.

[0131] To facilitate control of surgical instruments, the user control system 2104 may include a set of master controls. These master controls can be manipulated by the surgeon 2110-1 to control the movement of the surgical instruments (e.g., by utilizing robotic and / or teleoperation technologies). The master controls can be configured to detect various hand, wrist, and finger movements of the surgeon 2110-1. In this way, the surgeon 2110-1 can intuitively perform procedures using one or more surgical instruments. In some examples, the user control system 2104 implements user control system 806.

[0132] The auxiliary system 2106 may include one or more computing devices configured to perform the primary processing operations of the surgical system 2100. In such a configuration, the one or more computing devices included in the auxiliary system 2106 may control and / or coordinate operations performed by various other components of the surgical system 2100 (e.g., the manipulation system 2102 and the user control system 2104). For example, the computing device included in the user control system 2104 may transmit instructions to the manipulation system 2102 via one or more computing devices included in the auxiliary system 2106. As another example, the auxiliary system 2106 may receive and process image data representing images captured by an imaging device attached to one of the manipulator arms 2112 from the manipulation system 2102.

[0133] In some examples, the assistive system 2106 may be configured to present visual content to surgical team members 2110 who may not have access to the images provided to surgeon 2110-1 at the user control system 2104. For this purpose, the assistive system 2106 may include a display monitor 2114 configured to display one or more user interfaces (such as images of the surgical area (e.g., 2D images, 3D images)), information associated with patient 2108 and / or surgical procedures, and / or any other visual content that may be available for a particular implementation. For example, the display monitor 2114 may display an image of the surgical area along with additional content displayed concurrently with the image (e.g., graphical content, contextual information, etc.). In some embodiments, the display monitor 2114 is implemented as a touchscreen display, and surgical team members 2110 may interact with the touchscreen display (e.g., via touch gestures) to provide user input to the surgical system 2100.

[0134] The operating system 2102, the user control system 2104, and the auxiliary system 2106 can be communicatively coupled to each other in any suitable manner. For example, such as Figure 21As shown, the operating system 2102, user control system 2104, and auxiliary system 2106 can be communicatively coupled via control line 2116, which can represent any wired or wireless communication link, as may be required for a particular implementation. Therefore, the operating system 2102, user control system 2104, and auxiliary system 2106 can each include one or more wired or wireless communication interfaces, such as one or more local area network interfaces, Wi-Fi network interfaces, cellular interfaces, etc.

[0135] In some examples, a non-transitory computer-readable medium may be provided for storing computer-readable instructions, based on the principles described herein. When executed by a processor of a computing device, the instructions may direct the processor and / or the computing device to perform one or more operations, including one or more operations described herein. Such instructions may be stored and / or transmitted using any of a variety of known computer-readable media.

[0136] As used herein, a non-transitory computer-readable medium can include any non-transitory storage medium that contributes to providing data (e.g., instructions) that can be read and / or executed by a computing device (e.g., by the processor of the computing device). For example, a non-transitory computer-readable medium can include, but is not limited to, any combination of non-volatile storage media and / or volatile storage media. Exemplary non-volatile storage media include, but are not limited to, read-only memory, flash memory, solid-state drives, magnetic storage devices (e.g., hard disks, floppy disks, magnetic tapes, etc.), ferroelectric random access memory (“RAM”), and optical discs (e.g., compact discs, digital video discs, Blu-ray discs, etc.). Exemplary volatile storage media include, but are not limited to, RAM (e.g., dynamic RAM).

[0137] Figure 22 An exemplary computing device 2200 is illustrated, which may be specifically configured to perform one or more processes described herein. Any systems, computing devices, and / or other components described herein may be implemented by computing device 2200.

[0138] like Figure 22 As shown, computing device 2200 may include communication interface 2202, processor 2204, storage device 2206, and input / output (“I / O”) module 2208, which are communicatively connected to each other via communication infrastructure 2210. Although Figure 22 An exemplary computing device 2200 is shown, but Figure 22 The components shown are not intended to be limiting. Additional or alternative components may be used in other embodiments. A more detailed description will now follow. Figure 22 Components of the computing device 2200 shown.

[0139] Communication interface 2202 can be configured to communicate with one or more computing devices. Examples of communication interface 2202 include, but are not limited to, wired network interfaces (such as network interface cards), wireless network interfaces (such as wireless network interface cards), modems, audio / video connections, and any other suitable interfaces.

[0140] Processor 2204 generally refers to any type or form of processing unit capable of processing data and / or interpreting, executing, and / or directing the execution of one or more of the instructions, procedures, and / or operations described herein. Processor 2204 may perform operations by executing computer-executable instructions 2212 (e.g., application programs, software, code, and / or other executable data instances) stored in storage device 2206.

[0141] Storage device 2206 may include one or more data storage media, devices, or configurations, and may take any type, form, and combination of data storage media and / or devices. For example, storage device 2206 may include, but is not limited to, any combination of non-volatile media and / or volatile media described herein. Electronic data (including the data described herein) may be stored temporarily and / or permanently in storage device 2206. For example, data representing computer-executable instructions 2212 configured to instruct processor 2204 to perform any of the operations described herein may be stored within storage device 2206. In some examples, data may be arranged in one or more databases residing within storage device 2206.

[0142] I / O module 2208 may include one or more I / O modules configured to receive user input and provide user output. I / O module 2208 may include any hardware, firmware, software, or a combination thereof that supports input and output capabilities. For example, I / O module 2208 may include hardware and / or software for capturing user input, including but not limited to a keyboard or keypad, a touchscreen component (e.g., a touchscreen display), a receiver (e.g., an RF or infrared receiver), a motion sensor, and / or one or more input buttons.

[0143] I / O module 2208 may include one or more devices for presenting output to a user, including but not limited to a graphics engine, a display (e.g., a screen), one or more output drivers (e.g., display drivers), one or more audio speakers, and one or more audio drivers. In some embodiments, I / O module 2208 is configured to provide graphical data to the display for presentation to the user. The graphical data may represent one or more graphical user interfaces and / or any other graphical content, as may be suited to a particular implementation.

[0144] In the foregoing description, various exemplary embodiments have been described with reference to the accompanying drawings. However, various modifications and alterations may be made thereto, and additional embodiments may be implemented, without departing from the scope of the invention as set forth in the appended claims. For example, certain features of one embodiment described herein may be combined with or substituted for features of another embodiment described herein. Accordingly, the specification and drawings are to be considered illustrative rather than restrictive.

Claims

1. A system comprising: a memory storing instructions; and a processor communicatively coupled to the memory and configured to execute the instructions to: obtain anatomical property data representing properties associated with an anatomical surface to be covered by a physical medical element, the anatomical surface being within an internal space of a patient; and determine a placement guidance parameter set based on the anatomical property data, the placement guidance parameter set comprising one or more parameters configured to specify a suturing plan for suturing the physical medical element to the anatomical surface when one or more surgical instruments controlled by a computer-assisted surgical system hold the physical medical element in place on the anatomical surface.

2. The system of claim 1, wherein: the processor is further configured to execute the instructions to obtain element property data representing properties associated with the physical medical element; and determining the placement guidance parameter set is further based on the element property data.

3. The system of claim 2, wherein the properties associated with the physical medical element represented by the element property data comprise at least one of a physical dimension of the physical medical element, a material type used by the physical medical element, a tensile strength of the physical medical element, and a deformation property of the physical medical element.

4. The system of claim 1, wherein the properties associated with the anatomical surface to be covered by the physical medical element represented by the anatomical property data comprise at least one of a tissue type constituting the anatomical surface, a size of the anatomical surface, and a location of the anatomical surface within the patient.

5. The system of claim 1, wherein the placement guidance parameter set comprises one or more parameters configured to specify a preferred orientation of the physical medical element when the physical medical element is being placed on the anatomical surface with the one or more surgical instruments controlled by the computer-assisted surgical system.

6. The system of claim 5, wherein the processor is further configured to execute the instructions to graphically indicate the preferred orientation of the physical medical element when the physical medical element is being placed on the anatomical surface with the one or more surgical instruments controlled by the computer-assisted surgical system within an image of the internal space.

7. The system of claim 1, wherein the placement guidance parameter set comprises one or more parameters configured to specify a preferred insertion path within the internal space to be followed by the one or more surgical instruments when bringing the physical medical element into contact with the anatomical surface from outside of the patient.

8. The system of claim 7, wherein the processor is further configured to execute the instructions to graphically indicate, within the image of the internal space, the preferred insertion path within the internal space to be followed by the one or more surgical instruments when bringing the physical medical element into contact with the anatomical surface from outside the patient’s body.

9. The system of claim 1, wherein the processor is further configured to execute the instructions to graphically indicate, within the image of the internal space, the suturing plan.

10. The system of claim 1, wherein the suturing plan comprises information of at least one of a suturing pattern to be used for suturing the physical medical element to the anatomical surface, a spacing to be used between suturing lines for suturing the physical medical element to the anatomical surface, a type of line to be used for suturing the physical medical element to the anatomical surface, and a length of line required for suturing the physical medical element to the anatomical surface.

11. The system of claim 1, wherein the processor is further configured to execute the instructions to direct the computer-assisted surgical system to automatically place the physical medical element on the anatomical surface using the one or more surgical instruments in accordance with the placement guidance parameter set without requiring a user to provide user input that controls movement of the one or more surgical instruments when the one or more surgical instruments place the physical medical element on the anatomical surface.

12. The system of claim 1, wherein obtaining the anatomical property data comprises: accessing image data, the image data representing images acquired by an imaging device and depicting the internal space of the patient; obtaining depth data, the depth data representing a depth map of the internal space depicted in the images acquired by the imaging device; and determining the anatomical property data based on the image data and the depth data.

13. The system of claim 12, wherein determining the anatomical property data based on the image data and the depth data comprises: segmenting the images based on the image data and the depth data; and identifying the anatomical surface to be covered by the physical medical element based on the segmentation.

14. The system of claim 1, wherein the processor is further configured to execute the instructions to: direct a display device to present a virtual medical element representing the physical medical element over a target region within an image of the internal space of the patient, the target region depicting the anatomical surface to be covered by the physical medical element; receive user input when the virtual medical element is presented over the target region, the user input setting at least one of a pose of the virtual medical element within the image and a size of the virtual medical element; and determine, based on the user input and depth data representing a depth map of the internal space, a physical dimension of the physical medical element, the physical dimension defining a surface area of the physical medical element. ​ ​ wherein determining the placement guidance parameter set is further based on the physical dimension defining the surface area of the physical medical element.

15. The system of claim 14, wherein: the processor is further configured to execute the instructions to access deformation data representing a deformation model of the physical medical element; and determining the physical dimension of the physical medical element is further based on the deformation data.

16. The system of claim 14, wherein determining the physical dimension of the physical medical element comprises: identifying a plurality of pixels within the image that are covered by the virtual medical element based on the pose and the size of the virtual medical element; determining a two-dimensional pixel area of the plurality of pixels; determining a depth value for each pixel of the plurality of pixels based on the depth data; determining a surface area of the anatomical surface to be covered by the physical medical element based on the two-dimensional pixel area and the depth values; and causing the physical dimension to be based on the surface area of the anatomical surface.

17. The system of claim 16, wherein the surface area of the physical medical element defined by the physical dimension is greater than an area represented by the two-dimensional pixel area.

18. The system of claim 14, wherein a pixel area of the virtual medical element is substantially similar to a pixel area of the target region.

19. The system of claim 14, wherein the processor is further configured to execute the instructions to output data representing the physical dimension.

20. The system of claim 14, wherein the processor is further configured to execute the instructions to project the virtual medical element onto a physical material from which the physical medical element is to be cut.

21. The system of claim 14, wherein receiving the user input comprises receiving user input adjusting at least one of the pose of the virtual medical element within the image and the size of the virtual medical element.

22. The system of claim 21, wherein the processor is further configured to execute the instructions to dynamically adjust the rendering of the virtual medical element in substantially real-time in response to receiving the user input adjusting at least one of the pose of the virtual medical element within the image and the size of the virtual medical element to depict the at least one of the pose and the size of the virtual medical element adjusted by the user input.

23. The system of claim 14, wherein the rendering of the virtual medical element is at least partially transparent to allow a user’s visualization of the target region when the virtual medical element is rendered over the target region.

24. The system of claim 1, wherein: the anatomical surface comprises a hernia; and the physical medical element is a mesh patch configured to cover the hernia.

25. A system comprising: a memory storing instructions; and a processor communicatively coupled to the memory and configured to execute the instructions to: ​ obtaining anatomical property data representing properties associated with an anatomical surface to be covered by a physical medical element, the anatomical surface being within an internal space of a patient; determining, based on the anatomical property data, a suture plan for suturing the physical medical element to the anatomical surface when the one or more surgical instruments controlled by a computer-assisted surgical system hold the physical medical element in place on the anatomical surface; and graphically indicating the suture plan within an image of the internal space.

26. The system of claim 25, wherein the suture plan comprises information of at least one of a suture pattern to be used for suturing the physical medical element to the anatomical surface, a spacing to be used between suture lines for suturing the physical medical element to the anatomical surface, a type of line to be used for suturing the physical medical element to the anatomical surface, and a length of line needed to suture the physical medical element to the anatomical surface.

27. The system of claim 25, wherein graphically indicating the suture plan comprises instructing a display device to present a virtual suture guide over a target region within the image of the internal space, the target region depicting the anatomical surface to be covered by the physical medical element, the virtual suture guide being configured to guide a user to perform suturing the physical medical element to the anatomical surface.

28. A non-transitory computer-readable medium storing instructions that, when executed, direct a processor of a computing device to: obtain anatomical property data representing properties associated with an anatomical surface to be covered by a physical medical element, the anatomical surface being within an internal space of a patient; and determine, based on the anatomical property data, a set of placement guide parameters comprising one or more parameters configured to specify a suture plan for suturing the physical medical element to the anatomical surface when one or more surgical instruments controlled by a computer-assisted surgical system hold the physical medical element in place on the anatomical surface.

29. A non-transitory computer-readable medium storing instructions that, when executed, direct a processor of a computing device to: obtain anatomical property data representing properties associated with an anatomical surface to be covered by a physical medical element, the anatomical surface being within an internal space of a patient; determine, based on the anatomical property data, a suture plan for suturing the physical medical element to the anatomical surface when the one or more surgical instruments controlled by a computer-assisted surgical system hold the physical medical element in place on the anatomical surface; and graphically indicate the suture plan within an image of the internal space.

Images