Systems and methods for performing operations performed based on external body wall data and internal depth data associated with a computer-assisted surgical system

Abstract

An example operating management system is configured to obtain external body wall data representing a three-dimensional model of an external body wall of a patient, obtain internal depth data representing a depth map of an internal space for the patient, and perform an operation associated with a computer-assisted surgical system based on the external body wall data and the internal depth data, the computer-assisted surgical system configured to perform a procedure with respect to the patient.

Description

[0001] Related applications

[0002] This application claims priority to U.S. Provisional Patent Application No. 62 / 888,236, filed August 16, 2019, entitled “SYSTEMS AND METHODS FOR PERFORMANCE OF EXTERNAL BODY WALL DATA AND INTERNAL DEPTH DATA-BASED PERFORMANCE OF OPERATIONS ASSOCIATED WITH A COMPUTER-ASSISTED SURGICAL SYSTEM”, the contents of which are incorporated herein by reference in their entirety. Background Technology

[0003] Computer-assisted surgical systems are commonly used to perform minimally invasive and / or other types of surgical procedures within a patient's internal space. For example, multiple surgical instruments can be coupled to the manipulator arms of a computer-assisted surgical system, inserted into the patient through one or more ports (e.g., small openings or incision sites) within the patient's external body wall, and then robotically and / or remotely controlled to perform surgical procedures within the patient. Proper positioning of one or more ports within the patient's external body wall allows for adequate access to the target anatomical structures within the patient's body with one or more surgical instruments, minimizing the chance of collisions between manipulator arms and improving the effectiveness of the surgical procedure. However, proper port positioning depends on many factors that may be patient-specific. For example, proper port positioning often depends on the size and shape of the patient's external body wall, as well as the size, shape, and location of anatomical features within the patient's internal space. Furthermore, other operations associated with the computer-assisted surgical system (e.g., the positioning of setup joints to which the manipulator arms are attached) may depend on specific patient characteristics. Summary of the Invention

[0004] The following description presents a simplified summary of one or more aspects of the systems and methods described herein. This invention is not a comprehensive overview of all anticipated aspects, nor is it intended to identify key or essential elements of all aspects, nor to define the scope of any or all aspects. Its sole purpose is to present one or more aspects of the systems and methods described herein as a prelude to the detailed embodiments presented below.

[0005] An exemplary system includes a memory storing instructions and a processor communicatively coupled to the memory and configured to execute instructions to obtain external body wall data representing a three-dimensional model of the patient's external body wall, obtain internal depth data representing a depth map of the patient's internal space, and perform operations associated with a computer-assisted surgical system based on the external body wall data and the internal depth data, the computer-assisted surgical system being configured to perform procedures concerning the patient.

[0006] An exemplary method includes obtaining external body wall data representing a three-dimensional model of the patient's external body wall through an operations management system, obtaining internal depth data representing a depth map for the patient's internal space through the operations management system, and performing operations associated with a computer-assisted surgical system based on the external body wall data and the internal depth data through the operations management system, the computer-assisted surgical system being configured to perform procedures concerning the patient.

[0007] An exemplary non-transitory computer-readable medium storage instruction, when executed, directs a processor of a computing device to obtain external body wall data representing a three-dimensional model of the patient's external body wall, obtain internal depth data representing a depth map for the patient's internal space, and perform operations associated with a computer-assisted surgical system based on the external body wall data and the internal depth data, the computer-assisted surgical system being configured to perform procedures concerning the patient.

[0008] Attached illustrations

[0009] The accompanying drawings illustrate various embodiments and are part of the specification. The illustrated embodiments are merely examples and are not intended to limit the scope of this disclosure. Throughout the drawings, the same or similar reference numerals indicate the same or similar elements.

[0010] Figure 1 An exemplary operation management system based on the principles described herein is illustrated.

[0011] Figure 2 The illustration shows an exemplary configuration based on the principles described herein, in which, Figure 1 The system performs operations associated with a computer-assisted surgical system based on external body wall data representing a three-dimensional model of the patient's external body wall and internal depth data representing a depth map of the patient's internal space.

[0012] Figure 3 An exemplary implementation based on the principles described herein is illustrated, in which, Figure 1 The system obtains external body wall data and internal depth data from depth sensors included in the imaging device.

[0013] Figure 4 An exemplary implementation based on the principles described herein is illustrated, in which the depth sensor is implemented by a time-of-flight sensor included in an imaging device.

[0014] Figure 5 An exemplary implementation based on the principles described herein is shown, in which the lighting system is implemented by a single lighting source.

[0015] Figure 6 An exemplary implementation based on the principles described herein is illustrated, in which the lighting system is implemented by a single lighting source.

[0016] Figure 7 An exemplary implementation based on the principles described herein is illustrated, in which an illumination source is integrated into a time-of-flight sensor.

[0017] Figure 8 An exemplary structural implementation of an imaging device based on the principles described herein is illustrated.

[0018] Figure 9 A cross-sectional view of the axis of an imaging device based on the principles described herein is depicted.

[0019] Figure 10 An exemplary implementation based on the principles described herein is illustrated, in which the depth sensor is implemented by a visible light camera included in an imaging device.

[0020] Figure 11 An exemplary configuration based on the principles described herein is shown, in which, Figure 1 The system obtains external body wall data from an external body wall data source.

[0021] Figure 12 An exemplary configuration based on the principles described herein is shown, in which, by Figure 1 The operations performed by the system are further based on kinematic data generated by the computer-assisted surgical system.

[0022] Figure 13 An exemplary implementation of a computer-assisted surgical system based on the principles described herein is shown.

[0023] Figure 14 This is a simplified diagram illustrating an exemplary implementation of a control system based on the principles described herein.

[0024] Figure 15 This is a simplified diagram of the method for selecting port locations based on the principles described in this article.

[0025] Figures 16A to 16B This is a simplified diagram of the different positions and orientations of end effectors within the workspace, based on the principles described in this article.

[0026] Figure 17 An exemplary method based on the principles described herein is illustrated.

[0027] Figure 18 An exemplary computing device based on the principles described herein is illustrated. Detailed Implementation

[0028] This document describes systems and methods for performing operations associated with a computer-assisted surgical system based on external body wall data and internal depth data. For example, an exemplary operations management system may obtain external body wall data representing a three-dimensional model of the patient's external body wall, obtain internal depth data representing a depth map of the patient's internal space, and perform operations associated with a computer-assisted surgical system configured to perform procedures concerning the patient based on the external body wall data and internal depth data.

[0029] The systems and methods described herein advantageously use both external body wall data and internal depth data to perform operations associated with a computer-assisted surgical system. This allows the operation to be more precise, accurate, and efficient than, or relative to, operations performed by conventional computer-assisted surgical systems that cannot simultaneously access both types of data. These and other advantages and benefits of the systems and methods described herein will become apparent here.

[0030] Figure 1 An exemplary operation management system 100 (“System 100”) is illustrated, configured to perform operations based on external body wall data and internal depth data associated with a computer-assisted surgical system. As shown, System 100 may include (but is not limited to) a storage facility 102 and a processing facility 104 selectively and communicatively coupled to each other. Facilities 102 and 104 may each include or be implemented by hardware and / or software components (e.g., processors, memory, communication interfaces, instructions stored in memory for execution by the processor, etc.). For example, facilities 102 and / or 104 may be implemented by any component within the computer-assisted surgical system itself. As another example, facilities 102 and / or 104 may be implemented by a computing device that is separate from and communicatively coupled to the computer-assisted surgical system. In some examples, facilities 102 and 104 may be distributed among multiple devices and / or multiple locations, such as those that may serve a particular implementation.

[0031] Storage facility 102 may maintain (e.g., store) executable data used by processing facility 104 to perform one or more of the operations described herein. For example, storage facility 102 may store instructions 106 that may be executed by processing facility 104 to perform one or more of the operations described herein. Instructions 106 may be implemented by any suitable application, software, code, and / or other instance of executable data. Storage facility 102 may also maintain any data received, generated, managed, used, and / or transmitted by processing facility 104.

[0032] Processing facility 104 may be configured to perform (e.g., execute instructions 106 stored in storage facility 102 to perform) various operations described herein. For example, processing facility 104 may be configured to obtain external body wall data representing a three-dimensional model of the patient's external body wall, obtain internal depth data representing a depth map of the patient's internal space, and perform operations associated with a computer-assisted surgical system configured to perform procedures concerning the patient based on the external body wall data and the internal depth data. These and other operations that may be performed by system 100 (e.g., processing facility 104) are described herein.

[0033] Figure 2 An exemplary configuration is illustrated in which system 100 performs operations 202 associated with computer-assisted surgical system 204 based on external body wall data 206 representing a three-dimensional model of the patient's external body wall and internal depth data 208 representing a depth map of the patient's internal space. System 100 may obtain the external body wall data 206 and internal depth data 208 in any suitable manner, examples of which are provided herein.

[0034] The computer-assisted surgical system 204 can be implemented by any suitable surgical system that uses robotics and / or remote manipulation technology to perform procedures concerning the patient (e.g., minimally invasive surgical procedures). An exemplary computer-assisted surgical system is described herein.

[0035] Operation 202 may include any suitable operation performed by the computer-assisted surgical system 204. Where system 100 is implemented by the computer-assisted surgical system 204 itself, operation 202 may be performed by the computer-assisted surgical system 204. Examples of operation 202 are described herein.

[0036] Various exemplary ways in which system 100 can obtain external body wall data 206 and internal depth data 208 will now be described.

[0037] Figure 3An exemplary embodiment 300 is illustrated, wherein system 100 obtains external body wall data 206 and internal depth data 208 from a depth sensor 302 included in an imaging device 304. As shown, the depth sensor 302 is configured to generate depth data 306 representing a depth map of the scene imaged by the imaging device 304. As described herein, depending on the positioning of the imaging device 304 relative to the patient, the depth data 306 may represent either external body wall data 206 or internal depth data 208.

[0038] Imaging device 304 may be implemented by an endoscope or other camera device configured to capture images of the scene. In some examples, imaging device 304 may be configured to be attached to and controlled by computer-assisted surgical system 204. In alternative examples, imaging device 304 may be handheld and manually operated by an operator (e.g., a surgeon).

[0039] In some examples, the scene captured by imaging device 304 may include a surgical area associated with the patient. In some examples, the surgical area may be entirely located within the patient and may include a region within the patient's body located at or near the site of a planned, being performed, or already performed surgical procedure. For example, for a minimally invasive surgical procedure performed on tissue within the patient, the surgical area may include the tissue, the anatomical structures beneath the tissue, and the space surrounding the tissue, such as the space in which surgical instruments for performing the surgical procedure are located. In some example embodiments, a surgical area entirely located within the patient's body may be referred to as an "internal space." As described herein, any internal anatomical structures of the patient (e.g., blood vessels, organs, and / or tissues) and / or surgical instruments located within the internal space may be referred to as objects and / or structures.

[0040] In some examples, the surgical area included in the scene captured by imaging device 304 may also include an area outside the patient. For example, imaging device 304 may be used to image the patient's external body wall.

[0041] The depth sensor 302 included in the imaging device 304 can be implemented by any suitable sensor configured to generate depth data 306. For example, as described herein, the depth sensor 302 can be implemented by a time-of-flight sensor, a stereo camera, and / or any other suitable component that may serve a particular implementation. Depending on the positioning of the imaging device 304, the depth data 306 may represent a depth map of the patient's external body walls or a depth map of the patient's internal space.

[0042] In embodiment 300, system 100 is configured to obtain external body wall data 206 by guiding depth sensor 302 to scan (e.g., image) the patient's external body wall when imaging device 304 is outside the patient. In this configuration, the depth data 306 generated by depth sensor 302 represents a depth map for the external body wall and may be referred to herein accordingly as "external depth data".

[0043] The depth sensor 302 can scan the patient's external body wall in any suitable manner. In some examples, the imaging device 304 is coupled to the computer-assisted surgical system 204 (e.g., a manipulator arm attached to the computer-assisted surgical system 204) while the depth sensor 302 scans the external body wall. Alternatively, the imaging device 304 can be manually held by a user (e.g., a surgeon) while the depth sensor 302 scans the external body wall. In some examples, the scan can be performed while the patient is being inhaled.

[0044] System 100 can receive depth data 306 acquired by depth sensor 302 while imaging device 304 is located outside the patient in any suitable manner. For example, system 100 can guide depth sensor 302 to transmit depth data 306 to system 100. System 100 can then use depth data 306 as external body wall data 206.

[0045] In implementation 300, imaging device 304 and depth sensor 302 are also used by system 300 to acquire internal depth data 208. For example, depth sensor 302 can be aimed at the patient's internal space through a camera port formed through the patient's external body wall. This can be done, for example, by inserting imaging device 304 through the camera port such that the distal end of imaging device 304 is located within the patient's internal space. In this configuration, system 100 can guide depth sensor 302 to scan the internal space to acquire depth data 306. In this configuration, depth data 306 represents a depth map of the patient's internal space and may therefore be referred to herein as "internal depth data".

[0046] System 100 can receive depth data 306 acquired by depth sensor 302 while imaging device 304 is aimed at the patient's internal space in any suitable manner. For example, system 100 can guide depth sensor 302 to transmit depth data 306 to system 100. System 100 can then use depth data 306 as internal depth data 208.

[0047] Figure 4 An exemplary embodiment 400 is illustrated, wherein the depth sensor 302 is implemented by a time-of-flight sensor 402 included in the imaging device 304. Although the time-of-flight sensor 402 is... Figure 4 As shown and mentioned in the examples provided herein, but which are separate from (i.e. physically different from) the visible light camera also included in imaging device 304, any other type of depth sensor may be additionally or alternatively used to implement depth sensor 302. For example, depth sensor 302 may alternatively be implemented by a structured light sensor, an interferometer, and / or any other suitable sensor configured to acquire depth data that may serve a particular implementation.

[0048] In implementation 400, system 100 can obtain depth data 306 by guiding time-of-flight sensor 402 to acquire and receive depth data 306 from time-of-flight sensor 402. For example, system 100 can guide time-of-flight sensor 402 to acquire external depth data representing a depth map of the patient's external body wall by scanning the external body wall when imaging device 304 is outside the patient. System 100 can receive external depth data from time-of-flight sensor 402 and use the external depth data as external body wall data 206. System 100 can also guide time-of-flight sensor 402 to acquire internal depth data 208 while time-of-flight sensor 402 is aimed at the patient's internal space through a camera port formed through the patient's external body wall.

[0049] Therefore, in embodiment 400, system 100 is communicatively coupled to imaging device 304 via bidirectional communication link 404 and communicatively coupled to illumination system 406 via communication link 408. Communication links 404 and 408 can each be implemented via any suitable wired and / or wireless communication medium that may serve a particular embodiment. As described herein, system 100 can use communication links 404 and 408 to guide time-of-flight sensor 402 to acquire depth data 306 and to receive depth data 306 from time-of-flight sensor 402.

[0050] As shown in the figure, imaging device 304 includes a time-of-flight sensor 402 and a visible light camera 410 (“camera 410”), which is configured to generate image data 412 representing a two-dimensional visible light image of the scene. The time-of-flight sensor 402 may be implemented by one or more photodetectors (e.g., one or more single-photon avalanche diode (“SPAD”) detectors), a CCD sensor, a CMOS sensor, and / or any other suitable configuration configured to acquire depth data of the scene. The camera 410 may be implemented by any suitable image sensor, such as a charge-coupled device (“CCD”) image sensor, a complementary metal-oxide-semiconductor (“CMOS”) image sensor, etc.

[0051] In some examples, system 100 may be configured to control the operation of imaging device 304 (e.g., by controlling the operation of camera 410 and time-of-flight sensor 402). For example, system 100 may include one or more camera control units (“CCUs”) configured to control various parameters of camera 410 and / or time-of-flight sensor 402 (e.g., activation time, auto exposure, etc.).

[0052] System 100 may be additionally or alternatively configured to provide operating power to components included in imaging device 304. For example, although imaging device 304 is communicatively coupled to system 100, system 100 may transmit operating power to camera 410 and time-of-flight sensor 402 in the form of one or more power signals.

[0053] System 100 can be configured to use imaging device 304 and illumination system 406 to acquire depth data 306 and image data 412. In some examples, depth data 306 and image data 412 can be used to generate stereoscopic images of the scene. This will be described in more detail below.

[0054] The lighting system 406 can be configured to emit light 414 (e.g., in the direction of system 100) to illuminate a scene to be imaged by the imaging device 304. The light 414 emitted by the lighting system 406 may include visible and / or invisible light (e.g., infrared light). As shown, the light 414 can travel through the imaging device 304 to the scene (e.g., through an illumination channel within the imaging device 304 that may be implemented by one or more optical fibers, light guides, lenses, etc.). Various implementations and configurations of the lighting system 406 are described herein.

[0055] As shown in the figure, the light 414 emitted by the illumination system 406 can be reflected from the surface 416 within the scene imaged by the imaging device 304. When the imaging device 304 is outside the patient, the surface 416 represents the surface of the patient's external body wall. When the imaging device 304 is aimed at the patient's internal space, the surface 416 represents a surface within the internal space (e.g., the surface of organs and / or other tissues).

[0056] Visible light camera 410 and time-of-flight sensor 402 can each detect reflected light 414. Visible light camera 410 can be configured to generate image data 412 representing a two-dimensional visible light image of a scene including surface 416 based on the detected light. Time-of-flight sensor 402 can be configured to generate depth data 306 based on the detected light. Image data 412 and depth data 306 can each have any suitable format.

[0057] To generate a stereoscopic image of the scene, system 100 can direct the lighting system 406 to emit light 414. System 100 can also activate (e.g., turn on) a visible light camera 410 and a time-of-flight sensor 402. The light 414 travels into the scene and is reflected from a surface 416 (and, in some examples, one or more other surfaces in the scene). Both the camera 410 and the time-of-flight sensor 402 detect the reflected light 414.

[0058] Camera 410 (and / or other circuitry included in imaging device 304) can generate image data 412 representing a two-dimensional visible light image of the scene based on detected light 414. This can be performed in any suitable manner. Visible light camera 410 (and / or other circuitry included in imaging device 304) can transmit image data 412 to system 100. This can also be performed in any suitable manner.

[0059] The time-of-flight sensor 402 can generate depth data 306 representing a depth map of the scene (e.g., a depth map of surface 416) based on the detected light 414. This can be performed in any suitable manner. For example, the time-of-flight sensor 402 can measure the amount of time it takes for photons of light 414 to travel from the illumination system 406 to the time-of-flight sensor 402. Based on this amount of time, the time-of-flight sensor 402 can determine the depth of the position of surface 416 relative to the time-of-flight sensor 402. The data representing this depth can be represented in the depth data 306 in any suitable manner. For example, the depth map represented by the depth data 306 can include an array of depth values ​​(e.g., Z-buffer values) corresponding to each pixel in the image.

[0060] The time-of-flight sensor 402 (and / or other circuitry included in the imaging device 304) can transmit depth data 306 to the system 100. This can be done in any suitable manner.

[0061] System 100 can receive image data 412 and depth data 306 and perform one or more processing operations on the image data 412 and depth data 306. For example, based on the image data 412 and depth data 306, system 100 can generate a right-side perspective image of the scene and a left-side perspective image representing the scene. This can be performed in any suitable manner. System 100 can then instruct a display device to simultaneously display the right-side perspective image and the left-side perspective image in a manner that forms a stereoscopic image of the scene. In some examples, the display device is included in and / or communicatively coupled to the computer-assisted surgical system 204.

[0062] Figure 5An exemplary embodiment 500 is shown, in which the lighting system 406 is implemented by a single light source 502. The light source 502 may be configured to emit visible light 414-1.

[0063] Visible light 414-1 may include one or more color components. For example, visible light 414-1 may include white light, which includes full-spectrum color components (e.g., red, green, and blue components). The wavelength of the red component is between approximately 945 nanometers and 800 nanometers (“nm”). The wavelength of the green component is between approximately 820 nm and 860 nm. The wavelength of the blue component is between approximately 750 nm and 790 nm.

[0064] In some examples, visible light 414-1 is biased to include a color component that is more than the other color component. For example, visible light 414-1 can be biased to include a blue component that is more than the red and green components.

[0065] In embodiment 500, the time-of-flight sensor 402 is configured to also detect visible light 414-1. Therefore, the same illumination source 502 can be used for both the camera 410 and the time-of-flight sensor 402.

[0066] Figure 6 An exemplary embodiment 600 is illustrated, in which the illumination system 406 is implemented by separate illumination sources 502-1 and 402-2. In embodiment 600, illumination source 502-1 is configured to emit visible light 414-1 detected by camera 410. Illumination source 502-2 is configured to emit light 414-2 reflected from surface 416 and detected by time-of-flight sensor 402. In some examples, light 414-2 is invisible light, such as infrared light. By having separate illumination sources 502 for camera 410 and time-of-flight sensor 402, camera 410 and time-of-flight sensor 402 can be configured to operate independently.

[0067] Figure 7 An exemplary embodiment 700 is shown, in which an illumination source 502-2 is integrated into a time-of-flight sensor 402. In embodiment 700, the system 100 can control (e.g., activate) the illumination source 502-2 by transmitting a command to the time-of-flight sensor 402.

[0068] Figure 8An exemplary structural embodiment of imaging device 304 is illustrated. As shown, imaging device 304 includes a camera 802 and an axis 804, which is coupled to and extends away from camera 802. Camera 802 and axis 804 together form the housing of imaging device 304. Imaging device 304 can be manually handled and controlled (e.g., by a surgeon performing surgical procedures on a patient). Alternatively, camera 802 can be coupled to a manipulator arm of computer-assisted surgical system 204. In this configuration, imaging device 304 can be controlled by computer-assisted surgical system 204 using robotic and / or remote operation technologies.

[0069] As shown in the figure, the illumination channel 806 can pass through the camera 802 and the axis 804. The illumination channel 806 is configured to provide a guide for the light emitted by the illumination system 406 to travel to the scene being imaged by the imaging device 304.

[0070] The distal end 808 of axis 804 can be positioned at or near the scene to be imaged by imaging device 304. For example, the distal end 808 of axis 804 can be inserted into a patient. In this configuration, imaging device 304 can be used to capture images of anatomical structures and / or other objects within the patient's body.

[0071] The camera 410 and time-of-flight sensor 402 can be located anywhere along axis 804 of the imaging device 304. Figure 8 In the example shown, camera 410 and time-of-flight sensor 402 are located at the distal end 808 of axis 804. This configuration may be referred to as a “cutting-edge” configuration. Alternatively, camera 410 and / or time-of-flight sensor 402 may be positioned further toward camera 802 and / or within camera 802. In these alternative configurations, optics (e.g., lenses, optical fibers, etc.) included in axis 804 and / or camera 206 can transmit light from the scene to camera 410 and / or time-of-flight sensor 402.

[0072] In some examples, the camera 410 and the time-of-flight sensor 402 may be offset at different distances from the distal end 808 of the axis 804. By offsetting the camera 410 and the time-of-flight sensor 402 from the distal end 808 of the axis 804, the imaging device 304 may adopt a tapered configuration with reduced dimensions (e.g., diameter) toward the distal end 808 of the axis 804, which may facilitate insertion of the imaging device 304 into the patient's internal space.

[0073] Figure 9 Depicting along Figure 8The figure shows a cross-sectional view of the axis 804 of the imaging device 304, taken by line 9-9. As shown, the axis 804 includes a relatively flat bottom surface 902. Referring to this bottom surface 902, the time-of-flight sensor 402 is positioned above the camera 410. This positioning allows for a narrower axis 804 compared to the axis of a conventional imaging device with two cameras side-by-side for acquiring stereoscopic images. It will be appreciated that the camera 410 and the time-of-flight sensor 402 can have any suitable relative positioning within the axis 804 as may be necessary for a particular implementation.

[0074] Figure 10 An exemplary embodiment 1000 is illustrated, wherein the depth sensor 402 is implemented by visible light cameras 410-1 and 410-2 included in the imaging device 304. In embodiment 1000, the system 100 can obtain depth data 306 by guiding camera 410-1 to acquire a first image (e.g., a first two-dimensional image) of the patient's internal space, by guiding camera 410-2 to acquire a second image (e.g., a second two-dimensional image) of the patient's internal space, and by generating a depth map represented by depth data 306 based on the first and second images.

[0075] exist Figure 10 In the figure, the first image acquired by camera 410-1 is represented by image data 412-1, and the second image acquired by camera 410-2 is represented by image data 412-2. As shown, image data 412-1 and 412-2 are transmitted to a depth data generator 1002 implemented by system 100. The depth data generator 1002 can use any visible image-based technique to determine depth data 306 based on image data 412-1 and 412-2.

[0076] Other configurations of the imaging device 304 are possible based on the systems and methods described herein. For example, the imaging device 304 may include multiple cameras 410 and / or multiple time-of-flight sensors 402. For illustration purposes, the imaging device 304 may include two cameras 410 combined with a single time-of-flight sensor 402. In these embodiments, depth data may be generated based on images acquired by the two cameras 410. The depth data generated by the time-of-flight sensor 402 can be used to fine-tune or otherwise enhance the depth data generated based on the images acquired by the two cameras 410.

[0077] In some examples, system 100 can obtain external body wall data 206 from sources other than imaging device 204. For example, Figure 11An exemplary configuration 1100 is shown, in which system 100 obtains external body wall data 206 from an external body wall data source 1102 (“source 1102”) different from imaging device 304. Source 1102 may be implemented by a computer-assisted computed tomography (CT) scanner, magnetic resonance imaging (MRI) device, ultrasound device, three-dimensional scanning (LIDAR) device, and / or any other suitable alternative imaging device. As another example, source 1102 may be implemented by a computing device configured to maintain previously acquired external body wall data 206. For example, external body wall data 206 may be generated for the patient during a first surgical procedure. External body wall data 206 may be stored by the computing device and used by the patient during a second surgical procedure following the first surgical procedure.

[0078] Figure 12 An exemplary configuration 1200 is shown, wherein the operation 202 performed by system 100 is further based on kinematic data 1202 generated by computer-assisted surgical system 204. Therefore, in configuration 1200, operation 202 is based on external body wall data 206, internal depth data 208, and kinematic data 1202. An exemplary operation 202 based on external body wall data 206, internal depth data 208, and kinematic data 1202 is described herein.

[0079] Kinematic data 1202 can represent any type of kinematic information associated with one or more components of the computer-assisted surgical system 204 (e.g., one or more manipulator arms and / or placement joints of the computer-assisted surgical system 204). Kinematic data 1202 can additionally or alternatively represent any type of kinematic information associated with one or more components coupled to the computer-assisted surgical system 204 (e.g., imaging device 304 and / or one or more surgical instruments). Such kinematic information can include (but is not limited to) information indicating the displacement, orientation, position, and / or movement of one or more components of the computer-assisted surgical system 204 and / or components coupled to the computer-assisted surgical system 204. For example, kinematic data 1202 of imaging device 304 generated when imaging device 304 is coupled to the computer-assisted surgical system 204 can indicate the positioning and / or orientation of imaging device 304 when imaging device 304 acquires depth data 306 and / or image data 412. Such positioning and / or orientation can be relative to a specific reference position and / or orientation, as may serve a particular implementation. For example, kinematic data 1202 can indicate that when depth sensor 302 acquires external depth data, imaging device 304 is a certain distance away from the patient's external body wall, or indicate that when depth sensor 302 acquires internal depth data 208, the distal end of imaging device 304 is inserted into the patient's body a certain distance.

[0080] Kinematic data 1202 can be generated by the computer-assisted surgical system 204 in any suitable manner. For example, one or more transducers and / or sensors within the computer-assisted surgical system 204 can track displacement, orientation, position, movement and / or other types of kinematic information and output kinematic data 1202 (or sensor output data used by the computer-assisted surgical system 204 to generate kinematic data 1202).

[0081] In some examples, system 100 can use kinematic data 1202 to register external body wall data 206 with internal depth data 208. For example, while depth sensor 302 (e.g., time-of-flight sensor 402) scans the patient's external body wall and generates depth data 306 as external body wall data 206, imaging device 304 can be attached to the manipulator arm of computer-assisted surgical system 204. Imaging device 304 can then be inserted into the patient's internal space to generate depth data 306 as internal depth data 208. During both operations, computer-assisted surgical system 204 can track the position of imaging device 304 and output kinematic data 1202 representing that position. System 100 can then use kinematic data 1202 to register external body wall data 206 with internal depth data 208.

[0082] As used herein, the registration of external body wall data 206 and internal depth data 208 refers to mapping the external body wall data 206 and internal depth data 208 in a manner that generates a three-dimensional model (also referred to herein as a "patient model") combining the patient's external body wall and the patient's internal space. In this way, system 100 can know the location of certain internal structures relative to different positions on the patient's external body wall. Therefore, operation 202 performed by system 100 can be based on the registration of external body wall data 206 and internal depth data 208.

[0083] Various examples of operations 202 that can be performed by system 100 relative to computer-assisted surgical system 204 based on external body wall data 206 and internal depth data 208 will now be provided. These examples are merely illustrations of the many different types of operations that can be performed by system 100 based on external body wall data 206 and internal depth data 208 according to the systems and methods described herein.

[0084] In some examples, system 100 can perform operation 202 by identifying port locations on the patient's external body wall based on external body wall data 206 and internal depth data 208, through which computer-assisted surgical system 204 inserts surgical instruments into the patient's internal space.

[0085] For illustration purposes, Figure 13 An exemplary embodiment of the computer-assisted surgical system 204 is shown. It will be appreciated that... Figure 13 The components shown are merely exemplary, and additional or alternative components may be included in the computer-assisted surgical system 204, as may be suited to a particular implementation.

[0086] As shown in the figure, the computer-assisted surgical system 204 includes a control system 1302, a user control system 1304, and an auxiliary system 1306 that are communicatively coupled to each other. The computer-assisted surgical system 204 can be used by a surgical team to perform computer-assisted surgical procedures on a patient 1308. As shown, the surgical team may include a surgeon 1310-1, an assistant 1310-2, a nurse 1310-3, and an anesthesiologist 1310-4, all of whom can be collectively referred to as "surgical team members 1310". Additional or alternative surgical team members may be present during surgical consultations, depending on the specific implementation.

[0087] although Figure 13 The illustration depicts a minimally invasive surgical procedure in progress; however, it should be understood that the computer-assisted surgical system 204 can be similarly used to perform open surgical procedures or other types of surgical procedures that can similarly benefit from the accuracy and convenience of the computer-assisted surgical system 204. Additionally, it should be understood that a surgical consultation employing the computer-assisted surgical system 204 can include not only the surgical phase of the surgical procedure (such as...) Figure 13 (Illustrated), and may also include preoperative, postoperative and / or other appropriate stages of a surgical procedure. A surgical procedure may include any procedure that uses manual and / or instrumental techniques on the patient to investigate or treat the patient's physical condition.

[0088] like Figure 13As shown, the manipulation system 1302 may include a plurality of manipulator arms 1312 (e.g., manipulator arms 1312-1 to 1312-4) to which a plurality of surgical instruments may be coupled. Each surgical instrument may be implemented by any suitable surgical tool (e.g., a tool with tissue interaction capabilities), medical tool, imaging device (e.g., an endoscope), sensing device (e.g., a force-sensing surgical instrument), diagnostic instrument, or similar tool that can be used to perform computer-assisted surgical procedures on the patient 1308 (e.g., by at least partially inserting into and being manipulated to perform computer-assisted surgical procedures on the patient 1308). Although the manipulation system 1302 is depicted and described herein as including four manipulator arms 1312, it will be appreciated that the manipulation system 1302 may include only a single manipulator arm 1312 or include any other number of manipulator arms that may serve a particular implementation.

[0089] The manipulator arm 1312 and / or surgical instruments attached to the manipulator arm 1312 may include one or more displacement transducers, orientation sensors, and / or positioning sensors for generating raw (i.e., uncorrected) kinematic information. One or more components of the computer-assisted surgical system 204 may be configured to use kinematic information to track (e.g., determine the position of surgical instruments) and / or control surgical instruments.

[0090] User control system 1304 can be configured to facilitate surgeon 1310-1's control of manipulator arm 1312 and surgical instruments attached to manipulator arm 1312. For example, surgeon 1310-1 can interact with user control system 1304 to remotely move or manipulate manipulator arm 1312 and surgical instruments. To this end, user control system 1304 can provide surgeon 1310-1 with images (e.g., high-resolution 3D images) of the surgical area associated with patient 1308 captured by an imaging system (e.g., any medical imaging system described herein). In some examples, user control system 1304 may include a stereoscopic viewer with two displays, where surgeon 1310-1 can view a stereoscopic image of the surgical area associated with patient 1308 and generated by a stereoscopic imaging system. Surgeon 1310-1 can use the images to perform one or more procedures, wherein one or more surgical instruments are attached to manipulator arm 1312.

[0091] To facilitate control of surgical instruments, the user control system 1304 may include a set of master controls. These master controls can be manipulated by the surgeon 1310-1 to control the movement of surgical instruments (e.g., by employing robotic and / or remote operation technologies). The master controls can be configured to detect various hand, wrist, and finger movements of the surgeon 1310-1. In this way, the surgeon 1310-1 can intuitively perform procedures using one or more surgical instruments.

[0092] The auxiliary system 1306 may include one or more computing devices configured to perform primary processing operations of the computer-assisted surgical system 204. In this configuration, the one or more computing devices included in the auxiliary system 1306 may control and / or coordinate operations performed by various other components of the computer-assisted surgical system 204, such as the manipulation system 1302 and the user control system 1304. For example, the computing devices included in the user control system 1304 may transmit instructions to the manipulation system 1302 via the one or more computing devices included in the auxiliary system 1306. As another example, the auxiliary system 1306 may receive and process image data representing images captured by an imaging device attached to one of the manipulator arms 1312 from the manipulation system 1302.

[0093] In some examples, the assistive system 1306 may be configured to present visual content to surgical team members 1310 who may not have access to the images provided to surgeon 1310-1 at the user control system 1304. For this purpose, the assistive system 1306 may include a display monitor 1314 configured to display one or more user interfaces (e.g., images of the surgical area (e.g., 2D images), information associated with patient 1308 and / or surgical procedures, and / or other visual content that may be available for a particular implementation). For example, the display monitor 1314 may display an image of the surgical area along with additional content displayed concurrently with the image (e.g., graphical content, contextual information, etc.). In some embodiments, the display monitor 1314 is implemented as a touchscreen display that surgical team members 1310 may interact with (e.g., via touch gestures) to provide user input to the computer-assisted surgical system 204.

[0094] The operating system 1302, the user control system 1304, and the auxiliary system 1306 can be communicatively coupled to each other in any suitable manner. For example, Figure 13As shown, the operating system 1302, the user control system 1304, and the auxiliary system 1306 can be communicatively coupled via a control line 1316, which can represent any wired or wireless communication link that can serve a particular implementation. Therefore, the operating system 1302, the user control system 1304, and the auxiliary system 1306 can each include one or more wired or wireless communication interfaces, such as one or more local area network interfaces, Wi-Fi network interfaces, cellular interfaces, etc.

[0095] Figure 14 This is a simplified diagram illustrating an exemplary embodiment of the control system 1302. Figure 14 As shown, the manipulation system 1302 may include a mobile trolley 1402, which enables the manipulation system 1302 to be transported from one location to another, such as between or within operating rooms, to better position the manipulation system 1302 near the patient's operating table. In an alternative embodiment, the manipulation system 1302 includes a fixed base.

[0096] Starting from the proximal end of the mobile cart 1402 is a placement structure 1404. Coupled to the distal end of the placement structure is a series of placement joints 1406. Coupled to the distal end of the placement joints 1406 is a manipulator 1408, such as a universal surgical manipulator. In some examples, the series of placement joints 1406 and manipulator 1408 may implement one of the manipulator arms 1312. Although the manipulation system 1302 is shown as having only a series of placement joints 1406 and corresponding manipulators 1408, it will be appreciated that the manipulation system 1302 may include more than one series of placement joints 1406 and corresponding manipulators 1408, such that the manipulation system 1302 is equipped with multiple manipulator arms.

[0097] like Figure 14 As shown, the mounting structure 1404 includes a two-part column, comprising column links 1410 and 1412. A shoulder joint 1414 is coupled to the upper or distal end of column link 1412. Coupled to shoulder joint 1414 is a two-part cantilever comprising cantilever links 1416 and 1418. At the distal end of cantilever link 1418 is a wrist joint 1420, and coupled to wrist joint 1420 is an orientation platform 1422.

[0098] The linkages and joints of the mounting structure 1404 include various degrees of freedom for changing the position and orientation (i.e., posture) of the orientation platform 1422. For example, the two-part column can be used to adjust the height of the orientation platform 1422 by moving the shoulder joint 1414 up and down along axis 1426. Additionally, the orientation platform 1422 can be rotated about the trolley 1402, the two-part column, and axis 1426 using the shoulder joint 1414. The horizontal position of the orientation platform 1422 can also be adjusted along axis 1426 using the two-part cantilever. The orientation of the orientation platform 1422 can also be adjusted by rotating the wrist joint 1420 about axis 1428. Therefore, limited by the movement constraints of the linkages and joints in the mounting structure 1404, the position of the orientation platform 1422 can be adjusted vertically above the trolley 1402 using the two-part column. The position of the orientation platform 1422 can also be adjusted radially and angularly about the trolley 1402 using the two-part cantilever and shoulder joint 1414, respectively. Furthermore, the angle orientation of the orientation platform 1422 can also be changed using the wrist joint 1420.

[0099] Orientation platform 1422 can be used as a mounting point for one or more manipulator arms. The ability to adjust the height, horizontal position, and orientation of orientation platform 1422 around mobile cart 1402 provides a flexible mounting configuration for positioning and orienting one or more manipulator arms around a workspace (such as a patient) located near mobile cart 1402. Figure 14 A single manipulator arm coupled to the orientation platform using a first placement joint 1430 is shown. Although only one manipulator arm is shown, it will be appreciated that multiple manipulator arms can be coupled to the orientation platform 1422 using additional first placement joints.

[0100] The first mounting joint 1430 forms the proximal portion of the mounting joint 1406 section of the manipulator arm. The mounting joint 1406 may further include a series of joints and links. Figure 14 As shown, the mounting joint 1406 includes at least links 1432 and 1434 coupled via one or more joints (not explicitly shown). The joints and links of the mounting joint 1406 include the ability to rotate the mounting joint 1406 about axis 1436 relative to the orientation platform 1422 using the first mounting joint 1430, the ability to adjust the height of the link 1434 relative to the orientation platform along axis 1438, and the ability to rotate the manipulator at least about axis 1440 at the distal end of the link 1434. The mounting joint 1406 may further include additional joints, links, and axes, allowing additional degrees of freedom to change the position and / or orientation of the manipulator 1408 relative to the orientation platform 1422.

[0101] Manipulator 1408 is coupled to the distal end of placement joint 1406 and includes additional links and joints that allow control of the position and orientation of surgical instrument 1442 mounted distal to manipulator 1408. Surgical instrument 1442 includes an elongated shaft 1444 coupled between manipulator 1408 and end effector 1446 via an optional articulated wrist 1448. Degrees of freedom in manipulator 1408 may allow control of roll, pitch, and yaw of elongated shaft 1444 relative to the distal end of placement joint 1406. In some examples, degrees of freedom in manipulator 1408 may further include the ability to advance and / or retract elongated shaft 1444 along insertion bracket or beam 1450 to move end effector 1446 closer to or further away from manipulator 1408 along the longitudinal axis of surgical instrument 1442. Additional control over the orientation of the end effector 1446 relative to the manipulator 1408 can be achieved using an optional wrist 1448. In some examples, the degrees of freedom for positioning the joint 1406 and the manipulator 1408 can be further controlled to maintain a remote center 1452 around a point on the surgical instrument 1442. In some examples, the remote center 1452 may correspond to a port within the patient's body, such that the remote center 1452 remains stationary when the surgical instrument 1442 is used to limit pressure on the patient's anatomy at the remote center 1452. In some examples, the surgical instrument 1442 may be an imaging device, such as an endoscope, clamp, surgical tool (e.g., a cauterizer or scalpel), etc.

[0102] Controlling the insertion position of the surgical instrument 1442 into the patient's internal space, such as by inserting the elongated shaft 1444 through a cannula located at a port for accessing the patient's internal anatomy, is desirable for the flexible operation of the manipulation system 1302 and the surgical instrument 1442. In some examples, if the port is located too close to the target tissue, the surgical instrument 1442 and the end effector 1446 may not have sufficient range of motion to enter, interact with, and manipulate the target tissue. If the port is located too far from the target tissue, the end effector 1446 may not be able to reach the target tissue. If the port position is improperly chosen, there may be intermediate tissue between the port and the target tissue, and the elongated shaft 1444 and the end effector 1446 may not be able to manipulate around the intermediate tissue, and / or the elongated shaft 1444 and the end effector 1446 may not have a comfortable or practical approach orientation to the target tissue. When the control system 1302 includes multiple manipulators 1408 and multiple instruments 1442, placing their corresponding ports too close together may increase the likelihood of interference and / or collisions between manipulator arms (e.g., corresponding beams 1450 and / or manipulators 1408), instruments 1442 and / or other parts of the control system 1302.

[0103] Conventional methods for selecting port locations typically rely on general port placement rules empirically determined from previous use of the manipulation system 1302, and common sense based on a basic understanding of the workspace configuration (e.g., the typical anatomy of a patient for a surgical procedure). As an example, in the case of upper abdominal surgery, recommendations for port location may include placing the port for imaging equipment (e.g., an endoscope) at the umbilicus and positioning additional ports along a diagonal perpendicular to the target anatomy, through the umbilicus, and at recommended intervals. Additional recommendations may include positioning one or more ports above (above) or below (below) the diagonal to accommodate instruments 1442 with different types of end effectors 1446. While these types of guidelines can provide good port locations, guidelines are not always flexible enough to accommodate variations in the workspace (e.g., patients with larger or smaller anatomy and / or patients with anatomical abnormalities due to previous procedures, lesions, etc.), changes in instruments and / or procedures, changes in operator preferences, etc.

[0104] Therefore, refer to Figures 13-14 System 100 can use external body wall data 206 and internal depth data 208 to identify port locations on the external body wall of patient 1308, through which computer-assisted surgical system 204 inserts surgical instruments (e.g., surgical instrument 1442) into the internal space of patient 1308.

[0105] For example, system 100 can identify port locations based on external body wall data 206 and internal depth data 208, port locations that allow surgical instruments 1442 to enter structures within the patient's internal space while avoiding collisions with attached surgical instruments 1442. As another example, system 100 can identify port locations that allow surgical instruments 1442 to enter structures within the internal space without collisions between manipulator arms (e.g., manipulator arm 1312-1) to which surgical instruments 1442 are attached, and manipulator arms (e.g., manipulator arm 1312-2) and manipulator arms. These operations can be performed in any suitable manner. For example, based on external body wall data 206 and internal depth data 208, system 100 can determine the depth of the structure and its relative position to various locations on the external body wall of the patient 1308. Based on this, system 100 can select appropriate port locations on the external body wall that allow access to the structure while preventing (or at least minimizing) collisions between surgical instruments 1442 and / or between manipulator arms 1312 (e.g., collisions between beams 1450).

[0106] As another example, system 100 can identify port locations that allow surgical instruments 1442 and the manipulator arm 1312 to which surgical instruments 1442 are attached to avoid unintentional contact with the patient 1308. For example, based on external body wall data 206 and internal depth data 208, system 100 can determine the positioning of the manipulator arm 1312 to avoid contact with the patient's external body wall and / or any other external features (e.g., face). This positioning can be used to determine the port location.

[0107] In some examples, one or more additional types of data may be used in conjunction with external body wall data 206 and internal depth data 208 to identify port locations. For example, system 100 may determine at least one of the following for candidate port locations: an accessibility metric, anthropomorphism metric, collision volume, and collision metric. The accessibility metric indicates the ability of a surgical instrument to reach a target structure located within the patient's internal space using the candidate port location; the anthropomorphism metric indicates how easily a user can manipulate a surgical instrument introduced into the patient's internal space via the candidate port location; the collision volume is for the portion of the computer-assisted surgical system near the candidate port location and corresponds to the volume swept by the portion of the computer-assisted surgical system near the candidate port location; and the collision metric indicates the probability of collisions between portions of the computer-assisted surgical system near the candidate port location. System 100 may use one or more of these metrics in conjunction with external body wall data 206 and internal depth data 208 to identify port locations (e.g., by designating the candidate port location as the port location).

[0108] For illustration purposes, Figure 15 This is a simplified diagram of a method 1500 for selecting a port location according to some embodiments. Method 1500 can be used in conjunction with the methods described herein based on external body wall data and internal depth data to select a port location on the patient's external body wall through which a computer-assisted surgical system inserts surgical instruments into the patient's internal space. One or more of operations 1510-1590 of method 1500 can be performed by system 100. Embodiments related to method 1500 are described more fully in PCT Publication No. WO2019089226A2, the contents of which are incorporated herein by reference in their entirety.

[0109] In some embodiments, method 1500 can be used to identify port locations, evaluate each port location, evaluate combinations of port locations, assist an operator in selecting and applying appropriate port locations, etc. In some examples, method 1500 can be used to evaluate the port locations of one or more surgical instruments (e.g., surgical instrument 1442) that are remotely operated using a manipulation system (such as manipulation system 1302). Figure 15 The operations shown are illustrative only. Method 1500 may include additional or alternative operations that may serve a particular implementation.

[0110] In operation 1510, a patient model is received. As described herein, a patient model can be generated based on external body wall data 206 and internal depth data 208.

[0111] In operation 1520, an initial set of possible port locations (also referred to herein as “candidate port locations”) is identified. In some examples, common sense about the target tissue of the procedure (e.g., the location of the lesion to be biopsied or resected) is mapped to patient model data acquired during operation 1510, and multiple possible port locations are identified on the patient’s external body wall. In some examples, possible port locations are limited to those portions of the external body wall within a threshold distance of the target anatomy, thus restricting possible port locations to locations accessible using available surgical instruments. In some examples, possible port locations may be limited based on general common sense about the anatomy, for example, limiting port locations for upper abdominal procedures to port locations on the anterior portions of the patient’s anatomy located below the thoracic cavity and above the lumbar line. Each possible port location may correspond to the location of existing openings and / or potential incision sites in the patient’s external anatomy.

[0112] In operation 1530, the target workspace (e.g., the patient's internal space) is identified. In some examples, the location of the target tissue and the procedure to be performed on the target tissue are used to identify the procedure site envelope or workspace surrounding the target tissue, where one or more surgical instruments will be manipulated to enter, grasp, manipulate, and / or otherwise interact with the target tissue. For example, an end effector for grasping, suturing, and cutting can use the target workspace (including locations close to the target tissue) to hinge jaws to a desired orientation, move the jaws around the target tissue, perform grasping, suturing, and cutting of the target tissue, and then withdraw from the target tissue. In some examples, the target workspace can be determined using kinematic models of the corresponding surgical instruments and end effectors and by identifying the movement of the surgical instruments and / or end effectors through the sweep volume of the procedure being performed.

[0113] In operation 1540, the placement of the imaging device is identified. In some examples, the position of the imaging device can be set to a default position determined based on the procedure to be performed (e.g., an upper abdominal procedure using a port located at the umbilicus), operator preferences, operator orientation, etc. In some examples, in addition to identifying the placement of the imaging device, additional information associated with the imaging device may be obtained, including the model of the imaging device, the viewing direction of the imaging device, one or more items in the field of view of the imaging device (e.g., the range of angles relative to the view approach direction captured by the imaging device, the aspect ratio of the images captured by the imaging device, the actual or perceived working distance between the imaging device and the target anatomical structure and / or the target workspace, etc.).

[0114] In operation 1550, each possible port location identified during operation 1520 is iterated to evaluate its suitability as a port location. While considering each possible port location, the analysis of operations 1552-1556 is repeated to determine a metric that can be used to characterize the level of suitability of the corresponding aspect for use with the envisioned procedure.

[0115] In operation 1552, an accessibility metric for the port location is determined. An accessibility metric is a kinematic measure of the extent to which a surgical instrument inserted into the workspace via the port location can reach the target tissue and / or target workspace identified during operation 1530. In some embodiments, the accessibility metric may address the ability of a surgical instrument to reach the target tissue from the port location. In some examples, the accessibility metric may be determined by determining an articulated volume (also known as an accessible sweep volume) within the patient's anatomy, which is reachable by an end effector (e.g., end effector 1446) via an elongated axis (e.g., elongated axis 1444) of a generally conical space articulated surgical instrument (e.g., surgical instrument 1442) having an apex at the port location (e.g., remote center 1452) during pitch and yaw and insertion level changes. In some examples, when the surgical instrument includes an articulated wrist (e.g., articulated wrist 1448), the reachable scan volume may additionally include points reachable by the articulated wrist when the pitch, yaw, and insertion levels of the surgical instrument are also adjusted. In some examples, pitch and / or yaw may be limited by the range of motion of the surgical instrument or the manipulator on which the surgical instrument is mounted, and / or insertion depth may be limited by the length of the elongated axis and / or the relative position of the remote center relative to the machine. In some examples, additional factors that may further limit the reachable scan volume include the capabilities of the manipulator, the current positioning and / or orientation of one or more joints of the manipulator, the model of the manipulator, the orientation of the patient, the orientation of the operating table on which the patient is placed, the position of the manipulator relative to the patient, etc. In some examples, one or more kinematic models of the surgical instrument and / or the manipulator on which the surgical instrument is mounted can be used to determine the reachable scan volume.

[0116] In some embodiments, the accessibility metric can address the ability of a surgical instrument to reach and manipulate around target tissue from its port location, and can be characterized as the ability to reach a flexible workspace associated with the target workspace identified during operation 1520. In some examples, a flexible scan volume similar to the reachable scan volume described above can be determined, wherein points in the flexible scan volume are further restricted to those points reachable within the workspace, depending on the ability to reach these points within the articulated range of the articulated wrist. In some examples, one or more kinematic models of the surgical instrument and / or the manipulator on which the surgical instrument is mounted can be used to determine the flexible reachable scan volume.

[0117] In some examples, the accessibility metric may be a binary pass-through metric indicating whether the target tissue is reachable and / or flexibly reachable from the port location using a surgical instrument. In some examples, the accessibility metric may be a simulated value, such as one in the range of 0 and 1, including 0 and 1, indicating the relative quality of accessibility and / or flexibly reachability. In some examples, the simulated value may be assigned based on how much of the target tissue the surgical instrument is reachable from the port location (e.g., how much of the target tissue is within the reachable scan volume). In some examples, the simulated value may be assigned based on how much of the surgical instrument's insertion range is used to reach the target tissue, where 0 indicates unreachable and 1 indicates that the surgical instrument can reach the target tissue from the port location using a predetermined percentage of full insertion. In some examples, the simulated value may be determined based on Equation 1, based on the distance between the target tissue and half of the full insertion of the surgical instrument, where the full insertion is length L and the distance between the port location and the target tissue is d. In some examples, other equations may be used.

[0118] Length simulation reachability metric = 1 - |d - 0.5L| / 0.5L Equation 1

[0119] In some examples, the simulated values ​​can be determined based on the distance between the target tissue and the centerline of the scanned volume, such that the closer the target tissue is to the centerline of the scanned volume, the higher the corresponding accessibility metric. In some examples, the simulated values ​​can be determined according to Equation 2, where 'a' is the angle between the centerline of the scanned volume and the line between the port location and the target tissue, and A is the maximum pitch and / or yaw angle of the surgical instrument. In some examples, other equations that favor a target tissue location closer to the centerline can be used.

[0120] Angle simulation accessibility measure = a / A Equation 2

[0121] In some examples, both length reachability metrics and angle-simulated reachability metrics can be used with their values, which are combined using any trigonometric norm function (e.g., minimum, multiplication, etc.).

[0122] In operation 1554, the collision volume is determined. For the surgical instrument to be manipulated within the workspace, one or more portions of the surgical instrument and / or the manipulator to which the surgical instrument is mounted, proximal to the port location, also undergo motion. This motion causes one or more portions of the surgical instrument and / or the manipulator to which the surgical instrument is mounted to move through a swept-through volume (also known as the collision volume or area of ​​activity) outside the patient and / or workspace. When more than one surgical instrument and corresponding manipulator and / or repositionable arm are used, the overlap between their respective collision volumes indicates the potential for a collision to occur during the procedure. In some examples, the collision volume at the port location can be determined using one or more kinematic models of the surgical instrument, the manipulator to which the surgical instrument is mounted, and / or the repositionable arm to which the manipulator is mounted, noting the collision volume when the surgical instrument is manipulated across the port location through its full range of motion. In some examples, the portions of the surgical instrument, manipulator, and / or repositionable arm used to generate the collision volume can be subsets of the joints and links of the surgical instrument, manipulator, and / or repositionable arm, e.g., only those... Figure 14 Beam 1450 in the example.

[0123] In operation 1556, an anthropomorphic measure of port location is determined. This anthropomorphic measure captures how easily an operator can manipulate the end effector toward the target tissue and use the port location to manipulate the end effector around the target tissue. In some examples, when surgical instruments and end effectors are to be manipulated such that movement of the input control device relative to the display device causes corresponding movement of the surgical instruments and end effectors (e.g., the surgical instruments and end effectors move as if they were surgical instruments held in the operator's hand), the most natural way to orient the end effector toward the workspace might be to move it from the lower left (as if held in the left hand) or from the lower right (as if held in the right hand) toward the target tissue. These concepts are... Figures 16A-16B As shown in the figure, Figures 16A-16B This is a simplified diagram of the different end effector positions and orientations within the workspace according to some embodiments. Figure 16A View 1610 of a workspace that can be captured by an imaging device, the placement of which is determined during operation 1540 and its end effector is introduced into the workspace using a first set of port positions, is shown. In some examples, view 1610 can be obtained by placing the imaging device at a known imaging distance from the target tissue, which is positioned at the center of view 1610. Two planes (in Figure 16A The projection lines (shown in the diagram) indicate the main diagonals 1612 and 1614 of view 1610 and can roughly correspond to the ideal approach direction of surgical instruments and / or end effectors. Figure 16AAlso shown is a first end effector 1620 approaching the center point of the workspace along the insertion axis 1625. The difference between the insertion axis 1625 and the main diagonal 1612 of view 1610 is shown as an angle 1629. Figure 16A A second end effector 1630 is also shown near the center point of the workspace along the insertion axis 1635. The difference between the insertion axis 1635 and the main diagonal 1614 of view 1610 is shown as an angle 1639.

[0124] As another example, Figure 16B Another view 1660 of the workspace is shown, which can be captured by an imaging device whose placement is determined during operation 1540, and whose end effector is introduced into the workspace using a second set of port positions. In some examples, view 1660 can be obtained by placing the imaging device at a known imaging distance from the target tissue, with the target tissue positioned at the center of view 1660. Two planes (in Figure 16B The projection lines (shown in the diagram) indicate the main diagonals 1662 and 1664 of view 1660 and can roughly correspond to the ideal approach direction of surgical instruments and / or end effectors. Figure 16B The first end effector 1670 is also shown, which is close to the center point of the workspace along the insertion axis 1675. The difference between the insertion axis 1675 and the main diagonal 1662 of view 1660 is shown as an angle 1679. Figure 16B A second end effector 1680 is also shown near the center point of the workspace along the insertion axis 1685. The difference between the insertion axis 1685 and the main diagonal 1664 of view 1660 is shown as angle 1689.

[0125] Because angles 1629 and 1639 are smaller than angles 1679 and 1689, they indicate that end effectors 1620 and 1630 are closer to the center point of the workspace more naturally than end effectors 1670 and 1680. Therefore, the first set of port positions associated with end effectors 1620 and 1630 is considered more anthropomorphic than the second set of port positions and is assigned a higher anthropomorphic metric. In some examples, the anthropomorphic metric of the port position can be determined using Equation 15 or Equation 16, where b corresponds to the angle between the insertion axis of the end effector from the port position and the main diagonal.

[0126] Anthropomorphism metric = (90-b) / 90 Equation 15

[0127] Anthropomorphism metric = (180-b) / 180 Equation 16

[0128] In some examples, additional information about the imaging device obtained during operation 1540 (e.g., imaging device type, aspect ratio, field of view, working distance, etc.) can be used to help locate views 1610 and / or 1660 and determine the orientation of the main diagonals 1612, 1614, 1662 and / or 1664.

[0129] In some embodiments, the anthropomorphic measure may also take into account human factors, such as the operator's dominant hand preference. In some examples, when an operator indicates a preference for using a particular surgical instrument in a particular hand, the angle used for the anthropomorphic measure should be determined using the main diagonal of that hand (e.g., the main diagonal 1612 and / or 1662 of the right-hand surgical instrument and / or the main diagonal 1614 and / or 1664 of the left-hand surgical instrument), even if the other main diagonal may have a small angle relative to the insertion axis of the surgical instrument. In some examples, right-hand and left-hand anthropomorphic measures can be determined for port location, such that right-hand and left-hand assessments can be considered during the remainder of method 1500.

[0130] Return to reference Figure 15 In operations 1560 and 1520, each of the possible combinations of port positions identified was iterated to evaluate the suitability of the combination of port positions for the procedure. When the procedure is performed using two surgical instruments, each combination of port positions includes two port positions. More generally, when the procedure is performed using n surgical instruments, then each combination of port positions includes n port positions. The analyses of operations 1562 and 1564 were repeated while considering each possible combination of port positions to determine an aggregate scoring metric that can be used to characterize the suitability of the combination of port positions for use with the intended procedure.

[0131] In operation 1562, a collision metric is determined for the combination of port locations. A collision metric is a kinematic measurement that provides an indication of the likelihood or impossibility of a collision occurring in portions of surgical instruments, manipulators, and / or repositionable arms located near the port locations in the combination. In some examples, the collision metric may be determined based on the amount of overlap between collision volumes determined for each port location in the combination during operation 1554. Where there is more overlap in the collision volumes, the likelihood of a collision increases and the collision metric decreases. In some examples, the collision metric may be determined based on the percentage overlap between each collision volume and other collision volumes. In some examples, the percentage overlap between a collision volume and other collision volumes is determined based on the ratio of the total collision volume overlapping with other collision volumes to the total collision volume. In some examples, this can be translated into an overlap metric as shown in Equation 5.

[0132] Overlap metric = 1 - (CV of overlap) / (total CV) Equation 5

[0133] When a combination of port locations includes two port locations, the overlap metric can be used as a collision metric. When a combination of port locations includes three or more port locations, the collision metric can be determined by aggregating the overlap metrics for each corresponding collision volume. In some examples, any trigonometric norm function (such as minimum, multiplication, etc.) can be used to aggregate the overlap metric for each corresponding collision volume.

[0134] In operation 1564, an aggregate scoring metric is determined for combinations of port locations. In some examples, the aggregate scoring metric can be determined by aggregating the reachability metric for each port location in the combination, the anthropomorphic metric for each port location in the combination, and the collision metric for the combination. In some examples, a weighted sum can be used to perform the aggregation, where the weights are pre-assigned and / or adjustable by the operator. In some examples, zero weights can be used to omit the corresponding metrics from the aggregation. In some examples, the aggregation can be determined by using any trigonometric norm function (e.g., minimum, multiplication, etc.) to combine the metrics. In some examples, the aggregate scoring metric can be used to indicate the fitness of a combination of port locations relative to other combinations of port locations.

[0135] In operation 1570, one or more combinations of port locations are displayed to the operator. For example, system 100 may instruct a display device to display a graphical representation of one or more combinations of port locations.

[0136] In some examples, any suitable display device, including tablets, computer screens, simulators, etc., can be used to display a combination of port locations and corresponding evaluations to the operator. In some examples, the combination of port locations and corresponding evaluations can be displayed as a 2D projection, a 3D image on a stereoscopic display, etc. In some examples, the order in which combinations of port locations can be displayed can be based on their relative aggregated rating metrics, with the highest-rated combination displayed first. In some examples, one or more lists, menus, etc., can be used to allow the operator to select from combinations of evaluations. In some examples, the corresponding evaluations can be displayed as one or more lines of text indicating a defined value for each of the accessibility, anthropomorphism, and / or collision metrics, as well as the aggregated rating metric. In some examples, one or more lines of text can indicate the relative weight of each metric and optionally provide the operator with a mechanism to adjust the weights. In some examples, one or more mechanisms for adding additional constraints (e.g., human factor constraints, such as the dominant hand of one of the surgical instruments) can also be provided.

[0137] In operation 1580, the port location selection is received from the operator. In some examples, the port location selection can be made by indicating (e.g., using procedure 1570) that the current combination of port locations being displayed is the selected combination. In some examples, other selection mechanisms can be used, such as selecting from a list, etc.

[0138] In procedure 1590, the operator is provided with guidance to place the port at a port location selected during procedure 1580. In some examples, the guidance for placing the port at one of the selected port locations may include one or more laser targets projected onto the port location, pointing the port location using a manipulator, projection onto the patient, tactile guidance for manual positioning of the manipulator, augmented reality overlay on a stereoscopic image of the patient, etc.

[0139] In some examples, system 100 may perform operation 202 by identifying placement positions for the manipulator arm of computer-assisted surgical system 204 based on external body wall data 206 and internal depth data 208. System 100 may then instruct computer-assisted surgical system 204 to configure the manipulator arm in the placement position. These operations may be performed in any suitable manner. For example, a placement position may be selected such that the manipulator arm does not come into contact with the patient and / or another manipulator arm while surgical instruments attached to the manipulator arm are inserted into the patient and / or being used in the patient's body. In some examples, the placement position may be further determined based on kinematic data generated by computer-assisted surgical system 204. In some examples, the placement position is determined by determining the position of one or more placement joints of the manipulator arm.

[0140] Figure 17 An exemplary method 1700 that can be performed by an operations management system (e.g., system 100 and / or any implementation thereof) is illustrated. Although Figure 17 The illustration shows an exemplary operation according to one embodiment, but other embodiments may omit, add, reorder, and / or modify it. Figure 17 Any of the operations shown.

[0141] In operation 1702, the operation management system obtains external body wall data representing a three-dimensional model of the patient's external body wall. Operation 1702 can be performed in any of the manner described herein.

[0142] In operation 1704, the operation management system obtains internal depth data representing a depth map for the patient's internal space. Operation 1704 can be performed in any of the manner described herein.

[0143] In operation 1706, the operation management system performs operations associated with a computer-assisted surgical system based on external body wall data and internal depth data. This computer-assisted surgical system is configured to perform procedures concerning the patient. Operation 1706 can be performed in any of the manner described herein.

[0144] In some examples, a non-transitory computer-readable medium may be provided for storing computer-readable instructions, based on the principles described herein. When executed by a processor of a computing device, the instructions may direct the processor and / or the computing device to perform one or more operations, including one or more operations described herein. Such instructions may be stored and / or transmitted using any of a variety of known computer-readable media.

[0145] As used herein, a non-transitory computer-readable medium may include any non-transitory storage medium that contributes to providing data (e.g., instructions) that can be read and / or executed by a computing device (e.g., by a processor of the computing device). For example, a non-transitory computer-readable medium may include (but is not limited to) any combination of non-volatile storage media and / or volatile storage media. Exemplary non-volatile storage media include (but are not limited to) read-only memory, flash memory, solid-state drives, magnetic storage devices (e.g., hard disks, floppy disks, magnetic tapes, etc.), ferroelectric random access memory (“RAM”), and optical discs (e.g., compact discs, digital video discs, Blu-ray discs, etc.). Exemplary volatile storage media include (but are not limited to) RAM (e.g., dynamic RAM).

[0146] Figure 17 An exemplary computing device 1700 is illustrated, which may be specifically configured to execute one or more programs described herein. Any systems, computing devices, and / or other components described herein may be implemented by computing device 1700.

[0147] like Figure 17 As shown, computing device 1700 may include communication interface 1702, processor 1704, storage device 1706, and input / output (“I / O”) module 1708, which are communicatively coupled to each other via communication infrastructure 1710. Although Figure 17 An exemplary computing device 1700 is shown, but Figure 17 The components illustrated are not intended to be limiting. Additional or alternative components may be used in other embodiments. A more detailed description will now follow. Figure 17 Components of the computing device 1700 shown.

[0148] Communication interface 1702 can be configured to communicate with one or more computing devices. Examples of communication interface 1702 include (but are not limited to) wired network interfaces (e.g., network interface cards), wireless network interfaces (e.g., wireless network interface cards), modems, audio / video connections, and any other suitable interfaces.

[0149] Processor 1704 generally refers to any type or form of processing unit capable of processing data and / or interpreting, executing, and / or directing the execution of one or more of the instructions, programs, and / or operations described herein. Processor 1704 can perform operations by executing computer-executable instructions 1712 (e.g., application programs, software, code, and / or other executable data instances) stored in storage device 1706.

[0150] Storage device 1706 may include one or more data storage media, devices, or configurations, and may take any type, form, and combination of data storage media and / or devices. For example, storage device 1706 may include (but is not limited to) any combination of non-volatile media and / or volatile media described herein. Electronic data (including the data described herein) may be stored temporarily and / or permanently in storage device 1706. For example, data representing computer-executable instructions 1712 configured to boot processor 1704 to perform any of the operations described herein may be stored within storage device 1706. In some examples, data may be arranged in one or more databases residing within storage device 1706.

[0151] I / O module 1708 may include one or more I / O modules configured to receive user input and provide user output. I / O module 1708 may include any hardware, firmware, software, or a combination thereof that supports input and output capabilities. For example, I / O module 1708 may include hardware and / or software for capturing user input, including (but not limited to) a keyboard or keypad, a touchscreen component (e.g., a touchscreen display), a receiver (e.g., an RF or infrared receiver), a motion sensor, and / or one or more input buttons.

[0152] I / O module 1708 may include one or more devices for presenting output to a user, including (but not limited to) a graphics engine, a display (e.g., a screen), one or more output drivers (e.g., display drivers), one or more audio speakers, and one or more audio drivers. In some embodiments, I / O module 1708 is configured to provide graphical data to the display for presentation to the user. The graphical data may represent one or more graphical user interfaces and / or any other graphical content that may be available for a particular implementation.

[0153] Various exemplary embodiments have been described in the foregoing description with reference to the accompanying drawings. However, it will be apparent that various modifications and changes can be made thereto, and additional embodiments can be implemented without departing from the scope of the invention as set forth in the appended claims. For example, certain features of one embodiment described herein may be combined with or substituted with features of another embodiment described herein. Accordingly, the specification and drawings are to be considered illustrative rather than restrictive.

Claims

1. A system comprising: Memory that stores instructions; and A processor, communicatively coupled to the memory and configured to execute the instructions to: By guiding the imaging device to scan the patient's external body wall when the imaging device is located outside the patient, external body wall data is obtained, which represents a three-dimensional model of the patient's external body wall and is based on the external depth data of the external body wall; By guiding the imaging device to scan the patient's internal space, internal depth data is obtained, which represents a depth map of the patient's internal space; and Operations associated with a computer-assisted surgical system are performed based on the external body wall data and the internal depth data. This computer-assisted surgical system is configured to execute procedures related to the patient. The distal end of the imaging device can be inserted into the patient's internal space. The imaging device described herein includes a depth sensor and a visible light camera, and The depth sensor is implemented by a time-of-flight sensor.

2. The system of claim 1, wherein obtaining the internal depth data includes guiding the depth sensor to acquire the internal depth data when the depth sensor in the imaging device is aimed at the internal space through a camera port formed through the patient's external body wall.

3. The system according to claim 2, wherein obtaining the external body wall data comprises: The depth sensor in the imaging device is guided to acquire external depth data by scanning the external body wall when the imaging device is outside the patient; Receive the external depth data from the depth sensor; as well as The external depth data is used as the external body wall data of the three-dimensional model representing the patient's external body wall.

4. The system according to claim 3, wherein: When the depth sensor acquires the external depth data and the internal depth data, the imaging device is attached to the manipulator arm of the computer-assisted surgical system; The computer-assisted surgical system is configured to generate kinematic data for the imaging device when the depth sensor acquires the external depth data and the internal depth data; The processor is further configured to execute the instructions to register the external body wall data with the internal depth data based on the kinematic data; and The operation is performed based on the registration of the external body wall data and the internal depth data.

5. The system of claim 2, wherein the visible light camera is configured to acquire a visible light image of the interior space.

6. The system of claim 5, wherein the visible light camera is a first visible light camera, and wherein obtaining the external body wall data comprises: Guide the first visible light camera to acquire a first image of the external body wall; The imaging device includes a second visible light camera to acquire a second image of the external body wall; as well as The external depth data is generated based on the first image and the second image.

7. The system of claim 6, wherein obtaining the external body wall data further comprises scanning the patient's external body wall using at least one of a computer-aided computed tomography scanner (CT scanner), a magnetic resonance imaging (MRI) device, an ultrasound device, or a three-dimensional scanning device.

8. The system of claim 1, wherein performing the operation includes identifying port locations on the patient's external body wall based on the external body wall data and the internal depth data, and the computer-assisted surgical system inserts surgical instruments into the patient's internal space through the port locations.

9. The system of claim 8, wherein identifying the port location includes using the external body wall data and the internal depth data to identify the port location on the external body wall, the port location allowing the surgical instrument to enter the structure within the internal space through the port location while avoiding collision with additional surgical instruments.

10. The system of claim 8, wherein identifying the port location is further based on kinematic data generated by the computer-assisted surgical system.

11. The system of claim 8, wherein the surgical instruments are attached to the manipulator arm of the computer-assisted surgical system.

12. The system of claim 11, wherein identifying the port location includes identifying the port location such that the surgical instrument is configured to enter a structure within the internal space, and the manipulator arm does not collide with different manipulator arms.

13. The system of claim 11, wherein identifying the port location includes identifying the port location such that the surgical instrument and the manipulator arm avoid unintentional contact with the patient.

14. The system of claim 8, wherein the processor is further configured to execute the instructions to direct the display device to display a graphical representation of the port location.

15. The system according to claim 8, wherein: The processor is further configured to execute the instructions to determine at least one of the following for a candidate port location: Accessibility metric, which indicates the ability of the surgical instrument to reach a target structure located within the patient's internal space using candidate port locations. Anthropomorphic measurements indicate that the user can easily manipulate the surgical instruments introduced into the patient's internal space through the candidate port locations. The collision volume of the portion of the computer-assisted surgical system near the candidate port location, the collision volume corresponding to the volume swept by the portion of the computer-assisted surgical system near the candidate port location, or Collision metric, which indicates the likelihood of a collision between portions of the computer-assisted surgical system near the candidate port location; and The port location is further identified based on at least one of the reachability metric, the anthropomorphism metric, the collision volume, or the collision metric.

16. The system of claim 1, wherein performing the operation includes identifying a placement position for the manipulator arm of the computer-assisted surgical system based on the external body wall data and the internal depth data.

17. The system of claim 16, wherein identifying the placement location is further based on kinematic data generated by the computer-assisted surgical system.

18. The system of claim 16, wherein the processor is further configured to execute the instructions to instruct the computer-assisted surgical system to position the manipulator arm in the placement position.

19. The system of claim 1, wherein both the external depth data and the internal depth data are generated during a surgical procedure during which the computer-assisted surgical system is configured to perform the procedure concerning the patient.

20. A non-transitory computer-readable medium storing instructions, which, when executed, instruct a processor of a computing device to: By guiding the imaging device to scan the patient's external body wall when the imaging device is located outside the patient, external body wall data is obtained, which represents a three-dimensional model of the patient's external body wall and is based on the external depth data of the external body wall; By guiding the imaging device to scan the patient's internal space, internal depth data is obtained, which represents a depth map of the patient's internal space; as well as Operations associated with a computer-assisted surgical system are performed based on the external body wall data and the internal depth data. This computer-assisted surgical system is configured to execute procedures related to the patient. The distal end of the imaging device can be inserted into the patient's internal space. The imaging device described herein includes a depth sensor and a visible light camera, and The depth sensor is implemented by a time-of-flight sensor.

21. The non-transitory computer-readable medium of claim 20, wherein obtaining the internal depth data includes guiding the depth sensor to acquire the internal depth data as the depth sensor in the imaging apparatus is aimed at the internal space through a camera port formed through the patient's external body wall.

22. The non-transitory computer-readable medium of claim 21, wherein obtaining the external body wall data comprises: The depth sensor in the imaging device is guided to acquire external depth data by scanning the external body wall when the imaging device is outside the patient; Receive the external depth data from the depth sensor; as well as The external depth data is used as the external body wall data of the three-dimensional model representing the patient's external body wall.

23. The non-transitory computer-readable medium according to claim 22, wherein: When the depth sensor acquires the external depth data and the internal depth data, the imaging device is attached to the manipulator arm of the computer-assisted surgical system; The computer-assisted surgical system is configured to generate kinematic data for the imaging device when the depth sensor acquires the external depth data and the internal depth data; The non-transitory computer-readable medium further includes instructions, when executed, to instruct the processor to: register the external body wall data with the internal depth data based on the kinematic data; and The operation is performed based on the registration of the external body wall data and the internal depth data.

24. The non-transitory computer-readable medium of claim 21, wherein the visible light camera is configured to acquire a visible light image of the interior space.

25. The non-transitory computer-readable medium of claim 21, wherein the visible light camera is a first visible light camera, and wherein obtaining the external body wall data comprises: Guide the first visible light camera to acquire a first image of the external body wall; The imaging device includes a second visible light camera to acquire a second image of the external body wall; as well as The external depth data is generated based on the first image and the second image.

26. The non-transitory computer-readable medium of claim 25, wherein obtaining the external body wall data further comprises scanning the patient's external body wall using at least one of a computer-aided computed tomography scanner (CT scanner), a magnetic resonance imaging (MRI) device, an ultrasound device, or a three-dimensional scanning device.

27. The non-transitory computer-readable medium of claim 20, wherein performing the operation includes identifying port locations on the patient's external body wall based on the external body wall data and the internal depth data, the computer-assisted surgical system being capable of inserting surgical instruments into the patient's internal space through the port locations.

28. The non-transitory computer-readable medium of claim 27, wherein identifying the port location includes using the external body wall data and the internal depth data to identify a position on the external body wall for the port location, the port location allowing the surgical instrument to enter the structure within the internal space through the port location while avoiding collision with additional surgical instruments.

29. The non-transitory computer-readable medium of claim 27, wherein identifying the port location is further based on kinematic data generated by the computer-assisted surgical system.

30. The non-transitory computer-readable medium of claim 27, wherein the surgical instruments are attached to the manipulator arm of the computer-assisted surgical system.

31. The non-transitory computer-readable medium of claim 30, wherein identifying the port location includes identifying the port location such that the surgical instrument is configured to enter a structure within the internal space, and the manipulator arm does not collide with different manipulator arms.

32. The non-transitory computer-readable medium of claim 30, wherein identifying the port location includes identifying the port location such that the surgical instrument and the manipulator arm avoid unintentional contact with the patient.

33. The non-transitory computer-readable medium of claim 27, further comprising, when executed, instructions to instruct the processor to: instruct a display device to display a graphical representation of the port location.

34. The non-transitory computer-readable medium of claim 27, further comprising instructions that, when executed, direct the processor to perform the following: For candidate port locations, at least one of the following must be determined: Accessibility metric, which indicates the ability of the surgical instrument to reach a target structure located within the patient's internal space using candidate port locations. Anthropomorphic measurements indicate that the user can easily manipulate the surgical instruments introduced into the patient's internal space through the candidate port locations. The collision volume of the portion of the computer-assisted surgical system near the candidate port location, the collision volume corresponding to the volume swept by the portion of the computer-assisted surgical system near the candidate port location, or A collision metric, which indicates the likelihood of a collision between portions of the computer-assisted surgical system near the candidate port location. The identification of the port location is further based on at least one of the reachability metric, the anthropomorphism metric, the collision volume, or the collision metric.

35. The non-transitory computer-readable medium of claim 20, wherein performing the operation includes identifying a placement position for the manipulator arm of the computer-assisted surgical system based on the external body wall data and the internal depth data.

36. The non-transitory computer-readable medium of claim 35, wherein identifying the placement location is further based on kinematic data generated by the computer-assisted surgical system.

37. The non-transitory computer-readable medium of claim 35, further comprising, when executed, instructions to direct the processor to: instruct the computer-assisted surgical system to position the manipulator arm in the placement position.

Images