Determining contoured physical distance within subject based on deformable three-dimensional model

Abstract

An illustrative dynamic measurement system may be configured to generate a deformable 3D model of a scene based on an image of the scene, identify a first point located on an anatomical object in the scene, and determine a dynamic measurement representative of a contoured physical distance between the first point and a second point in the scene based on the deformable 3D model. The dynamic measurements may be dynamically updated as one or more objects move within the scene.

Description

[0001] Background Information This application claims priority to U.S. Provisional Patent Application No. 63 / 405,561, filed on September 12, 2022, the contents of which are incorporated herein by reference in their entirety. Background Art

[0002] During a medical procedure (such as a procedure utilizing a computer-assisted medical system), an imaging device may be used to provide images (e.g., stereoscopic video) of internal anatomical structures within a subject (e.g., to a surgeon). In some cases, it may be desirable to measure various distances associated with the internal anatomical structures of the procedure. For example, it may be desirable to measure the size of a hernia within a subject so that a mesh patch may be appropriately sized to fit the hernia. As another example, it may be desirable to determine how far the tip of a surgical instrument is from tissue within the subject.

[0003] Unfortunately, direct physical access to internal anatomical structures may not be possible (e.g., during minimally invasive medical procedures), which may make such measurements difficult to accurately determine. For example, the surface contours of the internal anatomical structures may obscure portions of the internal anatomical structures from the imaging device and / or portions of the internal anatomical structures may be outside the field of view of the imaging device. Summary of the invention

[0004] The following description presents a simplified overview of one or more aspects of the systems and methods described herein. This overview is not an extensive review of all contemplated aspects, is not intended to identify key or important elements of all aspects, nor is it intended to outline the scope of any or all aspects. Its purpose is to present one or more aspects of the systems and methods described herein as a prelude to the specific embodiments presented below.

[0005] An illustrative system includes a memory storing instructions and one or more processors communicatively coupled to the memory. The one or more processors may be configured to execute the instructions to perform a process including: generating a deformable three-dimensional (3D) model of a scene based on an image of the scene; identifying a first point located on an anatomical object in the scene; and determining a dynamic measurement value representing a contour physical distance between the first point and a second point in the scene based on the deformable 3D model. The dynamic measurement value may be dynamically updated as one or more objects within the scene move.

[0006] An illustrative method includes: generating, by at least one computing device and based on an image of the scene, a deformable 3D model of a scene; identifying, by at least one computing device, a first point on an anatomical object in the scene; and determining, by at least one computing device and based on the deformable 3D model, a dynamic measurement representing a contoured physical distance between the first point and a second point in the scene, wherein the dynamic measurement is dynamically updated as one or more objects within the scene move.

[0007] An illustrative non-transitory computer-readable medium may store instructions that, when executed, direct a processor of a computing device to perform a process comprising: generating a deformable 3D model of a scene based on an image of the scene; identifying a first point on an anatomical object in the scene; and determining a dynamic measurement representing a contoured physical distance between the first point and a second point in the scene based on the deformable 3D model, wherein the dynamic measurement is dynamically updated as one or more objects within the scene move. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The accompanying drawings illustrate various embodiments and are part of the specification. The illustrated embodiments are examples only and do not limit the scope of the present disclosure. Throughout the drawings, the same or similar reference numerals represent the same or similar elements.

[0009] Figure 1 An illustrative embodiment including a dynamic measurement system is shown.

[0010] Figure 2 Another illustrative embodiment including a dynamic measurement system is shown.

[0011] Figure 3 An illustrative method of operating a dynamic measurement system is shown.

[0012] Figure 4 Another illustrative method of operating a dynamic measurement system is shown.

[0013] Figure 5A and Figure 5B An illustrative implementation of generating a deformable 3D model using a dynamic measurement system is shown.

[0014] Fig. 6A and Figure 6B An illustrative implementation of determining dynamic measurements using a dynamic measurement system is shown.

[0015] Figure 7 An illustrative embodiment of a display that may be generated using a dynamic measurement system is shown.

[0016] Figure 8 An illustrative computer-assisted medical system that may incorporate a dynamic measurement system is shown.

[0017] Fig. 9 An illustrative computing system in accordance with the principles described herein is shown. DETAILED DESCRIPTION

[0018] An illustrative dynamic measurement system can be configured to determine a dynamic measurement of a contour physical distance between (multiple) points within a scene based on a deformable 3D model of the scene. For example, the dynamic measurement system can be configured to generate a deformable 3D model of the scene based on an image of the scene (e.g., a region within a subject of a medical procedure) (e.g., captured by an imaging device). The dynamic measurement system can also be configured to identify a first point located on an anatomical object in the scene and determine a dynamic measurement value representing a contour physical distance between the first point and a second point in the scene based on the deformable 3D model.

[0019] In some embodiments, a deformable 3D model can be generated in real time based on an image of a scene during a medical procedure. This can allow the deformable 3D model to depict the movement of one or more objects in the scene as they deform (e.g., due to breathing and / or forces applied by instruments during the medical procedure), which can cause the physical distance of the contour to change. Thus, based on the deformable 3D model, dynamic measurements can be dynamically updated as one or more objects in the scene move.

[0020] The principles described herein can produce improved dynamic measurements compared to conventional techniques that are not based on deformable 3D models, and provide other benefits described herein. For example, determination of dynamic measurements based on a deformable 3D model can allow for more accurate and / or efficient determination of dynamic measurements. To illustrate, determination of dynamic measurements based on a deformable 3D model can take into account surface contours of anatomical objects, which can reduce occlusion issues caused by surface contours, and / or take into account anatomical objects located outside the field of view of an imaging device, which can increase the area used to determine dynamic measurements. In addition, determination of dynamic measurements based on a deformable 3D model can allow dynamic measurements to be dynamically updated, such as when one or more anatomical objects within a scene are deformed.

[0021] Figure 1An exemplary embodiment 100 is shown, which is configured to determine a dynamic measurement value representing a physical distance along a contour of the scene based on a deformable 3D model of the scene. As shown, the embodiment 100 includes a dynamic measurement system 102, which is configured to generate a deformable 3D model of the scene based on an image of the scene, identify a first point on an anatomical object located in the scene, and determine a dynamic measurement value representing a physical distance along a contour between the first point and a second point in the scene based on the deformable 3D model. The embodiment 100 may include additional or alternative components that may serve a particular embodiment. In some examples, the embodiment 100 or certain components of the embodiment 100 may be implemented by a computer-assisted medical system.

[0022] The dynamic measurement system 102 may be implemented by one or more computing devices and / or computer resources (e.g., processors, memory devices, storage devices, etc.) that may serve a particular implementation. As shown, the dynamic measurement system 102 may include, but is not limited to, a memory 104 and a processor 106 that are selectively and communicatively coupled to each other. The memory 104 and the processor 106 may each include or be implemented by computer hardware configured to store and / or process computer software. Figure 1 Various other components of computer hardware and / or software not explicitly shown in FIG. 1 may also be included in dynamic measurement system 102. In some examples, memory 104 and / or processor 106 may be distributed among multiple devices and / or multiple locations as may serve a particular implementation.

[0023] The memory 104 may store and / or otherwise maintain executable data used by the processor 106 to perform any of the functions described herein. For example, the memory 104 may store instructions 108 that may be executed by the processor 106. The memory 104 may be implemented by one or more memories or storage devices (including any memory or storage device described herein that is configured to store data in a temporary or non-temporary manner). The instructions 108 may be executed by the processor 106 to cause the dynamic measurement system 102 to perform any of the functions described herein. The instructions 108 may be implemented by any suitable application, software, code, and / or other executable data instance. In addition, the memory 104 may also maintain any other data that is accessed, managed, used, and / or transmitted by the processor 106 in a particular embodiment.

[0024] The processor 106 may be implemented by one or more computer processing devices including a general purpose processor (e.g., a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor, etc.), a special purpose processor (e.g., an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), etc.), an image signal processor, or the like. Using the processor 106 (e.g., when the processor 106 is directed to perform operations represented by instructions 108 stored in the memory 104), the dynamic measurement system 102 may perform various operations described herein.

[0025] Figure 2 Another illustrative embodiment 200 is shown, which is configured to determine a dynamic measurement value representing a physical distance along the outline of the scene based on a deformable 3D model of the scene. As shown, the embodiment 200 includes a dynamic measurement system 202 that is communicatively coupled to an imaging device 204 and a user interface 206 (e.g., wired and / or wirelessly). The embodiment 200 may include additional or alternative components that may serve a particular embodiment. In some examples, the embodiment 200 or certain components of the embodiment 200 may be implemented by a computer-assisted medical system.

[0026] The imaging device 204 may be implemented by an endoscope or other suitable device configured to capture and output images (e.g., images, videos, image frame sequences, etc.) of the scene 208. In some embodiments, the imaging device 204 may include, but is not limited to, one or more of the following: a video imaging device, an infrared imaging device, a visible light imaging device, an invisible light imaging device, an intensity imaging device (e.g., a color, grayscale, black and white imaging device), a depth imaging device (e.g., a stereo imaging device, a time-of-flight imaging device, an infrared imaging device, a red, green, and blue (RGB) imaging device, a red, green, and blue and depth (RGB-D) imaging device, a light detection and ranging (LIDAR) imaging device, etc.).

[0027] In some implementations, the imagery may include image data (e.g., color, grayscale, saturation, intensity, brightness, depth, etc.) captured by the imaging device 204. In some cases, the image data may be associated with data points represented in a common coordinate system, such as 3D voxels or two-dimensional (2D) pixels of an image captured by the imaging device 204. In some implementations, the imaging device 204 may be moved relative to the scene 208 to capture images of the scene 208 at different viewpoints.

[0028] Scene 208 may include an environment (e.g., an area within a subject of a medical procedure) and / or one or more objects within the environment. For example, scene 208 may include anatomical object 210. Anatomical object 210 may include an object associated with a subject (e.g., a body of a living animal, a human or animal cadaver, a portion of a human or animal anatomical structure, tissue removed from a human or animal anatomical structure, a non-tissue artifact, a training model, etc.). In some embodiments, anatomical object 210 may include tissue of the subject (e.g., an organ, soft tissue, connective tissue, etc.). Other non-anatomical objects may also be included within scene 208, such as physical tools used in a medical procedure (e.g., a scalpel, scissors, forceps, clamps, etc.) and / or other objects (e.g., staplers, meshes, sponges, etc.).

[0029] The dynamic measurement system 202 may implement or be similar to the dynamic measurement system 102, and may be configured to receive images of the scene 208 from the imaging device 204. In some embodiments, the dynamic measurement system 202 may be configured to fuse images of the scene 208 captured by the imaging device 204 at different viewpoints of the scene 208. In some examples, the fusion may include merging aligned (or overlapping) voxels or pixels, such as by blending the intensity and / or depth values ​​of the aligned voxels or pixels. The blending may include weighted blending, in which the blended data points are weighted based on one or more factors, such as which camera of the stereoscopic device has the best view of the data point (e.g., by weighting the data captured by the camera with the best viewing angle more heavily). The fusion may additionally or alternatively include stitching together non-overlapping voxels or pixels, such as by stitching together the images along non-overlapping boundaries of the images. Therefore, the fusion of images at different viewpoints may allow the image of the scene 208 to include a larger area than the single field of view of the imaging device 204.

[0030] As shown, the dynamic measurement system 202 includes a deformable 3D model generator 212, which is configured to generate a deformable 3D model 214 based on an image of a scene 208. For example, the deformable 3D model generator 212 can be configured to generate a point cloud having a plurality of nodes, the plurality of nodes representing surface points on one or more objects (e.g., anatomical object 210) within the scene 208 as depicted in the image captured by the imaging device 204. The deformable 3D model generator 212 can also be configured to generate vertices associated with 3D positions corresponding to the 3D positions of the plurality of nodes from the image. In the case where the plurality of nodes are based on 2D imagery, the deformable 3D model generator 212 can be configured to determine depths associated with the plurality of nodes, such as by processing a stereo image captured by the imaging device 204. Additionally or alternatively, a depth map of the scene 208 can be generated using a depth sensor.

[0031] The deformable 3D model generator 212 may also be configured to deform the deformable 3D model 214 over time as one or more objects within the scene 208 move. For example, an anatomical object 210 within the scene 208 may deform during a medical procedure. Such deformation may be caused by expansion and / or contraction of the anatomical object 210 (e.g., when a subject of the medical procedure breathes), by movement of another object in the scene 208 (e.g., an object physically connected to the anatomical object 210), and / or by a force applied to the anatomical object 210 (e.g., by a physical tool, a person's finger, etc.). In addition to or in lieu of the anatomical object 210, other non-anatomical objects may move within the scene 208 (e.g., a physical tool may move relative to the anatomical object 210).

[0032] To illustrate, the 3D positions of the vertices of the deformable 3D model 214 may track the 3D positions of the plurality of nodes associated with the vertices as the 3D positions of the plurality of nodes in the imagery captured by the imaging device 204 are updated as one or more objects within the scene 208 move. This may allow the deformable 3D model 214 to deform over time as one or more objects within the scene 208 move. In some implementations, the deformable 3D model generator 212 may be configured to detect deformation of the deformable 3D model 214, such as by comparing the 3D positions of the vertices of the deformable 3D model 214 at two or more different points in time. Additionally or alternatively, a deformable or non-deformable first 3D model may be generated at a first point in time, and a deformable or non-deformable second 3D model may be generated at a second point in time different from the first point in time, such that the first 3D model and the second 3D model may be compared to each other to detect deformation.

[0033] In some implementations, the deformable 3D model generator 212 may use a simultaneous localization and mapping (SLAM) heuristic to build and / or update a map of the scene 208 while keeping track of the locations of objects within the scene 208. For example, the SLAM heuristic may be configured to generate a point cloud having a plurality of nodes representing surface points on one or more objects within the scene 208, and the SLAM heuristic may be configured to derive and / or associate vertices of the deformable 3D model 214 with 3D locations corresponding to the 3D locations of the plurality of nodes as the imaging device 204 views the scene 208 in real time. The SLAM heuristic may also be configured to derive and / or associate additional vertices of the deformable 3D model 214 with 3D locations corresponding to the 3D locations of the additional nodes as the imaging device 204 moves relative to the scene 208 to capture additional areas of the scene 208, while also keeping track of the 3D locations of previous vertices of the deformable 3D model 214 and the 3D locations of the associated previous nodes as one or more objects within the scene 208 move and / or deform. In some implementations, the SLAM heuristics may be configured to track the pose of the imaging device 204 as it moves relative to the scene 208 (eg, using vision software).

[0034] Other suitable configurations may also be used to generate and / or update the deformable 3D model 214 as one or more objects within the scene 208 move. For example, the deformable 3D model generator 212 may be configured to generate the deformable 3D model 214 based on pre-operative images of the scene 208. Additionally or alternatively, the movement of the one or more objects within the scene 208 may be determined based on kinematic data representing the movement of the one or more objects over time. For example, the kinematic data may be generated by or associated with a computer-assisted medical system that is communicatively coupled to the one or more objects (e.g., a physical tool).

[0035] The dynamic measurement system 202 also includes a contour physical distance module 216 configured to determine a dynamic measurement representing a contour physical distance along the scene 208 based on the deformable 3D model 214 and output the dynamic measurement to the user interface 206. For example, the contour physical distance may represent a distance between two or more points within the scene 208, which may extend over a physical surface of one or more objects within the scene 208. The dynamic measurement may be represented by any suitable value, such as a discrete value (e.g., a distance, a range, a percentage, etc.) representing the contour physical distance.

[0036] For illustration, the outline physical distance module 216 may be configured to determine a 3D outline that may extend along the surface of one or more objects of the deformable 3D model 214 and connect two or more points within the scene 208, such that the 3D outline may represent the outline physical distance between the two or more points. In some embodiments, the two or more points may be associated with vertices of the deformable 3D model 214. In addition, the outline physical distance module 216 may be configured to identify one or more additional vertices of the deformable 3D model 214 between the two or more points on the 3D outline. This may allow the outline physical distance module 216 to determine the intermediate distance of each segment of the linear segmented route that passes through the 3D positions of each adjacent vertex of the deformable 3D model 214. Based on the intermediate distances, the outline physical distance module 216 may calculate the dynamic measurement value as the sum of the intermediate distances. The sum of the intermediate distances may provide an estimate of the precise outline physical distance, which may become more accurate as more vertices and / or intermediate distances are defined. Additionally or alternatively, the outline physical distance module 216 may determine the direct point-to-point distance between the 3D positions of each of the two or more points.

[0037] In some embodiments, the dynamic measurement representing the physical contour distance may change as one or more objects move within the scene 208. Therefore, the contour physical distance module 216 can be configured to dynamically update the dynamic measurement as one or more objects move. For example, the contour physical distance module 216 can be configured to recalculate the intermediate distance between the 3D positions of the identified vertices of the deformable 3D model 214 when the 3D positions of the identified vertices are updated as one or more objects move. Other suitable configurations for determining dynamic measurements can be used. In addition, the dynamic measurement system 202 can include additional or alternative components that can serve specific embodiments.

[0038] The user interface 206 may be configured to receive dynamic measurements from the dynamic measurement system 202. The user interface 206 of the illustrated embodiment includes a display device 218. The display device 218 may be implemented by a monitor or other suitable device configured to display information to a user. For example, the display device 218 may be configured to display dynamic measurements received from the dynamic measurement system 202. In some embodiments, the display device 218 may also be configured to display an image of the scene 208 captured by the imaging device 204 and / or a deformable 3D model 214 generated by the dynamic measurement system 202. Additionally or alternatively, the user interface 206 may include any suitable device (e.g., a button, a joystick, a touch screen, a keyboard, a handle, etc.) configured to receive user input, such as for identifying a point within the scene 208 to determine a dynamic measurement.

[0039] In some embodiments, the dynamic measurement system 202 can be configured to determine multiple dynamic measurements within the scene 208. As an illustrative example, the dynamic measurement system 202 can be configured to determine the distance between the physical tool and the multiple anatomical objects 210 within the scene 208. In addition, the dynamic measurement system 202 can be configured to mark, track and / or present multiple dynamic measurements. For example, the dynamic measurement system 202 can be configured to mark (e.g., highlight) the physical tool, the multiple anatomical objects 210, and / or the distance between the physical tool and the multiple anatomical objects 210 (e.g., on the display of the display device 218). The dynamic measurement system 202 can also be configured to track and update multiple dynamic measurements when the physical tool moves relative to the multiple anatomical objects 210. The dynamic measurement system 202 can also be configured to present multiple dynamic measurements to the user (e.g., label them) (e.g., on the display of the display device 218).

[0040] Figure 3 An illustrative method 300 is shown that may be performed by the dynamic measurement system 202. Figure 3 Exemplary operations according to one embodiment are shown, but other embodiments may omit, add to, reorder, and / or modify Figure 3 In addition, Figure 3 Each operation depicted in may be performed in any manner described herein.

[0041] As shown, at operation 302, the dynamic measurement system 202 can generate a deformable 3D model 214 of the scene 208 based on the image of the scene 208. For example, the dynamic measurement system 202 can (e.g., using SLAM heuristics) generate a point cloud with multiple nodes representing surface points on one or more objects (e.g., anatomical object 210) within the scene 208, and derive vertices associated with 3D positions corresponding to the 3D positions of the multiple nodes. When the 3D positions of the multiple nodes move with the movement of one or more objects within the scene 208, the 3D positions of the vertices of the deformable 3D model 214 can be updated with the corresponding 3D positions of the multiple nodes. This can allow the deformable 3D model 214 to deform over time with the movement of one or more objects within the scene 208.

[0042] At operation 304, the dynamic measurement system 202 may also identify a first point on the anatomical object 210 located in the scene 208. In some embodiments, identifying the first point may include detecting a user input specifying the first point on the anatomical object 210. For example, the dynamic measurement system 202 may be configured to receive the user input by displaying the imagery of the scene 208 and / or the deformable 3D model 214 captured by the imaging device 204 on the display device 218. The designation of the first point may be performed as a discrete event (e.g., a touch gesture, a button press, a mouse click, a button release, etc.) at any point of the imagery of the scene 208 and / or the deformable 3D model 214 on the display device 218.

[0043] Upon receiving the user input, the dynamic measurement system 202 can associate the first point with a 3D location (e.g., a vertex) on the deformable 3D model 214. In some implementations, the first point can be identified on an outer surface of the anatomical object 210, such as a feature on the anatomical object 210 (e.g., the edge of a hernia).

[0044] In some embodiments, identifying the first point can include implementing and applying an artificial intelligence algorithm (such as a machine learning algorithm) to specify a first point located on the anatomical object 210 in the scene 208. Any suitable form of artificial intelligence and / or machine learning can be used, including, for example, deep learning, neural networks, etc. For example, a machine learning algorithm can be generated by a machine learning program and applied to a recognition operation. In some embodiments, the machine learning algorithm can be directed to identify the anatomical object 210 and / or features of the anatomical object 210 within the scene 208. The machine learning algorithm can operate as a recognition function applied to a single and / or fused image to classify the anatomical object 210 in the image.

[0045] In addition to or in lieu of a machine learning algorithm, other suitable methods may be used to identify a first point on the anatomical object 210. For example, the dynamic measurement system 202 may be configured to identify a first point on the anatomical object 210 within the scene 208 by implementing and applying an object recognition algorithm. For example, the object recognition algorithm may be used to identify a predetermined type of object (e.g., the anatomical object 210) within image data received from the imaging device 204, such as by comparing the imaging data received from the imaging device 304 with model object data of the predetermined type of object. Such model object data may be stored in a model database that may be communicatively coupled to the dynamic measurement system 202.

[0046] In some embodiments, the dynamic measurement system 202 can also identify a second point in the scene 208. The second point can be spaced a distance from the first point in the scene 208. For example, the second point can be identified on the same anatomical object 210 as the first point, on an anatomical object 210 different from the first point, on a non-anatomical object (e.g., a physical tool) within the scene 208, and / or on another area within the scene 208. In some embodiments, the second point can correspond to another feature on the anatomical object 210 (e.g., the opposite edge of a hernia). In some embodiments, the first point and the second point are spaced far enough apart that one point can be located outside the field of view of the imaging device.

[0047] Identifying the second point may include detecting a user input specifying the second point in the scene 208. For example, the dynamic measurement system 202 may be configured to receive the user input by displaying the image of the scene 208 captured by the imaging device 204 and / or the deformable 3D model 214 on the display device 218. The designation of the second point may be performed as a discrete event (e.g., a touch gesture, a button press, a mouse click, a button release, etc.) at any point of the image of the scene 208 and / or the deformable 3D model 214 on the display device 218.

[0048] Once the user input has been received, the dynamic measurement system 202 can associate the second point with a 3D location (eg, a vertex) on the deformable 3D model 214. In some implementations, the second point can be identified on an outer surface of the anatomical object 210.

[0049] Additionally or alternatively, identifying the second point may include implementing and applying an artificial intelligence algorithm (such as a machine learning algorithm) to specify the second point in scene 208. Any suitable form of artificial intelligence and / or machine learning may be used, including, for example, deep learning, neural networks, etc. For example, a machine learning algorithm may be generated by a machine learning program and applied to a recognition operation. In some embodiments, the machine learning algorithm may be directed to recognize objects and / or features of objects within scene 208. The machine learning algorithm may operate as a recognition function applied to a single and / or fused image to classify objects in the image.

[0050] In addition to or in lieu of a machine learning algorithm, other suitable methods may be used to identify a second point in the scene 208. For example, the dynamic measurement system 202 may be configured to identify a second point in the scene 208 by implementing and applying an object recognition algorithm. For example, the object recognition algorithm may be used to identify a predetermined type of object (e.g., an anatomical object 210, a physical tool, etc.) in image data received from the imaging device 204, such as by comparing the image data received from the imaging device 204 with model object data of the predetermined type of object. Such model object data may be stored in a model database that may be communicatively coupled to the dynamic measurement system 202.

[0051] At operation 306, the dynamic measurement system 202 may also determine a dynamic measurement value representing a physical distance of a contour between the first point and a second point in the scene 208 based on the deformable 3D model 214. For example, the dynamic measurement system 202 may be configured to determine a distance between 3D positions of vertices associated with the first point and the second point along the surface of the deformable 3D model 214. In some embodiments, the dynamic measurement system 202 may identify intermediate vertices between the first point and the second point along the surface of the deformable 3D model 214 to derive a 3D contour connecting the first point and the second point through the intermediate vertices. The dynamic measurement value may be calculated as a sum of intermediate distances between the 3D positions of the intermediate vertices on the 3D contour.

[0052] The dynamic measurement system 202 can be configured to dynamically update the dynamic measurement values ​​as one or more objects in the scene 208 move. For example, when the anatomical object 210 deforms, the 3D positions of the vertices of the deformable 3D model 214 associated with at least one of the first point or the second point may move. Additionally or alternatively, the 3D positions of the intermediate vertices on the 3D contour extending between the first point and the second point can be updated as one or more objects in the scene 208 move. These changes in the 3D positions of the vertices may affect the intermediate distances between the vertices. Therefore, when the 3D positions of the vertices are updated to dynamically update the dynamic measurement values, the intermediate distances can be recalculated.

[0053] In some implementations, the dynamic measurement value may be dynamically updated based on the 3D position of the vertex corresponding to each consecutive image frame of the imagery captured by the imaging device 204. Additionally or alternatively, the dynamic measurement value may be dynamically updated based on the 3D position of the vertex corresponding to multiple image frames over time. For example, the dynamic measurement value may represent a combination of measurements over multiple image frames (e.g., an average, mean, median, etc.).

[0054] In some embodiments, the dynamic measurement value can represent a difference in the physical distance of the outline based on the movement of one or more objects within the scene 208. For example, the dynamic measurement system 202 can detect changes in the 3D positions of vertices on the 3D outline of the deformable 3D model 214 and compare the distance between the 3D positions of the vertices at two different time points. In addition or alternatively, the change in distance between the 3D positions of the vertices can be determined by comparing the 3D positions of the vertices in a deformable or non-deformable first 3D model generated at a first time point with the 3D positions of the corresponding vertices in a deformable or non-deformable second 3D model generated at a second time point. Other suitable methods can also be used to determine the dynamic measurement value. For example, although the dynamic measurement value is described above as representing the physical distance of the outline between two points, additional points can be identified in the scene 208 to determine the dynamic measurement value.

[0055] In some embodiments, the method 300 may further include performing an operation based on the dynamic measurement value by the dynamic measurement system 202. For example, the dynamic measurement system 202 may be configured to instruct one or more display devices (e.g., the display device 218) to display an image depicting the scene 208 and / or the deformable 3D model 214. The dynamic measurement system 202 may also display the dynamic measurement value and / or the 3D contour (e.g., on the image depicting the scene 208 and / or the deformable 3D model 214). In some embodiments, the dynamic measurement system 202 may be configured to receive user input to specify one or more intermediate vertices defining the 3D contour of the deformable 3D model 214 so that the user can adjust the 3D contour.

[0056] The operation may also include determining the size (e.g., length, width, surface area, volume, etc.) of an object (e.g., anatomical object 210) and / or a feature of the object within scene 208 based on the dynamic measurement. As an example, the dynamic measurement system 202 may determine the size of a hernia on the anatomical object 210 based on the dynamic measurement so that the size of a mesh patch may be appropriately set to fit the hernia. To illustrate, the dynamic measurement may be dynamically updated as the hernia deforms (e.g., due to breathing and / or insufflation), so that the size of the mesh patch may be selected or adjusted based on the dynamic update of the dynamic measurement. As another example, the dynamic measurement system 202 may determine the length of the intestine (e.g., during a lower anterior resection procedure) based on the dynamic measurement, which may be dynamically updated as the length of the intestine is stretched and / or compressed. The dynamic measurement may also provide a reference for the size of the lungs (e.g., during thoracic surgery). For example, the dynamic measurement may be dynamically updated as the lungs deform. As another example, the dynamic measurement system 202 may be configured to determine how far the tip of a physical tool is from the anatomical object 210 during a medical procedure based on the dynamic measurements.

[0057] In some embodiments, the dynamic measurement system 202 can be configured to track changes (e.g., dynamically update) in the dynamic measurement value (e.g., due to movement of one or more objects within the scene 208). For example, the dynamic measurement system 202 can determine the change in the dynamic measurement value, such as by determining the difference between the dynamic measurement value and a previous dynamic measurement value. The change in the dynamic measurement value can indicate the effectiveness and / or progress of the surgical procedure (e.g., insufflation).

[0058] In addition, the dynamic measurement system 202 can be configured to instruct one or more display devices (e.g., display device 218) to display a change in the dynamic measurement value. The change in the dynamic measurement value can be represented by any suitable value, such as a discrete value representing a change in the dynamic measurement value (e.g., a distance, a range, a percentage, etc.). For illustration, the change in the dynamic measurement value can be displayed as a percentage (e.g., relative to an initial dynamic measurement value). In some embodiments, the dynamic measurement system 202 can be configured to selectively display one or both of the dynamic measurement value and the change in the dynamic measurement value, such as based on a user input specifying display of the dynamic measurement value and / or the change in the dynamic measurement value.

[0059] Figure 4 Another illustrative method 400 that may be performed by dynamic measurement system 202 is shown. Figure 4 Exemplary operations according to one embodiment are shown, but other embodiments may omit, add to, reorder, and / or modify Figure 4 In addition, Figure 4 Each operation depicted in may be performed in any manner described herein.

[0060] As shown, at operation 402, the dynamic measurement system 202 may generate a point cloud having a plurality of nodes based on an image of the scene 208 (e.g., captured by the imaging device 204), the plurality of nodes representing surface points on one or more objects (e.g., the anatomical object 210) within the scene 208. At operation 404, the dynamic measurement system 202 may also generate a deformable 3D model 214 having vertices having 3D positions associated with the 3D positions of the plurality of nodes.

[0061] At operation 406, the dynamic measurement system 202 can also identify two or more points within the scene 208. In some implementations, the points can be identified by receiving (e.g., via the user interface 206) a user input specifying two or more points on the imagery and / or the deformable 3D model 214 of the scene 208. In some implementations, at least one of the two or more points can be located on an anatomical object 210 within the scene 208. The remaining point(s) can be located on the same anatomical object 210, a different anatomical object 210, a physical tool, and / or another area within the scene 208.

[0062] At operation 408, the dynamic measurement system 202 may also identify vertices of the deformable 3D model 214 that form a 3D contour connecting two or more points. For example, the dynamic measurement system 202 may identify vertices of the deformable 3D model 214 that have 3D positions corresponding to the two or more points identified within the scene 208. The dynamic measurement system 202 may also identify one or more additional vertices of the deformable 3D model 214 that have 3D positions located between the vertices corresponding to the two or more points identified. A 3D contour may be formed to connect the identified vertices such that the 3D contour may extend along a surface of the deformable 3D model 214 between the two or more points identified.

[0063] At operation 410, the dynamic measurement system 202 may also determine a dynamic measurement value representing a physical distance of the contour between the two or more points based on the 3D contour. For example, the dynamic measurement system 202 may be configured to sum the distances between the 3D positions of adjacent vertices on the 3D contour of the deformable 3D model 214 to determine the dynamic measurement value.

[0064] In order to dynamically update the dynamic measurement values ​​as one or more objects (e.g., the anatomical object 210) within the scene 208 move, at operation 412, the dynamic measurement system 202 may determine whether one or more vertices on the 3D contour of the deformable 3D model 214 have moved. For example, the dynamic measurement system 202 may be configured to track the movement of a plurality of nodes over time as one or more objects within the scene 208 move and / or deform in the imagery captured by the imaging device 204. The dynamic measurement system 202 may update the 3D positions of the vertices of the deformable 3D model 214 with the updated 3D positions of the corresponding nodes, and determine whether the 3D position of any identified vertex on the 3D contour of the deformable 3D model 214 has moved.

[0065] If one or more of the identified vertices on the 3D contour have moved (yes at operation 412), the dynamic measurement system 202 may update the dynamic measurement value at operation 414. For example, the dynamic measurement system 202 may recalculate the sum of the distances between the updated 3D positions of the vertices on the 3D contour. If one or more vertices have not moved (no at operation 412), the dynamic measurement system 202 may continue to monitor the movement of the one or more vertices on the 3D contour. In some embodiments, the operation 412 of determining whether one or more vertices on the 3D contour of the deformable 3D model 214 have moved may be omitted. For example, the dynamic measurement value may be updated and / or recalculated at a selected interval without determining whether one or more vertices on the 3D contour of the deformable 3D model 214 have moved.

[0066] Figures 5A-6B 2 shows an illustrative example of determining dynamic measurement values ​​that may be performed by dynamic measurement system 202. For example, Figure 5A An embodiment 500 of an image 502 of a scene (e.g., scene 208) that may be captured by an imaging device 204 is shown. As shown, the image 502 includes a plurality of anatomical objects 504 (e.g., anatomical objects 504-1 through 504-3) in an environment 508 (e.g., a region within a subject of a medical procedure) and a physical tool 506 spaced apart from the plurality of anatomical objects 504. In the illustrated embodiment 500, a point cloud including a plurality of nodes 510 (e.g., nodes 510-1 through 510-n) has been generated by the dynamic measurement system 202. The nodes 510 may represent surface points on one or more objects (e.g., anatomical objects 504, physical tools 506, and / or environment 508) within the scene captured by the image 502.

[0067] Figure 5B An illustrative implementation 512 of a deformable 3D model 514 is shown, which can be generated by the dynamic measurement system 202 based on the image 502. The deformable 3D model 514 can implement or be similar to the deformable 3D module 214. As shown, the dynamic measurement system 202 can generate the deformable 3D model 514 by deriving vertices 516 (e.g., vertices 516-1 to 516-n) having 3D positions associated with the 3D positions of the plurality of nodes 510. Thus, the deformable 3D model 514 can depict one or more objects (e.g., anatomical objects 504, physical tools 506, and / or environment 508) within the scene captured by the image 502. In some embodiments, the deformable 3D model 514 can be generated using a SLAM algorithm that can derive the vertices 516 and track the positions of the vertices 516 as the imaging device 204 captures the image 502.

[0068] Fig. 6A An embodiment 600 of two or more points 602 (e.g., points 602-1 to 602-2) identified on a deformable 3D model 514 is shown. The two or more points 602 can be identified on the image 502 and / or the deformable 3D model 514, such as by receiving user input (e.g., via the user interface 206) to specify the two or more points 602 on the image 502 and / or the deformable 3D model 514. As shown, a first point 602-1 has been specified on a first anatomical object 504-1, and a second point 602-2 has been specified on a third anatomical object 504-3, which is separated from the first anatomical object 504-1 by the second anatomical object 504-2. The identified points 602 can be associated with vertices 516 of the deformable 3D model 514, which have 3D locations corresponding to the identified points 602.

[0069] The dynamic measurement system 202 can identify one or more intermediate vertices 604 (e.g., vertices 604-1 to 604-n) of the deformable 3D model 514 between the identified points 602. For example, the intermediate vertices 604 can include vertices 516 of the deformable 3D model 514 having 3D positions located between the identified points 602 along the surface of the deformable 3D model 514. In the illustrated embodiment 600, the intermediate vertices 604 extend from the first point 602-1 to the second point 602-2 along the surfaces of the first anatomical object 504-1, the second anatomical object 504-2, and the third anatomical object 504-3. The dynamic measurement system 202 can also form a 3D contour 606 connecting the identified points 602 through the intermediate vertices 604. For example, the 3D contour 606 may be formed by connecting the first point 602-1 and the second point 602-2 through the intermediate vertex 604 such that the 3D contour 606 extends from the first point 602-1 to the second point 602-2 along the surfaces of the first anatomical object 504-1, the second anatomical object 504-2, and the third anatomical object 504-3.

[0070] The dynamic measurement system 202 may determine a dynamic measurement value representing a physical distance of the contour between the identified points 602 based on the 3D contour 606. For example, the dynamic measurement system 202 may sum the distances between the 3D positions of the first point 602-1, the intermediate vertex 604, and the second point 602-2 to determine the dynamic measurement value.

[0071] In some embodiments, the movement of one or more objects within the scene depicted by image 502 may cause deformable 3D model 514 to deform. To illustrate, Figure 6BAnother embodiment 608 of the deformable 3D model 514 in a deformed state is shown. As shown, the second anatomical object 504-2 and the third anatomical object 504-3 have been deformed so that the 3D positions of the second point 602-2 and the intermediate vertex 604 located on the second anatomical object 504-2 and the third anatomical object 504-3 have moved. The dynamic measurement system 202 can thereby update the dynamic measurement value, such as by recalculating the sum of the distances between the updated 3D positions of the first point 602-1, the intermediate vertex 604, and the second point 602-2 on the 3D contour 606. The dynamic measurement system 202 can use the same intermediate vertex 604 to recalculate the sum of the distances between the first point and the second point 602, and / or the dynamic measurement system 2020 can identify one or more different intermediate vertices 604 to recalculate the sum of the distances between the first point and the second point 602.

[0072] Although Fig. 6A and Figure 6B The second point 602-2 is shown as being located on a different anatomical object 504 than the first point 602-1, but the second point 602-2 may be located on the same anatomical object 504 as the first point 602-1. Additionally or alternatively, the second point 602-2 may be located on the physical tool 506 and / or any other area of ​​the environment 508. Thus, the dynamic measurement value may be updated as one or more objects (e.g., the anatomical object 504 and / or the physical tool 506) of the deformable 3D model 514 move, which movement of the one or more objects may cause the identified points 602 and / or intermediate vertices 604 located on the one or more objects to move and affect the 3D contour 606.

[0073] In addition, although Fig. 6A and Figure 6B Intermediate vertices 604 are shown positioned along the surface of the deformable 3D model 514 between the first point and the second point 602 to form the 3D profile 606, but one or more of the intermediate vertices 604 may be omitted such that the 3D profile 606 may form a combination of point-to-point distances between the first point and the second point 602 and / or point-to-point distances between a surface point and the first point and the second point 602. Additionally or alternatively, the dynamic measurement system 202 may receive user input specifying selected intermediate vertices 604 so that the user may manipulate and / or adjust the 3D profile 606 on the deformable 3D model 514.

[0074] In some cases, the deformable 3D model 514 may be incomplete (e.g., in areas not captured by the imaging device 204), such that there may be holes or missing vertices in the deformable 3D model 514. In these cases, the dynamic measurement system 202 may be configured to perform dynamic interpolation to estimate the 3D positions of the missing vertices. For example, the dynamic measurement system 202 may interpolate the 3D positions of the missing vertices based on the 3D positions of nearby known vertices. In addition, the dynamic measurement system 202 may update the 3D positions of the missing vertices based on the movement of nearby known vertices as one or more objects within the scene move. In some embodiments, the dynamic measurement system 202 may be configured to perform dynamic interpolation when the 3D contour 606 passes through an incomplete area of ​​the deformable 3D model 514 (e.g., to reduce processing burden).

[0075] Figure 7 An illustrative implementation 700 of a display 702 that can be displayed on a display device 218 is shown. As shown, the display 702 includes a first display view 704 and a second display view 706. The first display view 704 can display an image 708 of a scene 208 captured by the imaging device 204. In the illustrated implementation 700, the image 708 depicts an anatomical object 710. The second display view 706 can display a deformable 3D model 712 of the anatomical object 710, which can be generated by the dynamic measurement system 202 based on the image 708. The deformable 3D model 712 can implement or be similar to the deformable 3D module 214 and / or the deformable 3D mode 514.

[0076] The display 702 may also depict a first point 714-1 on the anatomical object 710 (e.g., an outer surface of the anatomical object 710) in the image 708 and / or the deformable 3D model 712. For example, the dynamic measurement system 202 may receive user input (e.g., a touch gesture, a button press, a mouse click, a button release, etc.) at any point of the image 708 and / or the deformable 3D model 712 of the display 702 to specify the first point 714-1. The display 702 may also depict a second point 714-2 in the image 708 and / or the deformable 3D model 712. In the illustrated embodiment 700, the second point 714-2 is located on the anatomical object 710 (e.g., an outer surface of the anatomical object 710). The dynamic measurement system 202 may also receive user input (e.g., a touch gesture, a button press, a mouse click, a button release, etc.) at any point of the image 708 and / or the deformable 3D model 712 of the display 702 to specify the second point 714-2.

[0077] In some implementations, the first display view 704 and the second display view 706 can be coupled such that the image 708 in the first display view 704 moves simultaneously with the deformable 3D model 712 in the second display view 706. For example, the imaging device 204 can be moved (e.g., translated, rotated, scaled, etc.) such that the image 708 can depict the anatomical object 710 at different viewpoints in the first display view 704. As the image 708 of the anatomical object 710 moves, the deformable 3D model 712 can move simultaneously with the image 708 such that the viewpoint of the anatomical object 710 in the second display view 706 corresponds to the viewpoint of the anatomical object 710 in the first display view 704.

[0078] Additionally or alternatively, the first display view 704 and the second display view 706 may be decoupled such that the deformable 3D model 712 in the second display view 706 may be moved (e.g., translated, rotated, scaled, etc.) independently of the imagery 708 in the first display view 704. For example, a user may provide user input to move the deformable 3D model 712 within the second display view 706 without moving the imaging device 204 and / or without changing the imagery 708 displayed in the first display view 704. This may allow the user to view a larger area and / or a different viewpoint of the anatomical object 710 in the deformable 3D model 712 without moving the imaging device 204 and / or without changing the imagery 708 displayed in the first display view 704.

[0079] In some implementations, a user may specify a first point 714-1, move the deformable 3D model 712 in the second display view 706 without moving the imaging device 204 and / or without changing the image 708 displayed in the first display view 704, and specify a second point 714-2 on the deformable 3D model 712 in the second display view 706. This may allow the second point 714-2 to be specified on an area of ​​the anatomical object 710 that may be obscured in the image 708 and / or may be outside of the field of view of the imaging device 204 that captured the image 708. For example, the first point 714-1 may be located in a first field of view of the imaging device 204 that captured the image 708, and the second point 714-2 may be located in a second field of view of the imaging device 204 that is different from the first field of view.

[0080] Once the first point 714-1 and the second point 714-2 have been identified, the dynamic measurement system 202 can determine a dynamic measurement value representing a contour physical distance between the first point 714-1 and the second point 714-2. For example, the dynamic measurement system 202 can identify a 3D contour 716 extending between the first point 714-1 and the second point 714-2 along the surface of the anatomical object 710 and determine a distance of the 3D contour 716. In some embodiments, the display of the 3D contour 716 and / or the dynamic measurement value 718 can be displayed on the display device 218, such as on the image 708 in the first display view 704 and / or the deformable 3D model 712 in the second display view 706.

[0081] As described above, in some examples, the dynamic measurement system 202, the imaging device 204, the user interface 206, and / or the physical tool 506 can be associated with a computer-assisted medical system for performing a medical procedure on a body. Figure 8 An exemplary computer-assisted medical system 800 is shown that may be used to perform various types of medical procedures, including surgical and / or non-surgical procedures.

[0082] As shown, the computer-assisted medical system 800 may include a manipulator assembly 802 ( Figure 8 ), a manipulator cart 804, and an auxiliary device 806, all of which are communicatively coupled to each other. The medical team can use the computer-assisted medical system 800 to perform computer-assisted medical procedures or other similar operations on the body of a patient 808 or on any other body that can serve a specific embodiment. As shown, the medical team can include a first user 810-1 (such as a surgeon for a surgical procedure), a second user 810-2 (such as a patient-side assistant), a third user 810-3 (such as another assistant, a nurse, an intern, etc.), and a fourth user 810-4 (such as an anesthesiologist for a surgical procedure), all of which can be collectively referred to as users 810, and each user can control the computer-assisted medical system 800, interact with the computer-assisted medical system 800, or otherwise become a user of the computer-assisted medical system 800. During a medical procedure, there may be more, fewer, or alternative users that can serve a specific embodiment. For example, the team composition for different medical procedures or non-medical procedures may be different and include users with different roles.

[0083] Although Figure 8 A minimally invasive medical procedure (such as a minimally invasive surgical procedure) is shown being performed, but it should be understood that the computer-assisted medical system 800 can be similarly used to perform open medical procedures or other types of operations. For example, operations such as exploratory imaging operations, simulated medical procedures for training purposes, and / or other operations can also be performed.

[0084] like Figure 8 As shown, manipulator assembly 802 may include one or more manipulator arms 812 (e.g., manipulator arms 812-1 through 812-4) to which one or more instruments may be coupled. The instruments may be used in a computer-assisted medical procedure on patient 808 (e.g., in a surgical example, by being at least partially inserted into and manipulated within patient 808). Although manipulator assembly 802 is depicted and described herein as including four manipulator arms 812, it should be appreciated that manipulator assembly 802 may include a single manipulator arm 812 or any other number of manipulator arms that may serve a particular embodiment. Although Figure 8 The example of the manipulator arm 812 is shown as a robotic manipulator arm, but it should be understood that in some examples, one or more instruments can be partially or fully manually controlled, such as by being held and manually controlled by a person. For example, these partially or fully manually controlled instruments can be coupled to Figure 8 The computer-assisted instrumentation of the manipulator arm 812 is shown for use in conjunction with, or as an alternative to, it.

[0085] During a medical procedure, the user control device 804 can be configured to facilitate remote operation control of the manipulator arm 812 and instruments attached to the manipulator arm 812 by the user 810-1. To this end, the user control device 804 can provide the user 810-1 with an image of the operating area associated with the patient 808 captured by the imaging device. To facilitate the control of the instruments, the user control device 804 can include a set of master controls. These master controls can be manipulated by the user 810-1 to control the movement of the manipulator arm 812 or any instrument coupled to the manipulator arm 812.

[0086] The auxiliary device 806 may include one or more computing devices configured to perform auxiliary functions to support the medical procedure, such as providing insufflation, electrocautery energy, lighting or other energy for imaging equipment, image processing, or coordinating components of the computer-assisted medical system 800. In some examples, the auxiliary device 806 may be configured with a display monitor 814 that is configured to display one or more user interfaces, or graphical or textual information that supports the medical procedure. In some cases, the display monitor 814 may be implemented by a touch screen display and provide user input capabilities. The augmented content provided by the region-based augmentation system may be similar to or different from the content associated with the display monitor 814 or one or more display devices (not shown) in the operating area.

[0087] Manipulator assembly 802, user control device 804, and auxiliary device 806 may be communicatively coupled to each other in any suitable manner. Figure 8 As shown, the manipulator assembly 802, the user control device 804, and the auxiliary device 806 can be communicatively coupled via a control line 816, which can represent any wired or wireless communication link that can serve a particular embodiment. To this end, the manipulator assembly 802, the user control device 804, and the auxiliary device 806 can each include one or more wired or wireless communication interfaces, such as one or more local area network interfaces, Wi-Fi network interfaces, cellular interfaces, and the like.

[0088] In some embodiments, one or more of the processes described herein may be implemented at least in part as instructions embodied in a non-transitory computer-readable medium and executable by one or more computing devices. In general, a processor (e.g., a microprocessor) receives instructions from a non-transitory computer-readable medium (e.g., a memory, etc.) and executes those instructions to perform one or more processes, including one or more of the processes described herein. Such instructions may be stored and / or transmitted using any of a variety of known computer-readable media.

[0089] Computer-readable media (also known as processor-readable media) include any non-transitory media that participates in providing data (e.g., instructions) that can be read by a computer (e.g., by a processor of a computer). Such media can take a variety of forms, including, but not limited to, non-volatile media and / or volatile media. Non-volatile media can include, for example, optical or magnetic disks and other persistent memory. Volatile media can include, for example, dynamic random access memory ("DRAM"), which typically constitutes main memory. Common forms of computer-readable media include, for example, magnetic disks, hard disks, tapes, any other magnetic media, compact disk read-only memory ("CD-ROM"), digital video disks ("DVD"), any other optical media, random access memory ("RAM"), programmable read-only memory (("PROM"), electrically erasable programmable read-only memory ("EPROM"), FLASH-EEPROM (Flash EEPROM), any other memory chip or cartridge, or any other tangible medium from which a computer can read.

[0090] Fig. 9 An exemplary computing device 900 is shown, which may be specifically configured to perform one or more of the processes described herein. Any of the systems, computing devices, and / or other components described herein may be implemented by the computing device 900 .

[0091] like Fig. 9 As shown, computing device 900 may include a communication interface 902, a processor 904, a storage device 906, and an input / output ("I / O") module 908 communicatively connected to each other via a communication infrastructure 910. Fig. 9 An illustrative computing device 900 is shown in Fig. 9 The components shown in are not intended to be limiting. Additional or alternative components may be used in other embodiments. Fig. 9 Components of computing device 900 are shown in .

[0092] The communication interface 902 may be configured to communicate with one or more computing devices. Examples of the communication interface 902 include, but are not limited to, a wired network interface (such as a network interface card), a wireless network interface (such as a wireless network interface card), a modem, an audio / video connection, and any other suitable interface.

[0093] The processor 904 generally represents any type or form of processing unit capable of processing data and / or interpreting, executing and / or directing the execution of one or more of the instructions, processes and / or operations described herein. The processor 904 may perform operations by executing computer-executable instructions 912 (e.g., applications, software, code and / or other executable data instances) stored in the storage device 906.

[0094] The storage device 906 may include one or more data storage media, devices, or configurations, and may take any type, form, and combination of data storage media and / or devices. For example, the storage device 906 may include, but is not limited to, any combination of non-volatile media and / or volatile media described herein. Electronic data (including data described herein) may be temporarily and / or permanently stored in the storage device 906. For example, data representing computer executable instructions 912 configured to direct the processor 904 to perform any operation described herein may be stored within the storage device 906. In some examples, the data may be arranged in one or more databases residing within the storage device 906.

[0095] The I / O module 908 may include one or more I / O modules configured to receive user input and provide user output. The I / O module 908 may include any hardware, firmware, software, or combination thereof that supports input and output capabilities. For example, the I / O module 908 may include hardware and / or software for capturing user input, including but not limited to a keyboard or keypad, a touch screen component (e.g., a touch screen display), a receiver (e.g., an RF or infrared receiver), a motion sensor, and / or one or more input buttons.

[0096] The I / O module 908 may include one or more devices for presenting output to a user, including but not limited to a graphics engine, a display (e.g., a display screen), one or more output drivers (e.g., a display driver), one or more audio speakers, and one or more audio drivers. In some embodiments, the I / O module 908 is configured to provide graphical data to the display for presentation to the user. The graphical data may represent one or more graphical user interfaces and / or any other graphical content that may serve a particular implementation.

[0097] In the foregoing description, various exemplary embodiments have been described with reference to the attached drawings. However, it is apparent that various modifications and changes may be made to the various exemplary embodiments, and additional embodiments may be implemented, without departing from the scope of the invention as set forth in the following claims. For example, certain features of one embodiment described herein may be combined or replaced with features of another embodiment described herein. Therefore, the description and drawings should be regarded as illustrative rather than restrictive.

Claims

1. A system comprising: a memory for storing instructions; and One or more processors communicatively coupled to the memory and configured to execute the instructions to perform a process comprising the steps of: Generating a deformable three-dimensional model of the scene, i.e., a deformable 3D model, based on the image of the scene; identifying a first point on an anatomical object in the scene; and Based on the deformable 3D model, a dynamic measurement representing a contour physical distance between the first point and a second point in the scene is determined, wherein the dynamic measurement is dynamically updated as one or more objects within the scene move.

2. The system of claim 1, wherein at least one of the first point or the second point is on an outer surface of the anatomical object.

3. The system of claim 1, wherein at least one of the first point or the second point moves when the anatomical object is deformed.

4. The system of claim 1, wherein generating the deformable 3D model comprises using a simultaneous localization and mapping heuristic approach. 5 . The system of claim 1 , wherein identifying the first point comprises detecting user input specifying the first point within a selected one or both of the image or the deformable 3D model of the scene.

6. The system of claim 1, wherein identifying the first point comprises using a machine learning algorithm to specify the first point located on the anatomical object in the scene.

7. The system of claim 1, further comprising identifying the second point in the scene, identifying the second point comprising detecting user input specifying the second point within a selected one or both of the image or the deformable 3D model of the scene.

8. The system of claim 1, further comprising identifying the second point in the scene by using a machine learning algorithm to specify the second point in the scene.

9. The system of claim 1, wherein the image of the scene comprises a stereoscopic image captured by an imaging device.

10. The system of claim 1, wherein the first point is located in a first field of view of an imaging device that captures the imagery, and the second point is located in a second field of view of the imaging device that is different from the first field of view.

11. The system of any one of claims 1 to 10, wherein the process further comprises instructing one or more display devices to display the image depicting the scene in a first display view and to display the deformable 3D model in a second display view.

12. The system of claim 11, wherein the first display view is coupled to the second display view such that the image depicting the scene in the first display view and the deformable 3D model in the second display view move simultaneously.

13. The system of claim 11, wherein the first display view is decoupled from the second display view such that the deformable 3D model in the second display view moves independently of the image depicting the scene in the first display view.

14. The system of claim 11, wherein the process further comprises instructing the display device to display a 3D contour extending between the first point and the second point along a surface of the one or more objects within the scene, wherein the 3D contour represents the contour physical distance.

15. The system of any one of claims 1 to 10, wherein the process further comprises performing an operation based on the dynamic measurement.

16. A method comprising: Generating, by at least one computing device and based on an image of the scene, a deformable three-dimensional model, i.e., a deformable 3D model, of the scene; identifying, by the at least one computing device, a first point located on an anatomical object in the scene; and A dynamic measurement representing a contour physical distance between the first point and a second point in the scene is determined by the at least one computing device and based on the deformable 3D model, wherein the dynamic measurement is dynamically updated as one or more objects within the scene move.

17. The method of claim 16, wherein at least one of the first point or the second point is on an outer surface of the anatomical object.

18. The method of claim 16, wherein at least one of the first point or the second point moves when the anatomical object is deformed.

19. The method of claim 16, wherein generating the deformable 3D model comprises using a simultaneous localization and mapping heuristic approach.

20. The method of claim 16, wherein identifying the first point comprises using one or both of the image of the scene or the deformable 3D model to detect user input specifying the first point on the anatomical object.

21. The method of claim 16, wherein identifying the first point comprises using a machine learning algorithm to specify the first point located on the anatomical object in the scene.

22. The method of claim 16, further comprising identifying the second point within the scene by detecting user input specifying the second point in the scene using a selected one or both of the image or the deformable 3D model of the scene.

23. The method of claim 16, further comprising identifying the second point in the scene by specifying the second point in the scene using a machine learning algorithm.

24. The method of claim 16, wherein the image of the scene comprises a stereoscopic image captured by an imaging device.

25. The method of claim 16, wherein the first point is located in a first field of view of an imaging device that captures the imagery, and the second point is located in a second field of view of the imaging device that is different from the first field of view.

26. The method of any one of claims 16 to 25, further comprising instructing one or more display devices to display the imagery depicting the scene in a first display view and to display the deformable 3D model in a second display view.

27. The method of claim 26, wherein the first display view is coupled to the second display view such that the image depicting the scene in the first display view and the deformable 3D model in the second display view move simultaneously.

28. The method of claim 26, wherein the first display view is decoupled from the second display view such that the deformable 3D model in the second display view moves independently of the image depicting the scene in the first display view.

29. The method of claim 26, wherein the display device is configured to display a 3D contour extending between the first point and the second point along a surface of the one or more objects within the scene, wherein the 3D contour represents the contour physical distance.

30. A non-transitory computer-readable medium storing instructions that, when executed, direct a processor of a computing device to perform a process comprising: Generating a deformable three-dimensional model of the scene, i.e., a deformable 3D model, based on the image of the scene; identifying a first point on an anatomical object in the scene; and Based on the deformable 3D model, a dynamic measurement representing a contour physical distance between the first point and a second point in the scene is determined, wherein the dynamic measurement is dynamically updated as one or more objects within the scene move.

Images