Camera device for operation in conjunction with surgical tools

Abstract

The surgical imaging system includes at least one camera device having a camera body containing an image sensor configured to capture image data within a field of view. The scope extends from the camera body along its longitudinal axis. A camera orientation sensor is connected to the camera device. The camera orientation sensor detects the camera orientation of the camera device. A scope orientation sensor detects the scope orientation of the scope relative to the camera body. A controller monitors the camera orientation and scope orientation. In response to changes in the scope orientation relative to the camera orientation, the controller updates the rotation of the field of view of the image data.

Description

Technical Field

[0001] The present disclosure generally relates to a camera device or a camera probe for surgical applications, and more specifically, to a camera device having one or more configurations for cooperative operation with surgical tools.

Background Art

[0002] The operation of a camera probe (e.g., an endoscope, an arthroscope, a laparoscope, etc.) may require manual operation in combination with the operation and movement of surgical tools utilized in various patient procedures. Such operations can pose challenges, especially when viewing the narrow patient cavities commonly required by minimally invasive surgical procedures. The following disclosure provides various configurations and related methods of operation for improving the operation of a surgical imaging device and the associated presentation of image data showing various anatomical structures and tools for assisting in surgical procedures.

Summary of the Invention

[0003] Generally, the present disclosure provides a camera device, related configurations, and methods of operation that may improve the cooperative work of one or more cameras or camera probes in combination with surgical tools. In various implementations, the camera device may be configured to operate in relation to a surgical tool, including an elongate shaft that may be implemented to access an internal patient cavity for minimally invasive surgical procedures. For example, in some cases, the present disclosure may provide a surgical imaging system including one or more cameras or camera devices. The camera device may be a configuration including a camera orientation sensor and / or a scope orientation sensor. The camera orientation sensor may be configured to identify, for example, the spatial orientation of the camera device relative to gravity. Additionally, the scope orientation sensor may be configured to detect the rotation or scope orientation of the scope relative to the camera body of the camera device. By monitoring the relationship of the camera orientation between cameras and the scope orientation, the controller may adjust the display of a plurality of corresponding video feeds that can depict different portions of the patient's internal cavity.

[0004] In some implementations, the Disclosure may provide one or more horizontal detections of camera devices relative to gravity. For example, the orientation or rotation angle of the scope relative to the camera body may be configured to be monitored and updated in order to rotate the image data captured by one or more of the camera devices. By monitoring the camera orientation and scope rotation relative to gravity, the subject presented in the field of view can be maintained relative to the horizon even when the rotation angle of the scope is adjusted. As will be provided in more detailed embodiments throughout the following description, the camera orientation and / or scope orientation of one or more camera devices may be received by a surgical imaging system to adjust various angular relationships, viewing parameters, and / or select the display of image data received from each of the camera devices. In this way, the Disclosure may provide automatic or assisted viewing of one or more fields of view for presentation on a display.

[0005] These and other features, purposes, and advantages of this disclosure will become apparent upon reading the following description with reference to the accompanying drawings. [Brief explanation of the drawing]

[0006] [Figure 1] Figure 1 is an illustrative diagram showing the camera device of a surgical imaging system. [Figure 2] Figure 2 is a projection view of a camera device implemented in combination with surgical tools. [Figure 3] Figure 3 is a projection view showing the main body or enclosure of the camera device that defines the intersection point of the camera probe angles. [Figure 4] Figure 4 is a projection view of the camera device showing the focal area defined by the probe length and working distance. [Figure 5A] Figure 5A is an outline view of a camera device showing the intersection distance defined by at least one angled support surface of the main body. [Figure 5B]Figure 5B is a geometric model showing the intersection distance in comparison to the probe length and working distance of the camera device exemplified in Figure 5A. [Figure 6] Figure 6A is a side projection view showing an exemplary implementation of the camera device body, including multiple support surfaces. [Figure 6B] Figure 6B is a side projection view showing an exemplary implementation of the camera device body, including multiple support surfaces. [Figure 6C] Figure 6C is a side projection view showing an exemplary implementation of the camera device body, including multiple support surfaces. [Figure 6D] Figure 6D is a side projection view showing an exemplary implementation of the camera device body, including multiple support surfaces. [Figure 6E] Figure 6E is a top view showing an exemplary implementation of the camera device body, including multiple support surfaces. [Figure 7] Figure 7 is an illustrative diagram showing multiple compatible devices, including one or more cameras that can be tracked within a surgical coordinate system to track relative position and / or orientation. [Figure 8] Figure 8 shows a simulated display window generated by an imaging system that presents the positions of multiple compatible devices within a surgical coordinate system. [Figure 9] Figure 9 is an illustrative projection of a surgical camera device showing the operation of a camera orientation sensor. [Figure 10A] Figure 10A is an exemplary projection view of a camera device showing the first camera orientation and the first scope orientation. [Figure 10B] Figure 10B is an exemplary projection view of a camera device showing the first camera orientation and the second scope orientation. [Figure 10C] Figure 10C is an exemplary projection view of a camera device showing a second camera orientation and a second scope orientation. [Figure 11] Figure 11 is a flowchart showing a method for displaying multiple video feeds from multiple surgical cameras. [Figure 12] Figure 12 is a schematic block diagram of the surgical imaging system according to this disclosure. Detailed description [Modes for carrying out the invention]

[0007] The following description refers to the attached drawings illustrating specific possible implementations. Wherever possible, the same reference numerals are used throughout the drawings to refer to the same or similar parts. Naturally, other implementations may be utilized and structural and functional modifications may be made without departing from the scope of this disclosure.

[0008] Referring to Figures 1 to 4, in various implementations, the present disclosure may provide a camera or imaging device 10 designed for improved operation in conjunction with at least one of the endoscopes 14 (e.g., camera probe, arthroscope, laparoscope, etc.) forming a surgical tool 12 and a surgical imaging system 16. As shown in Figure 1, the camera device 10 may be configured to include a body 18 or enclosure that connects to a camera probe 20. The body 18 or enclosure of the camera device 10 may be configured to include at least one support surface 22 that can control the distance Hs and intersection angle θ between the shaft 24 of the surgical tool 12 and the camera probe 20. When implemented together, the support surface 22 of the body 18 extends beyond the distal end portion 20b of the camera probe 20 to a working distance L f The tool axis A is located relative to the focal area 30 of the camera device 10. T The intersection distance D can generally be defined by the intersection of two lines. int The configuration may also control the following: In this configuration, the support surface 22 formed by the main body 18 and the camera device 10 controls the field of view 32 and focal area 30 of the camera probe 20, and the crossing distance D int This allows the working end 34 of the surgical tool 12 to be aligned with the actuator.

[0009] The camera device 10 may be configured to maintain alignment with the shaft 24 of the surgical tool 12 by connecting to the shaft 24 via a collar 36 or retaining strap that surrounds and / or connects to at least a portion of the camera device 10, thereby holding a support surface 22. In the shown embodiment, the collar 36 forms part of a cannula 38 or access port. The cannula 38 may be configured to include at least one lumen 40 through which the shaft 24 of the surgical tool 12 and the probe 20 of the camera device 10 can extend from the external environment 42 into the patient cavity 44. In this configuration, the at least one lumen 40 formed by the collar 36 of the cannula 38 surrounds the shaft 24 and at least a portion of the camera device 10, and the support surface 22 is in contact with the shaft 24, compressed and held, thereby the tool axis A T It may be configured to align with the intersection angle θ defined by the support surface 22.

[0010] As best shown in FIGS. 1 and 2, the cannula 38 may be configured to include a first lumen 40a and a second lumen 40b each configured to receive a shaft 24 and a probe 20, respectively. In various implementations, the cannula 38 and / or the collar 36 may be formed of a flexible or elastically deformable material (e.g., silicone, rubber, or various similar elastomeric materials). When used in combination with the camera device 10 and a tool 12 oriented along the intersecting cannula 38 and / or the collar 36, the elastic material of the cannula 38 and / or the collar 36 may be stretched outwardly. In response to this stretching, due to the elastic nature of the cannula 38 and / or the collar 36, a compressive force is applied to the shaft 24 and the camera device 10, such that the shaft 24 is held within a recess 46 or a channel 48 formed by the support surface 22 and aligned with the camera probe 20 along the intersection angle θ. In this configuration, the pressure applied to the shaft 24 by the collar 36 and / or the cannula 38 may allow the surgical tool 12 to be manipulated and / or moved while the camera device 10 is held in a fixed relative position and orientation with respect to the surgical tool 12 throughout the associated movement.

[0011] As best shown in FIG. 3, the support surface 22 forms an angled wedge or otherwise forms a spacing that is angled with respect to the tool axis A T with respect to the camera axis A C at an intersection angle θ and may correspond to an alignment function or a positive spacing function. The recess 46 formed by the channel 48 may correspond to an elongated trough, guide, or retention mechanism that maintains the angular spacing between the camera axis A T with respect to the tool axis A C by preventing a change in the rotational orientation 50 of the camera device 10 with respect to the surgical tool 12. In the illustrated example, the body 18 extends along the camera axis A CIt may correspond to an interface adapter, which may include opposing interface surfaces 52a extending outward from the camera axis A. In this configuration, the interface surfaces 52a may be configured to provide a semi-flat surface for gripping and adjusting the orientation of the body 18, which may include a textured surface, a ridge 52c, a gripping element, or a similar configuration to improve tactile interaction. The probe 20 is hidden in the figure of the body 18 for clarity. However, the flange 52d of the body 18 may be configured to interconnect with the proximal end portion 20a of the probe, extending beyond one or more support surfaces 22 to the camera axis A. C The configuration may extend along the same line.

[0012] Referring again to Figures 1-4 in their entirety, in various embodiments, the combined interaction of the camera device 10 and surgical tool 12 with the cannula 38 and / or collar 36 is represented by arrow 50, with respect to the camera axis A of the position and orientation of the surgical tool 12 and shaft 24. C It may be possible to further manipulate and rotate the camera device 10 around the camera axis A. In this way, the orientation of the camera device 10 can be adjusted and rotated with respect to the orientation and position of the surgical tool 12, and selectively coupled to the surgical tool, such that the camera device 10 maintains a constant orientation with respect to the surgical tool 12 at the intersection angle θ. As will be discussed in various examples throughout the detailed description below, selective control of the relative movement between the camera device 10 and the surgical tool 12 may be particularly useful for adjusting the orientation of the body 18. For example, in some embodiments, the camera axis A C The camera device 10 may be configured to adjust the angular orientation of the main body 18 relative to the surgical tool 12 centered on the shaft 24, thereby aligning it with one of several different support surfaces 22 (e.g., a first support surface 22a, a second support surface 22b, etc.) and changing the intersection angle θ and / or spacing Hs. In this way, the camera device 10 can measure the intersection distance D int and / or can be used to selectively adjust the viewpoint of the working end 34 shown in the field of view 32. An example of a camera device 10 having multiple different support surfaces 22 will be further described and illustrated in Figures 6A to 6E.

[0013] As best illustrated in the embodiment shown in Figure 4, the main body 18 of the camera device 10 may be configured to include a first support surface 22a and a second support surface 22b that define a first intersection angle θ1 and a second intersection angle θ2, respectively. The first support surface 22a and the second support surface 22b are located on the camera axis A C It is positioned across the camera axis A on the opposing side of the main body 18. C The configuration may also form an elongated trapezoid having a primary axis extending parallel to it. In the example shown, the intersection angles θ1 and θ2, and the corresponding spacing distance H S1 H S2 The main body 18 is the camera axis A C They are approximately equal so as to include two perpendicular planes of reflective symmetry extending around it. In this configuration, the angular relationship between the camera device 10 and the surgical tool 12 may be the same when the shaft 24 aligns with either the first support surface 22a or the second support surface 22b. While such operation is convenient and may provide simplified operation, as will be further discussed with reference to Figures 5 and 6, the body 18 of the camera device 10 may form support surfaces of varying numbers and spacings Hs and intersection angles θ to suit various applications.

[0014] Before moving on to other embodiments of the main body 18 and including various configurations of the support surface 22, for clarity, the exemplary operation of the camera device 10 in combination with the endoscope 14 and surgical tools 12 will be described in more detail. Referring again to Figure 1, the camera device 10 may be implemented to acquire image data from a first viewpoint 56a of the field of view 32. For clarity, the field of view 32 may also be referred to as the first field of view 32. Furthermore, as previously stated, the endoscope 14 may be used in combination with the camera device 10 to acquire additional video or image data in a second field of view 58 showing the patient cavity 44 from a second viewpoint 56b. Throughout the operation of the imaging system 16, the control console or video controller 60 may be configured to receive image data from the camera device 10 and the corresponding image sensors of the endoscope 14, process the image data, and display the first field of view 32 and the second field of view 58 on the display 54. In the embodiment shown, each of the fields 32, 58 and the corresponding viewpoints 56a, 56b may be positioned and oriented by the controller 60 to display the corresponding image data on the display 54 at various ratios or positions. In this way, the combined operation of the surgical imaging system 16 can simultaneously provide the user (e.g., a physician, nurse, assistant, etc.) with various corresponding figures showing the patient cavity 44 from multiple different viewpoints 56. Such operation can improve the effectiveness of the surgical imaging system 16 and facilitate various minimally invasive procedures.

[0015] Referring further to Figures 1 to 4, as described above and as will be further explained in the detailed embodiments later, the camera device 10 is located on the camera axis A C The configuration may include multiple support surfaces 22 that are arranged axially around the camera axis A at various angular orientations. For example, as shown in Figures 1-4, the support surfaces 22a and 22b are located around the camera axis A C It is located on opposing side surfaces oriented 180° apart in the axial direction with respect to the camera axis A. As provided in later embodiments, the support surface 22 is located on the camera axis A. CThe points can be angularly separated at various intervals from the center. Therefore, the orientation of the first field of view 32 shown in Figure 1 may rotate in accordance with the angular orientation of the camera device 10, and as a result, the shaft 24 rotates around a perimeter 64 which may be formed by the sides or range of the viewing window 66. To avoid the visual complexity associated with the changes in the angular orientation of the camera device 10 and the corresponding representation of the surgical tool 12 in the first field of view 32, the controller 60 of the system 16 may be configured to detect the position of the surgical tool 12 and identify the corresponding angle associated with the first viewpoint 56a. In response to the orientation angle of the camera device 10 relative to the tool 12, the controller can rotate or manipulate the image data shown in the first field of view 32 to consistently represent the surgical tool 12 in a default or preferred relationship with respect to the perimeter 64 formed by the viewing window 66.

[0016] In the embodiment shown, the surgical tool 12 extends into the field of view 32 of the camera device 10 from the lower circumferential wall 64a around the perimeter 64 of the viewing window 66 or from a first angular orientation. To determine the rotational orientation 50 of the camera device 10, the controller 60 may be configured to identify one or more features of the surgical tool 12 in the image data captured in the field of view 32, such as the tool shaft 24 and the corresponding vector or path of the shaft 24 in the image data. Once identified, one or more features may indicate the rotational orientation 50 of the body 18 and the corresponding support surface 22 of the camera device 10 relative to the surgical tool 12. Based on this determination, the controller 60 may reorient or angularly align the image data so that the surgical tool 12 is consistently shown in the viewing window 66 having the same default or desired angular orientation, regardless of the rotation of the camera device 10 relative to the surgical tool 12. This operation allows the user of the camera device 10 to selectively align various support surfaces 22 of the main body 18 with respect to the surgical tool 12 in the field of view 32 shown in the viewing window 66, without changing the desired orientation of the corresponding image data, thereby adjusting the intersection angle θ and / or spacing A of the tool shaft 24 with respect to the camera probe 20. S It may become possible to adjust this.

[0017] In various implementations, the axial spacing and / or position of each support surface 22 can be pre-configured and / or identified by the controller 60 based on a serial number, model, or various identifiers indicating the spacing between the support surfaces 22 of the main body 18 relative to the field of view 32, thus simplifying the relevant algorithms and image processing that may be required to identify the relative angular orientation of the camera device 10 with respect to the surgical tool 12. Such information may be useful to the relevant orientation correction algorithm by identifying several finite positions around the viewing window 66 to predict the rotation of the field of view 32 as a result of the rotational orientation of the camera device 10. Such information may enable the controller 60 to resolve the orientation detected in the image data to one of several known angular orientations 52 (e.g., 60 degrees, 90 degrees, 120 degrees, 180 degrees, etc.). Further information describing the operation of the video controller 60, as well as exemplary underlying processors and technologies, will be described with reference to Figure 12.

[0018] Referring to Figures 5A and 5B, the relationship between the intersection angle θ and the spacing distance Hs is the intersection distance D between the working end 34 of the surgical tool 12 and the field of view 32. int The associated target distance D T This is further shown and explained with reference to a geographical model. As schematically shown in Figure 5A, the crossing distance D int The probe distance L of probe 20 P The working distance L associated with the optical element or imaging device located at the distal end portion 20b of the probe 20. f This may correspond to the sum or combination of the above. To clearly illustrate the relationship and corresponding interval provided by the camera device 10, a representative triangle 70 is shown in Figure 5B. In the embodiment corresponding to the second support surface 22b and the second intersection angle θ2, the first leg 70a of triangle 70 corresponds to the working distance L of the camera device 10. f The crossover distance D extends from the proximal end 20a of the probe 20 to the focal region 30 or the target region 72. intThis is represented by the second leg 70b of triangle 70, which may be configured to correspond to the distance Hs between the body 18 of the camera device 10 and the shaft 24 of the surgical tool 12. In this configuration, the hypotenuse 70c of triangle 70 corresponds to the tool distance D T Alternatively, the configuration may correspond to the range of the surgical tool 12 that extends beyond the main body 18 of the camera device 10 to the working end 34 or actuator. The tool distance D in this configuration T As shown in Figure 5A, the length of the shaft 24 and the working end 34 are defined, and the working end 34 may be positioned in the center of the target area 72 of the field of view 32 of the camera device 10. However, in various implementations, the intersection angle θ and / or intersection distance D int It may be beneficial to adjust the geometry to capture or record image data that is more suitable to the user's preference or operation of the surgical tool 12, which has variations in geometric shape, proportions, length, etc. Accordingly, exemplary variations of some of the geometric shapes of the main body 18 and support surface 22, which may be implemented to adjust the operation of the camera device 10 for various applications, are shown in Figures 6A to 6E.

[0019] Based on the interval Hs and intersection angle θ, the intersection distance D int and tool distance D T To clearly illustrate the relationship, the following equations define the geometric characteristics of the camera device 10 and allow for modification and adjustment of the various relationships described herein to suit various applications. As shown in Equations 1 and 2, the aforementioned relationships in relation to triangle 70 are represented by symbols. D int =L p +L f (Formula 1) tanθ²=(H s ) / D int (Formula 2) Based on equations 1 and 2, the length of the probe extending from the main body 18 of the camera device 10 can be defined by equation 3. (L p )=(H s ) / tanθ2-(L f ) (Formula 3) In this way, using equation 3, the known working distance L corresponding to the optical or image sensor of the camera device 10 is obtained. f The configuration may also identify the probe length based on the desired distance to the focal region 30. Similarly, the resulting working distance L f is the tool distance D T It may be calculated by intersecting with or based on Equation 4, and as a result, the working distance L f This can be calculated to correspond to the desired presentation of the surgical tool 12, particularly the working end 34, in the field of view 32. (L f )=(H s ) / tanθ2-(L p ) (Equation 4) Therefore, the relationship associated with triangle 70 can define at least one example of the operation of the camera device 10.

[0020] Referring here to Figures 6A to 6E, various embodiments of the camera device 10 are shown, illustrating changes in the intersection angle θ of the support surface 22, the spacing distance Hs, and the angular orientation. As shown in Figures 6A and 6B, the intersection angles θ1 and θ2 of the first support surface 22a and the second support surface 22b are shown individually or with the corresponding spacing distance H S1 and H S2 This can be modified in combination with the following. In some implementations, the interval distance Hs is the camera axis A C The direct correspondence to the intersection angle θ between multiple support surfaces 22 that are angularly distributed around a central point may be changed. The relationship between the intersection angle θ and the spacing distance Hs is a constant intersection distance D int To maintain this, it may be defined by Equation 2. In this configuration, the rotation of the camera device 10 relative to the shaft 24 of the surgical tool 12 is defined by the working distance L of the camera. f The configuration may adjust only the viewpoint of the field of view 32 relative to the working end 34 without changing the intersection of the focal area 30 or the target area 72 within it. Alternatively, in some implementations, the crossing distance D is adjusted based on the orientation or rotation of the corresponding support surfaces 22 that contact the body 18 and shaft 24 of the camera device 10. intIt may be beneficial or desirable to change the camera axis A. In such cases, the spacing distance Hs is maintained or changed in combination with the intersection angle. C The crossing distance may be moved along the axis to be closer to or further away from the distal end portion 20b of the probe 20. Thus, the design of the body 18 of the camera device 10 and the corresponding support surface 22 may be adjusted based on this disclosure to enable the camera device 10 to be implemented for various applications and user preferences.

[0021] As shown in Figures 6C to 6E, the number of support surfaces 22 of the main body 18 formed by the camera device 10 and the corresponding geometric shapes may be greatly changed depending on the application. As shown in Figure 6C, the main body 18 is on the camera axis A C The system comprises three support surfaces 22a, 22b, and 22c that can be angularly oriented at a spacing angle φ centered on the camera axis A. As shown, the spacing angle φ is constant between each of the support surfaces 22a, 22b, and 22c. However, the angular spacing of the support surfaces, and the corresponding shapes and proportions of the body 18 extending between the support surfaces 22, may be modified depending on the application. The intermediate surface 80 formed by the body 18 between the support surfaces 22 may correspond to various concave, convex, rounded, segmented, or other surface contours extending between the support surfaces 22. In various implementations, the intermediate surface 80 is positioned such that the camera device 10 is aligned with the camera axis A. C As a result of rotating around it, it may provide a smooth contour that allows the shaft 24 of the surgical tool 12 to slide smoothly over the intermediate surface 80 before engaging with the recess 46 or channel 48 formed by the support surface 22.

[0022] As shown in Figure 6D, the support surfaces 22a, 22b, 22c, and 22d are similarly spaced at an angle φ relative to the camera axis A CThe elements are evenly distributed angularly around a central point. As shown in Figures 6C and 6D, each of the intersection angles θ may differ among the support surfaces 22. For example, in Figure 6C, the second intersection angle θ2 is greater than the third intersection angle θ3. Although not shown in Figure 6C due to projection, θ1 may be less than θ2 or θ3. As shown in Figure 6D, each of the intersection angles θ may differ with respect to the corresponding first support surface 22a, second support surface 22b, third support surface 22c, and fourth support surface 22d. In the embodiment shown, the first intersection angle θ1 may be less than the second intersection angle θ2. The third intersection angle θ3 may be greater than the second intersection angle θ2, and the fourth intersection angle θ4 may be greater than the third intersection angle θ3. Although each of the intersection angles θ is described as different as corresponding to each of the support surfaces 22, one or more of the intersection angles θ may be the same depending on the desired configuration of the camera device 10.

[0023] Referring now to Figure 6E, yet another example of the geometric shape of the support surface 22 of the main body 18 is shown from a top view perspective of the main body 18 opposite the camera probe 20. In the example shown, the main body 18 is on the camera axis A C The configuration may include five separate support surfaces 22a to 22e distributed at various intervals and angles φ1 to φ5 centered on the camera axis A. In this configuration, the main body 18 may be spirally oriented outward along the spiral contour surface 82, thereby supporting the camera axis A C The maximum radial spacing of each of the support surfaces 22 around the camera axis A is gradually increased. C A gradual increase in the radial spacing of the support surfaces 22 from the first support surface 22a to the second support surface 22b, and from the second support surface 22b to the third support surface 22c, etc. Between the first support surface 22a and the fifth support surface 22e, a smooth contour may be provided on the intermediate surface 80, allowing the shaft 24 to slide smoothly between each of the support surfaces 22. Although not clearly shown in Figure 6E, the camera axis A CA gradual increase in the radial spacing of each of the support surfaces 22 can result in a corresponding increase in the intersection angles θ1 to θ5. In this configuration, the intersection angle θ associated with each of the support surfaces 22 can gradually increase as the body 18 of the camera device 10 rotates relative to the shaft 24 of the surgical tool 12. Thus, the camera device 10 can be implemented to provide a variety of features that can suit the preferences and / or requirements associated with various surgical procedures.

[0024] As mentioned above, the intersection angle θ corresponding to each of the support surfaces 22 can be changed individually or in combination with the spacing distance Hs. Referring to Figure 6E, as an example, each of the intersection angles θ1 to θ5 can be changed individually or in combination with the corresponding spacing distance Hs. S1 ~H S5 It can be varied in combination with (not shown). In various implementations, the intersection angle θ and the spacing distance Hs change as the working distance L changes as the intersection angle θ changes. f It can change while remaining constant. Referring again to Equation 4, the working distance L f The intersection distance D is directly proportional to the relationship between the spacing distance Hs and the intersection angle θ of a given camera device 10. By maintaining or adjusting the features of Equation 4, the intersection distance D int This can be varied or maintained over a range of intersection angles up to θ that is suitable for the desired operation of the camera device 10.

[0025] Based on the geometric shape of the main body 18, the spacing distance Hs is the working distance L associated with the focal area 30 within the field of view 32 of the camera device 10. f The angle of intersection θ can be changed in combination with the working distance L to ensure that it is maintained or changes between the support surface 22 that aligns with the shaft 24 of the surgical tool 12. f Maintaining this may be desirable in some implementations, but tool axis A T The working distance L between the support surfaces 22 changes in the viewpoint associated with one or more of the intersection angles θ, such that the position of the focal region 30 along the line changes. fIt may also be beneficial to modify this. Finally, although not shown in Figure 6E for simplicity, the spacing angle φ between the various support surfaces 22 may be changed based on the desired geometric shape of the main body 18 and adjusted to ensure that the operation of the camera device 10, in particular the rotational operation between the support surfaces 22 on the intermediate surface 80, is optimized for operation by the user during surgical procedures.

[0026] Referring here to Figures 7 and 8, additional features related to the operation of the imaging system 16 are described. As shown in Figure 7, in some implementations, the imaging system 16 may be configured to selectively present image data from multiple cameras or camera devices 10 at various viewpoints 56 having different orientations distributed around the patient cavity 44. As previously stated, the imaging system 16 may be configured to communicate with various camera devices 10, surgical tools 12, endoscopes 14, or various other surgical devices that may be implemented in combination herein. For clarity, these devices may generally be referred to as compatible devices 90, each of which may be used alone or in combination with the features and operations discussed herein. In the embodiments shown, three separate imaging devices or scopes are used in combination, including a first camera device 10a, a second camera device 10b, and an endoscope 14. Similar to the embodiments described above, the first camera device 10a may capture image data from the first viewpoint 56a, the endoscope 14 may capture image data from the second viewpoint 56b, and the second camera device 10b may be configured to capture image data from the third viewpoint 56c.

[0027] Each viewpoint 56 is shown in an exemplary field of view, including the first field of view 32, the second field of view 58, and the third field of view 92, which are associated with the first viewpoint 56a, the second viewpoint 56b, and the third viewpoint 56c, respectively. Due to the complexity of the various viewpoints available for demonstration in the system 16 and the first and second viewing windows 66a and 66b, it may be difficult for the user to maintain awareness of the spatial orientation of each viewpoint 56 in the corresponding fields of view 32, 58, and 92 relative to the anatomical structures of the patient and patient cavity 44. As will be further discussed with reference to Figure 8, the video controller 60 can assist the user in maintaining awareness of the relative spatial orientation and relative position of each device 90 associated with the imaging system 16 by tracking the orientation of each of the compatible devices 90 relative to each other. Furthermore, the orientation and relative position of the device 90 communicating with the imaging system 16 may be presented in relation to the anatomical graphic 100, or a simulated depiction of an anatomical structure or region near the patient cavity 44 may be presented to demonstrate the relative position and / or orientation of the device 90 with respect to the graphic 100.

[0028] As shown in Figure 8, graphic representations of each of the exemplary devices 10a, 10b, and 14 are shown to indicate the relative position and spatial orientation of the devices to the anatomical graphic 100. As shown, the anatomical graphic 100 may show a surface depiction 102 of the patient's relevant anatomical structure, and in some cases, it may further show an internal depiction 104 representing one or more internal anatomical features (e.g., bone, muscle, cartilage, organ, etc.) associated with the patient's local relative anatomical structure to the patient cavity 44. In some implementations, the anatomical graphic 100, including the surface depiction 102 and / or internal depiction 104, may correspond to a graphic representation that can be rendered based on one or more representative or patient-specific scans (e.g., X-ray, ultrasound, magnetic resonance imaging, computed tomography scan, etc.). In such embodiments, the anatomical graphic 100 may be configured to include one or more specific patient features that can further assist in guiding the surgeon to complete surgery or to identify one or more regions of interest 106 within the patient cavity 44. In this way, the spatial orientation and position associated with the device 90 may be shown in relation to anatomical graphics 100 to help the user of the imaging system 16 visualize and operate the relevant tools and devices to successfully complete various surgical procedures.

[0029] As will be discussed in more detail with reference to Figure 7, scan data, graphics, photographs, or other related visual representations associated with the surgical site or patient cavity 44 may be configured to be accessed via an external device or server 210 communicating with the system 16. For example, in some cases, anatomical graphics 100 or similar graphic information may be configured to be accessed in a procedure-specific database to provide representative graphics, images, or representative scans associated with the procedure. The procedure and corresponding graphics, scans, or image data (e.g., 100, 102, 104) may be configured to be identified or entered as initial setup procedures for the imaging system 16 as preparation for the procedure. Depending on the type of procedure, graphics 100 and / or images corresponding to the relevant patient anatomical structures, including surface and internal depictions 104, may be loaded into local memory (e.g., memory 200) and displayed on the display 54. In this way, scan or image data (e.g., 100, 102, 104) can be presented on a display 54 or various user interfaces (e.g., a tablet, computer console, touchscreen, etc.) to display voluntary viewpoints 56a, 56b, 56c and corresponding graphic representations 120 or selectable icons 126 in corresponding positions and / or orientations in local coordinate systems 118a, 118b, 118c on the display 54. In this way, the controller 60 can provide presentations of graphic representations 120 or selectable icons 126 of various other medical devices 12 in corresponding positions and orientations detected by the tracking device 114 in relation to corresponding patient anatomical structures mapped to the surgical coordinate system 116, such as camera devices 10a, 10b, one or more scopes 14, or corresponding positions and orientations. Such representations may provide an intuitive and visually clear user interface, allowing the user to easily identify desired fields of view 32, 58, 92 or corresponding viewing windows 66a, 66b, 66c for presentations to assist in surgical procedures.

[0030] Although the description primarily refers to camera devices 10a, 10b, one or more scopes 14, etc., the systems and methods described can similarly be applied to present similar positional and corresponding operational information for various surgical tools 12 that can communicate with the system 16. For example, an exemplary surgical tool may correspond to a shaver handpiece that can be used in combination with device 10a and endoscope 14. In such a case, the surgical tool 12 (e.g., shaver handpiece) may also be configured to include one or more tracking devices 114 that enable the corresponding position and orientation to be tracked by the system 16 in a surgical coordinate system 116. The device graphic 120 representing the surgical tool 12 may also be accessed via an external device, server 210, and / or memory to present the graphic 120 representing the surgical tool 12 in a viewing window 110, and may be configured to be positioned / oriented in the surgical coordinate system 116 with respect to the position / orientation of the corresponding graphic, scan, or image data (e.g., 100, 102, 104), as well as the position / orientation of the camera device(s) 10 and / or endoscope 14. Examples of surgical tools 12 that may be shown in the viewing window may include, but are not limited to, various surgical cutting tools (e.g., shavers, rasps, burrs, disecters, drills, sabers, receptors, blades, etc.) and ablation devices, catheters, pumps, suction or inhalation devices, and similar tools.

[0031] Referring further to Figure 8, the spatial orientation and position of the device 90 may be depicted as a representative graphic relative to the anatomical graphic 100 in the viewing window 110. To clearly demonstrate and track the orientation and position of the fitted devices 90 relative to each other, as well as the anatomical structure of the patient or patient cavity 44, the device 90 may be configured with one or more tracking devices 114 that can function alone or in combination to monitor and update the relative position and orientation of the device 90 within the surgical coordinate system 116. As shown, each orientation and position of the fitted devices 90 is represented by an individual local coordinate system 118. For example, a first local coordinate system 118a may be configured to define the position and orientation of the first camera device 10a and the corresponding first viewpoint 56a. Similarly, a second local coordinate system 118b may be configured to define the orientation and position of the endoscope 14 and the corresponding second field of view 58 within the surgical coordinate system 116. The third local coordinate system 118c may also define the relationship between the orientation and position of the second camera device 10b. During operation, the controller 60 of the system 16 may track the respective positions and orientations of the local coordinate system 118 relative to the surgical coordinate system 116 and accurately generate corresponding device graphics 120 that show the relative relationships between the compatible devices 90 shown in the viewing window 110.

[0032] To accurately track the position and orientation of the conforming device 90 in order to position and update the position of the local coordinate system 118, the controller 60 may be configured to monitor and maintain communication with a plurality of corresponding tracking devices 114. For example, in some implementations, the conforming device 90 may be configured to be tethered to the tracking system via a flexible tether 122 that can utilize one or more shape sensors (e.g., strain sensors associated with fiber Bragg grading) to track the local coordinate system 118 in the surgical coordinate system 116. During operation, the translational and / or rotational paths of the flexible tether 122 may be configured to be tracked through the surgical coordinate system 116. For example, one or more sensors associated with the flexible tether 122 may be configured to correspond to fiber Bragg grading sensors or optical sensors extending along the length of the tether 122, which can operate to detect the bending or degree of curvature, as well as the direction of curvature, based on signals received from corresponding shape sensors distributed along the length of the tether 122. In this way, the controller 60 may be configured to track the local coordinate systems 118 relative to each other within the surgical coordinate system 116. Furthermore, one or more flexible tethers 122 may be configured to connect to the patient's anatomical structures at predetermined positions, so that changes in the patient's relative position within the surgical coordinate system 116 are similarly updated and can be shown by the anatomical graphics 100. In this way, the local coordinate system 118 associated with each of the compatible devices 90 may be tracked throughout the operation of the imaging system 16 to inform the user of the relative position of the viewpoint 56 to the patient cavity 44 and / or the patient's anatomical structures.

[0033] In some implementations, the device graphic 120 may be configured to correspond to an interactive graphic that can be incorporated into a touchscreen or user interface display that can provide on-screen selections corresponding to selectable view icons 126 in the viewing window 110 on the display 54. In this way, the user may interact with a user interface associated with the viewing window 110, as shown in Figure 8, to selectively activate each of the corresponding fields of view 32, 58, and 92 for display by the imaging system 16. Furthermore, based on the orientation of the device 90 to the patient cavity 44 and / or the patient's anatomical structure, the controller 60 may be configured to mark the corresponding view and / or associated view icon 126 at the anatomical position of the view relative to the patient's anatomical structure. For example, in various embodiments, it may be beneficial to identify the position and orientation of the field of view associated with each viewpoint 56 in anatomical terms that can be identified based on the relative position and orientation of the surgical coordinate system 116 and / or the local coordinate system 118 relative to the patient's anatomical structure. In this configuration, the diagrams may be described using anatomical terms (e.g., anterior, medial-lateral, posterior, etc.), and these corresponding diagrams may be shown in space, labeled based on anatomical posture, and recorded and tracked for diagnostic purposes.

[0034] In various implementations, the tracking devices 114 or devices 114 associated with monitoring each of the local coordinate systems 118 of the adapted device 90 may be implemented as one or more tracking techniques that can be used individually or in combination. For example, in addition to or as an alternative to the flexible tether 122, the position of each of the local coordinate systems 118 of the adapted device 90 may be configured to be tracked via one or more radio triangulation, time-of-flight (ToF), and / or angle-of-arrival (AoA) detection methods or similar techniques provided via a radio communication interface (e.g., Zigbee, ultra-wideband, radio frequency, infrared, Bluetooth low energy, short-range radio communication, etc.). Such detection and tracking may provide the relative position of the local coordinate system 118 within the surgical coordinate system 116. Furthermore, the orientation of each of the local coordinate systems 118 may be tracked by one or more attached or integrated orientation sensors, which may take the form of one or more inertial or directional sensors (e.g., accelerometers, gyroscopes, magnetometers, etc.). The operation of the orientation sensor can provide a display of the orientation of each of the compatible devices 90 in a global coordinate system that can be aligned with the surgical coordinate system 116. In this way, a combination of one or more inertial measurements combined with one or more radio frequency position tracking methods may be processed and utilized by the video controller 60 to monitor the relative position and orientation in the local coordinate system 118 and generate a device graphic 120 similar to that shown in Figure 8. While certain techniques are described with reference to the tracking device 114, it should be understood that similar tracking techniques may be implemented, including but not limited to image or video-based object tracking, stereoscopic tracking (e.g., computer vision), strain gauges or strain arrays, or similar techniques.

[0035] Referring here to Figures 9 to 11, various embodiments of the camera device 10 are described with reference to the coordinated presentation of multiple image feeds presented on the display 54 of the imaging system 16. As shown in Figure 9, the camera device 10 may include a first tracking device 114 in the form of a camera orientation sensor 130, which may be located in or otherwise in relation to the camera body 132 or handle body of the camera device 10. During operation, the camera orientation sensor 130 tracks the longitudinal axis A L The camera orientation sensor 130 may be configured to detect the orientation of the camera body 132 in relation to a probe 20 or scope extending from the camera body 132 along the same direction. In this configuration, the camera orientation sensor 130 may be configured to detect the orientation of the camera body 132 and the scope or probe 20 with respect to a fixed bearing or direction (e.g., gravity).

[0036] As shown in Figure 9, arrow 134 represents the rotation of the camera body 132 detected by the camera orientation sensor 130. During operation, the controller 60 may be configured to receive image data from the camera device 10 showing the field of view 136, as well as orientation data from the camera orientation sensor 130 indicating the spatial orientation of the camera device 10. For reference, details A and B show that the coordinate system 138 of the camera device 10 is relative to the gravity vector 142 indicating the spatial orientation. Furthermore, the rotational position 144 of the field of view 136 is shown relative to the rotational range 146. The rotational range is along the longitudinal axis A L This represents the range of position 144 of the field of view 136 when rotated around . As shown in details A and B, the rotation associated with arrow 134 is approximately 90° along the longitudinal axis A L This can result in an exemplary rotation of the camera device 10 around the longitudinal axis. Corresponding to the rotation around the longitudinal axis, the coordinate system 138 is such that, with respect to the gravity vector 142, the longitudinal axis A L Rotates 90° around the longitudinal axis A. L The rotation of the camera device 10 around the longitudinal axis A L This can result in a change in the rotational position 144 of the field of view 136 within the rotational range 146, caused by the angular offset of the scope angle ρ relative to the field of view ρ.

[0037] In response to the detection of rotation of the camera device 10, the controller 60 may be configured to offset the image data presented in the field of view 136 in the direction of rotation so as to maintain a fixed rotational relationship with respect to the horizon 150, where objects and / or features can be defined perpendicular to the gravity vector 142. As shown in details A and B, the rotational position 144 of the field of view 136 may change within the rotational range 146. However, in response to changes in the direction of the gravity vector 142 and the corresponding rotation of the camera device 10, the image data presented on the display 54 may be rotated so as to maintain the relationship of objects depicted in the field of view 136 with respect to the horizon 150. Such operation of the camera device 10 may be referred to as “horizontal control”.

[0038] As previously mentioned with reference to Figures 7 and 8, when applied to multiple camera devices 10 or imaging devices, the adjustment of image data presented on the display 54 in a consistent orientation with respect to the horizon 150 can ensure that features captured in a field of view (e.g., a first field of view 32, a second field of view 58, etc.) are presented on the display 54 in a consistent orientation and relationship with respect to the horizon 150. Furthermore, in some implementations, one or more of the fields of view 32, 58, 92, etc., can be maintained in a fixed or user-selectable angular relationship with respect to the horizon 150 or the gravity vector 142. For example, the first camera device 10a may be presented with corresponding image data in the field of view 32 aligned to a first orientation with respect to the gravity vector 142, while image data captured by the second camera device 10b may be presented in a second consistent or fixed relationship with respect to the gravity vector 142. As shown in Figure 9, the relationship between the rotation of the field of view 136 and the gravity vector 142 may be optionally incorporated into the camera body 132, or it may be selected via a user input 152, which may otherwise be communicated with the camera device 10.

[0039] Referring here to Figures 10A, 10B, and 10C, exemplary implementations of the camera device 10, including a camera orientation sensor 130 and a scope orientation sensor 160, are shown. In the example shown in Figure 10, the camera orientation sensor 130 may be configured to be connected to the camera body 132 or the scope or probe 20 of the camera device. The probe 20 or scope may be configured to be connected to the camera body 132 via a rotary coupling 162. The scope or probe 20 is aligned along the longitudinal axis A with respect to the camera body 132. L It can rotate freely around the longitudinal axis A. The scope orientation sensor 160 may also correspond to a rotation sensor (e.g., a potentiometer, encoder, etc.), and the camera body 132 and the longitudinal axis A L The camera device may be configured to output a rotation signal indicating the rotational relationship between the probe 20 or scope around the horizon 150. Therefore, based on the camera orientation data supplied by the camera orientation sensor 130 and the scope orientation data supplied by the scope orientation sensor 160, the camera device 10 may be configured to provide various control features similar to the control with respect to the horizon 150 as described above.

[0040] First, referring to Figure 10, the camera body 132 is shown with a user input 152 oriented in an upward direction 164, generally opposite to the gravity vector 142. The longitudinal axis A relative to the camera body 132 LIn response to the rotation of the scope or probe 20 around the horizon, the controller 60 may be configured to detect changes in the rotation angle ρ of the scope or probe 20 relative to the camera body 132, as reported by the scope orientation sensor 160. As shown in details C and D, the rotational position 144 of the field of view 136 may change within the rotational range 146 as a result of the rotation of the scope or probe 20 via the rotational coupling 162. However, the camera orientation data recorded by the camera orientation sensor 130 may be configured to continue reporting that the gravity vector 142 remains constant, as illustrated with reference to the coordinate system 138. In response to the corresponding change in scope orientation data relative to the camera orientation data reported by the camera orientation sensor 130, the controller 60 may be configured to update or rotate the image frame associated with the image feed provided by the camera device 10 to correspond to the apparent change in the rotation angle ρ of the probe 20 or scope. In this way, objects or features presented in the field of view 136 may maintain a constant relationship with respect to the horizon 150, as shown in details C and D.

[0041] As shown in Figure 10C, the controller 60 may also be configured to detect changes in camera orientation data reported by the camera orientation sensor 130. As shown by comparing Figures 10B and 10C, the camera device 10, including the camera body 132 and the probe 20 or scope, is aligned along the longitudinal axis A L They rotate together by approximately 90° around the center. As shown with respect to coordinate system 138, the controller 60 may be configured to detect changes in the direction of the gravity vector 142, as shown in detail E, based on camera orientation data communicated by the camera orientation sensor 130. In response to the change in the direction of the gravity vector 142 with respect to coordinate system 138, without any change in scope orientation data reported by the scope orientation sensor 160, the controller 60, together with the change in the rotation position 144 of the field of view 136, determines that the orientation of the horizon 150 is along the longitudinal axis A LThis may allow rotation around the gravity vector 142. As a result, the image data presented in the field of view 136 on the display 54 may be shifted in position and orientation relative to the gravity vector 142. Thus, the controller 60 may selectively apply horizontal correction or rotation of the image data in response to detected changes in the direction of the gravity vector 142 and / or the rotation angle ρ of the scope or probe 20 relative to the camera body 132.

[0042] As shown in Figures 10A, 10B, and 10C, the rotational position 144 of the field of view 136 can be changed by adjusting the rotation angle ρ at the rotational coupling 162 while maintaining its relationship with the horizon 150. In an alternative configuration, the camera device 10 may be rotated without changing the rotation angle ρ, which may result in a change in the horizon 150 relative to the gravity vector 142 and a corresponding change in the image data of the field of view 136 presented on the display 54. Furthermore, similar to the camera device 10 discussed with reference to Figure 9, the reception of input to the user interface 152 may update the offset of the horizon 150 relative to the coordinate system 138. In this way, the combined operation of the camera orientation sensor 130 and the scope orientation sensor 160 can provide improved flexibility in the operation of the camera device 10, either individually or within a system having multiple imaging devices or camera devices.

[0043] As described in this disclosure, the camera orientation sensor 130 can correspond to various devices capable of detecting the orientation of the camera device 10 with respect to gravity, a geomagnetic field, or similar forces. For example, in various implementations, the camera orientation sensor 130 may be implemented as one or more of a gyroscope, accelerometer, magnetometer, and / or inertial measuring unit (IMU). As previously stated, the scope orientation sensor 160 can correspond to an encoder, potentiometer, or similar angular rotation sensor. In some implementations, the scope orientation sensor 160 may be implemented as an accelerometer, gyroscope, IMU, or similar device. In such implementations, the scope orientation data reported by the scope orientation sensor 160 may be interpreted by the controller 60 in relation to the camera orientation data reported by the camera orientation sensor 130. Thus, the orientation sensors 130, 160 may be flexibly implemented and incorporated into one or more of the probe 20 or scope and / or camera body 132 to provide the functions described in this disclosure.

[0044] Referring here to Figure 11, a flowchart is shown illustrating a method 170 for displaying image data from multiple surgical cameras, such as camera device 10. During operation, method 170 may be applied by controller 60 and may be initiated in response to receiving first and second image data from first and second camera devices 10a, 10b (174). Furthermore, simultaneously with or rapidly following the reception of image data, controller 60 may be configured to receive first camera orientation data from the first camera device 10a and second camera orientation data from the second camera device 10b (176). As discussed with reference to Figures 9 and 10, the camera orientation data may correspond to information captured by one or more of the camera orientation sensor 130, the scope orientation sensor 160, or more broadly, the tracking device 114. Thus, as shown in step 178, controller 60 may display the first image data at a first display angle and the second image data at a second display angle, one or more of which may be maintained or offset with respect to the gravity vector 142.

[0045] Using image data from camera devices 10a and 10b, as well as corresponding orientation data, the controller 60 may be configured to receive a video stream containing first image data and second image data. In the shown embodiment, the controller 60 may be configured to adjust the first video feed of the first image data relative to the gravity vector 142 in response to changes in orientation data communicated by one or more of the orientation and position data communicated by the camera orientation sensor 130, the scope orientation sensor 160, and / or the tracking device 114 (180). Furthermore, the controller may be configured to adjust the orientation or position of the second video feed from the second image data based on the relationships between the orientation data of the second camera device 10b (182). The orientation of the second image data may be configured to be adjusted based on the direction of the gravity vector 142 or relative to the first camera device 10a. In this way, the controller 60 can independently control the orientation and / or position of multiple video feeds from multiple camera devices 10a, 10b.

[0046] In general, the controller 60 may be configured to adjust the first and second image data so that each of the corresponding video feeds maintains a rotational orientation with respect to the gravity vector 142. In this way, the image data presented as parallel video feeds on the display 54 can be presented with a consistent orientation with respect to the gravity vector 142 or the horizon 150. Furthermore, in some implementations, the second image data may be maintained with respect to the first image data and / or the gravity vector 142 or the horizon 150 in a fixed rotational relationship or a user-selected angle offset. In such cases, the orientation data reported by each of the camera devices 10a and 10b may be configured to be interpreted by the controller 60 to adjust the image data in direct correspondence to the offset with respect to the gravity vector 142 or similarly the horizon 150. As an optional configuration, the second image data captured by the second camera device 10b may be offset by a fixed angle or a user-defined angle with respect to the rotational position 144 or rotational angle ρ of the first image data captured by the first camera device 10a. Thus, each of the camera devices 10a and 10b may be configured to capture and present image data in various fixed or adjustable relationships with respect to the gravity vector 142 or the horizon 150. While this will be discussed in detail with reference to the first and second image data, it should be understood that third, fourth, or additional video feeds may be similarly controlled and presented in response to data from sensors 114, 130, and 160, simultaneously or selectively, with similar relative or absolute angle or position adjustments.

[0047] In some implementations, the angular offset between the first camera device 10a or the second camera device 10b with respect to the gravity vector 142 or camera orientation data may be set or adjusted in response to input to the user interface 152 (184). In response to the angular setting input to the user interface 152 in step 184, the controller 60 may be configured to update the relationship between the second display rotation of the second image data and the first display rotation of the first image data (186). Once updated, the angular offset between the set of images captured by the first camera device 10a and the second camera device 10b may be configured to be displayed consistently with respect to the offset and to the horizon 150. Furthermore, the image data captured by each of the camera devices 10 may be displayed independently or relatively in response to changes in orientation data captured by orientation sensors 114, 130, 160, etc.

[0048] Referring here to Figure 12, a block diagram of the imaging system 16 is shown. As discussed throughout this disclosure, the system 16 may comprise imaging devices or camera devices 10a, 10b, 14 and may be configured to communicate with various surgical tools 12 via a controller 60. Devices 10a, 10b, 14 may comprise one or more light sources 192, an image sensor 194, and a user interface 196. In various implementations, devices 10, 14 may be configured to correspond to endoscopes, laparoscopes, arthroscopes, etc., having elongated probes 20 with narrow distal ends suitable for various non-invasive surgical techniques. For example, the distal end may have a diameter of less than 2 mm. As shown, devices 10, 14 may be configured to communicate with the controller 60 via a communication interface. Although shown connected via conductive connections, the communication interface may correspond to a wireless communication interface operating via one or more wireless communication protocols (e.g., Wi-Fi, 802.11b / g / n, etc.).

[0049] The light source 192 may correspond to various light emitters configured to produce light in the visible and / or near-infrared range. In various implementations, the light source 192 may be configured to include light-emitting diodes (LEDs), laser diodes, or other illumination technologies. The one or more image sensors 194 may correspond to various sensors and configurations, including, for example, charge-coupled device (CCD) sensors, complementary metal-oxide-semiconductor (CMOS) sensors, or similar sensor technologies.

[0050] In various implementations, one or more imaging devices (e.g., endoscope 14) may be configured to include one or more control circuits 190 configured not only to control the operation of one or more image sensors 194 and light sources 192, but also to process image data and / or communicate with a controller 60 or system controller. Furthermore, the control circuits 190 may communicate with a user interface 196 which may include one or more input devices, indicators, displays, etc. The user interface 196 may provide control of the imaging device 10, including the invocation of one or more routines, as discussed in this disclosure. The user interface may provide selection or toggling of one or more image feeds associated with the operation of the camera device 10 and / or endoscope 14. The control circuits 190 may be implemented by various forms of controllers, microcontrollers, application-specific integrated controllers (ASICs), and / or various control circuits or combinations thereof.

[0051] The controller 60 or system controller may be configured to include a processor 198 and memory 200. The processor 198 may be configured to include one or more digital processing units, such as a central processing unit (CPU) having one or more processing cores, a graphics processing unit (GPU), a digital signal processor (DSP), a field-programmable gate array (FPGA), or an application-specific integrated circuit (ASIC). In some configurations, multiple processing units are combined into a system-on-a-chip (SoC) configuration, while in other configurations, the processing units may correspond to individual components. During operation, the processor 198 executes program instructions stored in memory 200 to perform the operations described in this disclosure.

[0052] The memory 200 may be configured to include one or more data storage devices, for example, a magnetic or solid drive and a random access memory (RAM) device for storing digital data. The memory 200 may be configured to include one or more stored program instructions, object detection templates, image processing algorithms, etc. The memory 200 may be configured to include one or more object tracking routines and corresponding graphic generation routines, which may be implemented to work in conjunction with the tracking device 114 to monitor the position and spatial orientation of the local coordinate system 118. These routines may be configured to include instructions for processing relevant tracking information, generating relevant device graphics 120 and / or anatomical graphics 100, and outputting such information to the viewing window 110 on the display 54.

[0053] As described above, in some implementations, the controller 60 may correspond to a display or video controller. In such applications, the controller 60 may be configured to include one or more formatting circuits 204 that process image data received from the imaging device 10, communicate with the processor 198, and process the image data according to one or more of the operating methods discussed herein. The formatting circuits 204 may be configured to include one or more signal processing circuits, analog-to-digital converters, digital-to-analog converters, etc. The display controller may be in the form of an integrated interface (e.g., a touchscreen, input buttons, an electronic display, etc.) or may be configured to include a user interface 206 that is implemented by one or more connected input devices (e.g., a tablet) or peripheral devices (e.g., a keyboard, mouse, foot pedal, etc.).

[0054] As shown, the controller 60 also communicates with external devices or servers 210, which may correspond to a network, local or cloud-based servers, device hubs, central controllers, or various devices that can communicate with the controller 60, and more generally with the imaging system 16, via one or more wired (e.g., Serial Universal Serial Bus (USB), Universal Asynchronous Receiver / Transmitter (UART), etc.) and / or wireless communication interfaces (e.g., ZigBee, Ultra Wideband (UWB), Radio Frequency Identification (RFID), Infrared, Bluetooth®, Bluetooth® Low Energy (BLE), Near Field Communication (NFC), etc.) or similar communication standards or methods, including networks, local or cloud-based servers, device hubs, central controllers, or various devices that can communicate with the controller 60. For example, the controller 60 may receive updates to various modules and routines, and may also communicate sample image data from the imaging device 10 to a remote server for improved operation, diagnosis, and updates to the imaging system 16. The user interface 196, external server 210, and / or surgical control console 212 may communicate with the controller 60 via one or more I / O circuits 214. The I / O circuit 214 may support a variety of communication protocols, including but not limited to Ethernet / IP, TCP / IP, Universal Serial Bus, ProfiBus, ProfiNet, Modbus, and serial communication.

[0055] According to some aspects of the present disclosure, a camera device for working in cooperation with a surgical tool comprises a tool shaft. The camera device comprises a body having at least one angled support surface defining an intersection angle; a camera probe having a probe length (LP) connected to the body at a proximal end and extending to a distal end; the proximal end portion being spaced from the support surface by a distance (Hs) formed by the connection between the body and the support surface; and an optical element of a camera defining a field of view in relation to the distal end portion, wherein the support surface receives the tool shaft and aligns the field of view with the tool axis at the working distance of the camera.

[0056] Depending on the embodiment, the present disclosure may implement one or more of the following structures or configurations in various combinations. - The main body forms a sheath that connects to the outer wall of the camera probe. - The main body forms at least one wing extending from the sheath at the angle of intersection. - The at least one wing includes a plurality of wings that extend outward from the body and form a plurality of angled support surfaces. - The at least one angled support surface comprises a first angled support surface and a second angled support surface. - The first angled support surface forms a first intersection angle, and the second angled support surface forms a second intersection angle. - The first angled support surface has a first spacing distance (Hs1), and the second angled support surface has a second spacing distance (Hs2) with respect to the probe axis of the camera probe. - The support surface forms a channel that receives the tool shaft and aligns it with the intersection angle. - The intersection angle defines the intersection point between the camera axis extending along at least a portion of the camera probe and the work axis extending along at least a portion of the tool shaft. - The working axis extends from the tool shaft to the working end or actuator of the surgical tool connected to the tool shaft, and / or - The at least one support surface comprises a plurality of angled support surfaces, each having an intersection angle of a different size.

[0057] According to another aspect of the present disclosure, a camera device for operating in conjunction with a surgical tool comprises a tool shaft. The camera device comprises a body having at least one angled support surface defining an intersection angle; a camera probe connected to the body at a proximal end portion and having a probe length (Lp) extending to a distal end portion; the proximal end portion being spaced from the support surface by a distance (Hs) formed by the connection between the body and the support surface; and an optical element connected to the distal end portion of the camera probe, defining a field of view having a camera axis or focal axis, wherein the support surface receives the tool shaft and aligns the field of view with the tool axis at the working distance of the camera, and the camera probe is held in contact with the tool shaft by a collar extending around the tool shaft and at least a portion of the camera device.

[0058] Depending on the embodiment, the present disclosure may implement one or more of the following structures or configurations in various combinations. - The collar forms part of the cannula in which the camera probe and the tool shaft extend in the operating configuration. - The collar is formed of a deformable or elastic material (e.g., polymer, silicon, etc.) that connects to the angle support surface and maintains the tool shaft. - The cannula comprises at least one lumen for receiving the tool shaft and the camera shaft, the outer circumferential wall of the at least one lumen surrounding the tool shaft and the camera shaft, and connecting to the angled support to hold the tool shaft. - The at least one lumen comprises a first lumen for receiving the tool shaft and a second lumen for receiving the camera probe, and the outer wall is formed by the body of the cannula that forms the first lumen and the second lumen. - The at least one lumen comprises a first lumen for receiving the tool shaft and a second lumen for receiving the camera probe, and the outer wall is formed by the body of the cannula that forms the first lumen and the second lumen. - The angled support surface engages with the tool shaft and defines the distance between the camera probe and the tool shaft, and / or - The main body connects to the camera probe and forms an enclosure or housing that forms the intersection angle with respect to the camera axis.

[0059] According to yet another aspect of the present disclosure, a camera device for operating in conjunction with a surgical tool comprises a tool shaft. The camera device comprises a body having at least one support surface defining an intersection angle; a camera probe connected to the body at a proximal end and having a probe length (LP) extending to a distal end; the proximal end portion separated from the support surface by a distance (Hs) formed by the connection between the body and the support surface; and an optical element connected to the distal end portion of the camera probe, defining a field of view having a camera axis or focal axis, wherein the support surface receives the tool shaft and aligns the field of view with the tool axis of the surgical tool at the working distance of the camera, and the intersection angle defines the intersection point between the camera axis extending along at least a portion of the camera probe and the tool axis extending along at least a portion of the tool shaft.

[0060] Depending on the embodiment, the present disclosure may implement one or more of the following structures or configurations in various combinations. - The camera probe extends from the main body to the probe length, and the probe length is defined by the interval distance (Hs) obtained by dividing the tangent of the intersection angle of the camera device's field of view by the working distance (Lf). - The working distance (Lf) is defined as the intersection of the camera axis and the tool axis. - The probe length (LP) is defined based on the intersection angle. - The probe length (LP) is less than the intersection distance Dint at the intersection angle of the camera axis and the tool axis. - The probe length is smaller than the crossover distance Dint by the working distance (Lf) of the camera. - The at least one support surface comprises a first angled support surface having a first intersection angle and a first spacing, and a second angled surface having a second intersection angle and a second spacing. - The first intersection angle and the first interval define a first working length of the field of view along the camera axis, and the second intersection angle and the second interval define a second working length of the field of view along the camera axis. - The first intersection angle is different from the second intersection angle, the first interval is different from the second interval, and / or - The camera axis and the tool axis intersect at the first and second intersection angles with a crossing distance Dint, and the crossing distance is kept constant with respect to the first and second support surfaces.

[0061] According to a further aspect of the present disclosure, the imaging system comprises a plurality of surgical instruments having surgical tools and / or camera devices, a plurality of tracking devices connected to the plurality of surgical instruments, and a controller. The controller is configured to track at least one of the orientation and position of the surgical instruments in a surgical coordinate system, to generate a graphic representation of the surgical instruments based on the orientation and position, and to control the output associated with the selected surgical instrument in response to the selection of a graphic representation of the surgical instrument or an associated icon.

[0062] Depending on the embodiment, the present disclosure may implement one or more of the following structures or configurations in various combinations. - The output associated with the selected surgical instrument includes a command to display an image feed associated with the first camera of the camera device. - The output associated with the selected surgical instrument includes a command to display an image feed associated with the second camera of the camera device. - The controller is configured to generate anatomical graphics showing a portion of the anatomical features within the surgical coordinate system for the plurality of surgical instruments. - The plurality of tracking devices include patient sensors configured to monitor the orientation and position of the patient's anatomical features, and / or - The tracking device comprises at least one of a radio frequency communication interface, a computer vision system, and a flexible tether configured to track the relative position and orientation of the surgical instrument in the surgical coordinate system.

[0063] Naturally, any described process or step within a described process may be combined with other disclosed processes or steps to form a structure within the scope of this apparatus. The exemplary structures and processes disclosed herein are for illustrative purposes only and should not be construed as limitations.

[0064] Furthermore, it goes without saying that the aforementioned structure and method can be modified and altered without deviating from the concept of this device. And naturally, these concepts are intended to be covered by the following claims, unless otherwise explicitly stated in the language of those claims.

[0065] The above description is to be considered only for the exemplary embodiments. Those skilled in the art, or those who make or use the apparatus, will conceive of modifications to the apparatus. Therefore, the drawings and the embodiments described above are for illustrative purposes only and are not intended to limit the scope of the apparatus, which is understood to be defined by the following claims, which shall be interpreted in accordance with the principles of patent law, including the doctrine of equivalents. [Explanation of Symbols]

[0066] 10 Camera device 10a First camera device 10b Second camera device 16 Surgical Imaging Systems 18 Main unit 20 Camera probes 20a Proximal end portion 20b Distal end section 22 Support surface 22a First support surface 22b Second support surface 22c Third support surface 22d Fourth support surface 22e Fifth support surface 24 Tool Shafts 30 focal areas 32. First Perspective 34 Working end 36 Colors 38 Crossed Cannula 40 lumen 40a First lumen 40b Second lumen 42 External environment 44 Patient Cavities 48 channels 52 Angle orientation 52a Boundary surface 52b wings 52c ridge 52d flange 54 displays 56a First Perspective 56b A Second Perspective 56c The Third Perspective 58. Second Perspective 60 Video Controllers 64 Surroundings 64a Lower peripheral wall 66 Viewing window 66a First viewing window 66b Second viewing window 66c Viewing window 70 triangle 70a First leg 70b Second leg 70c hypotenuse 72 Target area 80 Intermediate surface 82 Contour Surface 90 equipment 92 The Third Perspective 100 Anatomical Graphics 102 Surface description 104 Internal depiction 106 Areas of Interest 110 Viewing window 114 Tracking device 116 Surgical Coordinate System 118 Local Coordinate System 118a First local coordinate system 118b Second local coordinate system 118c Third local coordinate system 120 Graphics 122 Flexible Tether 126 View Icons 130 Camera orientation sensor 132 Camera body 152 User Interface 160 Scope Orientation Sensor 190 Control circuits 192 Light source 194 Image Sensors 196 User Interface 198 processors 200 memory 204 Format Circuit 206 User Interface 210 servers 212 Surgical Control Console 214 I / O circuit

Claims

1. A camera device for operating in conjunction with a surgical tool equipped with a tool shaft, A main body having at least one angled support surface that defines the intersection angle, The probe is connected to the main body at the proximal end and extends to the distal end, with a probe length (L Ｐ A camera probe having ), wherein the proximal end portion is separated from the support surface by a distance (Hs) formed by the connection between the main body and the support surface, A camera device comprising: an optical element for a camera that defines a field of view in relation to the distal end portion, wherein the support surface receives the tool shaft and aligns the field of view with the tool axis at the working distance of the camera.

2. The camera device according to claim 1, wherein the main body is connected to the outer peripheral wall of the camera probe to form a sheath.

3. The camera device according to claim 2, wherein the main body forms at least one wing extending from the sheath at the intersection angle.

4. The camera device according to claim 3, wherein the at least one wing comprises a plurality of wings that extend outward from the main body and form a plurality of angled support surfaces.

5. The camera device according to any one of claims 1 to 4, wherein the at least one angled support surface comprises a first angled support surface and a second angled support surface.

6. The camera device according to claim 5, wherein the first angled support surface forms a first intersection angle, and the second angled support surface forms a second intersection angle.

7. The first angled support surface is located at a first spacing distance (H ｓ１ ) and the second angled support surface has a second spacing distance (H) with respect to the probe axis of the camera probe. ｓ２ A camera device according to any one of claims 1 to 6, having )

8. The camera device according to any one of claims 1 to 7, wherein the support surface forms a channel for receiving and aligning the tool shaft at the intersection angle.

9. The camera device according to any one of claims 1 to 8, wherein the intersection angle defines the intersection point between a camera axis extending along at least a portion of the camera probe and a work axis extending along at least a portion of the tool shaft.

10. The camera device according to any one of claims 1 to 9, wherein the working axis extends from the tool shaft to the working end or actuator of the surgical tool connected to the tool shaft.

11. The camera device according to any one of claims 1 to 10, wherein the at least one support surface comprises a plurality of angled support surfaces, each having an intersection angle of a different size.

12. A camera device for operating in conjunction with a surgical tool equipped with a tool shaft, A main body having at least one angled support surface that defines the intersection angle, The probe is connected to the main body at the proximal end and extends to the distal end, with a probe length (L ｐ A camera probe having ), wherein the proximal end portion is separated from the support surface by a distance (Hs) formed by the connection between the main body and the support surface, A camera device comprising: an optical element connected to the distal end portion of the camera probe, defining a field of view having a camera axis or focal axis, wherein the support surface of the optical element receives the tool shaft and aligns the field of view with the tool axis at the working distance of the camera, wherein the camera probe is held in contact with the tool shaft by a collar extending around the tool shaft and at least a portion of the camera device.

13. The camera device according to claim 12, wherein the collar forms a portion of the cannula into which the camera probe and the tool shaft extend in an operational configuration.

14. The camera device according to claim 13, wherein the collar is formed of a deformable or elastic material (e.g., polymer, silicon, etc.) that connects to the angled support surface and maintains the tool shaft.

15. The camera device according to claim 13, wherein the cannula comprises at least one lumen for receiving the tool shaft and the camera shaft, and the outer peripheral wall of the at least one lumen surrounds the tool shaft and the camera shaft and holds the tool shaft in relation to the angled support.

16. The camera device according to claim 15, wherein the at least one lumen comprises a first lumen for receiving the tool shaft and a second lumen for receiving the camera probe, and the outer circumferential wall is formed by the body of the cannula that forms the first lumen and the second lumen.

17. The camera device according to claim 15, wherein the at least one lumen comprises a first lumen for receiving the tool shaft and a second lumen for receiving the camera probe, and the outer circumferential wall is formed by the body of the cannula that forms the first lumen and the second lumen.

18. The camera device according to any one of claims 12 to 17, wherein the angled support surface engages with the tool shaft and defines the distance between the camera probe and the tool shaft.

19. The camera device according to any one of claims 12 to 18, wherein the main body connects to the camera probe and forms an enclosure or housing that forms the intersection angle with respect to the camera axis.

20. A camera device for operating in conjunction with a surgical tool equipped with a tool shaft, A main body having at least one support surface that defines the angle of intersection, The probe is connected to the main body at the proximal end and extends to the distal end, with a probe length (L Ｐ A camera probe having ), wherein the proximal end portion is separated from the support surface by a distance (Hs) formed by the connection between the main body and the support surface, A camera device comprising: an optical element connected to the distal end portion of the camera probe and defining a field of view having a camera axis or focal axis, wherein the support surface receives the tool shaft, aligns the field of view with the tool axis of the surgical tool at the working distance of the camera, and defines the intersection angle between the camera axis extending along at least a portion of the camera probe and the tool axis extending along at least a portion of the tool shaft.

21. The camera probe extends from the main body to the probe length, and the probe length is such that the tangent of the intersection angle is equal to the working distance (L) of the field of view of the camera device. ｆ The camera device according to claim 20, defined by the interval distance (Hs) obtained by dividing by the result of subtracting ).

22. The aforementioned working distance (L ｆ The camera device according to any one of claims 20 to 21, wherein the intersection point between the camera axis and the tool axis is defined as the intersection point between the camera axis and the tool axis.

23. The probe length (L) Ｐ The camera device according to any one of claims 20 to 22, wherein the angle is defined based on the aforementioned intersection angle.

24. The probe length (L Ｐ ) is less than the distance difference D ｉｎｔ at the intersection angle of the camera axis and the tool axis, for the camera device according to claim 23.

25. The probe length is the working distance (L) of the camera. ｆ ) only the aforementioned crossing distance D ｉｎｔ A camera device according to claim 23, which is shorter than the camera device described in claim 23.

26. The camera device according to any one of claims 20 to 25, wherein the at least one support surface comprises a first angled support surface having a first intersection angle and a first spacing, and a second angled surface having a second intersection angle and a second spacing.

27. The camera device according to claim 26, wherein the first intersection angle and the first spacing define a first working length of the field of view along the camera axis, and the second intersection angle and the second spacing define a second working length of the field of view along the camera axis.

28. The first intersection angle is different from the second intersection angle, The first interval is different from the second interval, The camera device according to claim 27, wherein at least one of the following is included.

29. The camera axis and the tool axis intersect at the first and second intersection angles, respectively, at a distance D. ｉｎｔ The camera device according to claim 28, wherein the lines intersect, and the intersection distance remains constant with respect to the first support surface and the second support surface.

30. An imaging system, Multiple surgical instruments, including surgical tools and / or camera devices, Multiple tracking devices connected to the aforementioned multiple surgical instruments, It is a controller, Tracking at least one of the orientation and position of the surgical instrument in the surgical coordinate system, Based on the orientation and position, a graphic representation of the surgical instrument is generated. An imaging system including a controller configured to control the output associated with the selected surgical instrument in response to the selection of the graphic representation of the surgical instrument or an associated icon.

31. The imaging system according to claim 30, wherein the output associated with a selected surgical instrument includes a command to display an image feed associated with a first camera of the camera device.

32. The imaging system according to claim 31, wherein the output associated with the selected surgical instrument includes a command to display an image feed associated with the second camera of the camera device.

33. The imaging system according to claim 31, wherein the controller is further configured to generate anatomical graphics showing a portion of the anatomical features in the surgical coordinate system for the plurality of surgical instruments.

34. The imaging system according to claim 33, wherein the plurality of tracking devices include patient sensors configured to monitor the orientation and position of the patient's anatomical features.

35. The imaging system according to any one of claims 30 to 34, wherein the tracking device includes at least one of a radio frequency communication interface, a computer vision system, and a flexible tether configured to track the relative position and orientation of the surgical instrument in the surgical coordinate system.

Images