Surgical imaging system with selective view

Abstract

The surgical imaging system includes a scope having optical elements aligned to a first tilt angle that defines the native viewpoint. An image sensor is configured to capture source image data within the field of view transmitted through the optical elements. A controller is configured to control the capture of source image data at the native viewpoint at the first tilt angle. The controller further selects a first subset of the source image data that includes the first portion of the field of view simulating a second tilt angle, and selectively outputs the modified source image data and the first image data to the display screen.

Description

[Background technology]

[0001] This disclosure relates, in general, to surgical imaging systems, and more specifically, to surgical cameras and control systems configured to selectively adjust the field of view. Today's surgical procedures employ a variety of advanced techniques to improve patient outcomes. However, such procedures can increase the complexity of the procedure and, accordingly, increase the number of specialized tools and associated equipment required to complete the procedure. This disclosure provides an improved imaging system that can improve the operation associated with some of these procedures. [Overview of the Initiative] [Means for solving the problem]

[0002] Surgical imaging systems have evolved to include a variety of specialized devices, including cameras and imaging devices designed to access remote patient cavities. These devices may include specialized optics and lenses that can be provided at different inclination angles, allowing the device to capture a field of view that can target anatomical features at the corresponding inclination angles. Interchangeable scopes with different inclination angles can offer highly specialized operation, but they may also require removal and replacement during surgical procedures to provide such functionality. In various implementations, this disclosure provides a surgical imaging system comprising a scope having optical elements aligned to an inclination angle that defines a native viewpoint. As will be discussed in detail in the following embodiments, the controller of the imaging system may process source image data captured at the native viewpoint of the scope and selectively generate modified image data having an inclination angle and a simulated viewpoint angle relative to the native viewpoint.

[0003] During operation, the scope of a surgical imaging system may capture source image data at its tilt angle throughout a variety of operating modes so that the resolution or dimensions of the field of view are consistently captured and processed to provide the operation discussed herein. For example, in various embodiments, the scope may include optical elements aligned at a tilt angle oriented at approximately 45° at the distal end. This orientation may define a native viewpoint associated with the scope and optical elements with respect to the scope axis along which the source image data is captured. Based on the source image data captured at the native viewpoint, the system's controller may output display data having the original viewpoint, or it may selectively generate modified image data having a different simulated viewpoint angle, offset from the native viewpoint. As discussed in the various detailed embodiments described below, modified source image data with a simulated viewpoint angle corresponds to modified image data that can be generated by remapping rays aligned to pixels in the field of view, and image data with offset central rays aligned to different incident angles and simulated viewpoint angles may be simulated. Therefore, the disclosed system and method may not only adjust or crop a portion of the field of view for display, but the system may also provide selective modification of source image data so that it appears as if it were captured from a different viewpoint offset from the native viewpoint.

[0004] These and other features, purposes, and advantages of this disclosure will become apparent upon reading the following description with reference to the attached drawings. [Brief explanation of the drawing]

[0005] [Figure 1] This is an explanatory diagram showing a surgical imaging system. [Figure 2A] This figure shows simulated image data captured from a native speaker's perspective. [Figure 2B] This figure shows image data modified to have a simulated viewpoint that has an angle with respect to the native viewpoint shown in Figure 2A. [Figure 2C] This figure shows image data modified to have a simulated viewpoint that has an angle with respect to the native viewpoint shown in Figure 2A. [Figure 2D] This is a flowchart showing a method for generating image data from a simulated viewpoint. [Figure 3A] This is a schematic diagram representing the incidence of multiple light rays on an optical element from a native perspective. [Figure 3B] This is a schematic diagram showing multiple correction rays generated to simulate corrected image data having a simulated viewpoint offset from the native viewpoint shown in Figure 3A. [Figure 3C] This is a schematic diagram showing multiple correction rays generated to simulate corrected image data having a simulated viewpoint offset from the native viewpoint shown in Figure 3A. [Figure 4] This is a representative diagram showing modified image data with a simulated viewpoint displayed on the screen, including an orientation queue for the simulated data relative to the source data. [Figure 5] This is a representative diagram showing modified image data with a simulated viewpoint displayed on the screen, including an orientation queue for the simulated data relative to the source data. [Figure 6A] An exemplary orientation queue is shown, illustrating a portion of the source image data indicated by the modified image data. [Figure 6B] An exemplary orientation queue is shown, illustrating a portion of the source image data indicated by the modified image data. [Figure 7A] An exemplary orientation queue is shown, illustrating a portion of the source image data indicated by the modified image data. [Figure 7B] An exemplary orientation queue is shown, illustrating a portion of the source image data indicated by the modified image data. [Figure 8] This is a modified block diagram illustrating the surgical imaging system described herein. [Modes for carrying out the invention]

[0006] The following description refers to the attached drawings, which illustrate specific possible implementations. Wherever possible, the same reference numerals are used throughout the drawings to refer to the same or similar parts. Naturally, other implementations may be utilized and structural and functional modifications may be made without departing from the scope of this disclosure.

[0007] Referring to Figure 1, the disclosure generally provides a surgical imaging system 10 that can be implemented with or communicate with a wide range of surgical consoles and devices, in particular a camera or scope configured to capture image data within the surgical field 12. As shown, the system 10 includes a scope 16 and a native viewpoint P with respect to the distal end portion 16b of the scope 16. n The camera device 14 may include a camera body 18 configured to capture image data. The proximal end portion 16a of the scope 16 may be connected to the camera body 18 and provide a user interface 20 which may include one or more user inputs (e.g., buttons, switches, dials, etc.). In this configuration, the operator of the camera device 14 can adjust and manipulate the field of view 12 captured by the scope 16 by operating and interacting with the camera body 18 and the user interface 20.

[0008] In various implementations, the scope 16 may include an optical element 28 that connects to the camera device 14 at its distal end portion 16b. The optical element 28 is the native viewpoint P nThat is, the image sensor of the camera device 14 may be aligned at an inclination angle θ that can define a camera viewpoint that captures source image data within the field of view 12. The image sensor may be located within the camera body 18 and optically coupled to an optical element 28 via optical coupling (e.g., a fiber optic cable). In some implementations, the image sensor, control circuit, and optical element 28 may all be located in the distal end portion 16b of the scope 16 (e.g., a camera with a tip). In such implementations, the illustrated camera body 18 may be omitted or incorporated to accommodate the operation of the image sensor and / or the circuitry associated with the user interface 20 and the relevant user operating surface of the camera device 14. In various implementations, the native viewpoint P may be as illustrated later by Figure 3. n The inclination angle θ defining this may be configured to position the central ray CR, which passes through the optical element 28 (e.g., a lens, prism, lens array, etc.) with little or no optical refraction, among a plurality of light rays incident on the optical element 28. Accordingly, the camera device 14 may be implemented in various hardware configurations to provide the various features and operations discussed herein.

[0009] Referring further to Figure 1, in various implementations, the imaging system 10 may include a controller 30 that can be integrated into the video console 32. During operation, the controller 30 controls the native viewpoint P, which can be defined by the tilt angle θ as described above. n The system can then receive source image data from the image sensor of the camera 18, which depicts a scene representing the light captured in the field of view 12. In various implementations, the field of view 12 defined by the optical element 28 may correspond to a wide-angle field of view, which in this specification is referred to as the full field of view or the entire field of view φ f It can be called this. In various implementations, the entire field of view φ f This may include a field of view greater than 110° or 120°, and in some implementations, may exceed 140° or 180°. In this embodiment, the total field of view φ f This is approximately 45° from the native viewpoint P nIt can correspond to a 140° field of view that can be captured. As discussed in the following examples, the controller 30 of the system 10 can be offset with respect to the tilt angle θ associated with the native viewpoint P n to provide generation of simulated modified image data representing a field of view 12 captured at one or more simulated viewpoint angles (e.g., θ s1 , θ s2 , etc.).

[0010] As described above, the image sensor of the camera 18 can capture source image data 50 having a central ray CR aligned at a tilt angle θ along the native viewpoint P n . During operation, the controller 30 can selectively generate modified image data having simulated viewpoint angles θ s1 , θ s2 that can be offset from the tilt angle θ. For example, as discussed in more detail with reference to FIG. 2, the controller 30 processes source image data indicating the full field angle φ f of the field of view 12 to generate modified image data 36 having a simulated viewpoint angle θ s1 offset from the tilt angle θ to provide a simulated viewpoint P s1 . In an example of a first simulated viewpoint P s1 , the simulated viewpoint angle θ s1 is offset by approximately 15° to provide the first simulated viewpoint P s1 at an angle of 30°. Similarly, the controller 30 can remap the source image data (e.g., pixel values associated with the image data) to generate modified image data 36 representing a second simulated viewpoint P s2 , and this modified image data can be remapped to represent a second simulated viewpoint angle θ s2 of approximately 70°. As discussed in the following examples, the simulated viewpoints P s1 and P s2 , as well as the corresponding simulated viewpoint angles θ s1 , θ s2This can be selectively activated to generate the corresponding corrected image data 36 and output it to the display screen 40. Simulated viewpoint angle θ s1 θ s2 While referring to specific angle values, the simulated viewpoint angle is the corresponding tilt angle in the range of approximately 0° to 90°, or the native viewpoint P in the range of approximately -30° to 45°. n It could be within the range of the offset from.

[0011] Before discussing the generation of corrected image data in more detail, the operation of the video console 32 may provide one or more visual cues 42 that can help the user visually recognize the relationship between the scope 16 and the image data depicted on the display screen 40 via the orientation cue 42a. In addition, the visual cue 42 represents the entire field of view φ represented by the corrected image data 36. f The relative position cue 42b may include a relative position cue 42b that can indicate a subset of source image data extending over the entire field of view φ. In the shown embodiment, the relative position cue 42b or viewpoint cue may be available for viewing on display for a subset of the field of view 12 that can be depicted by the modified image data 36, ​​across the entire field of view φ. f The region of the source image data within can be identified.

[0012] Referring to Figures 2A-2C, an example of the corrected image data 36 (for example, the first corrected image data 36a, the second corrected image data 36b) is the entire field of view φ of the field of view 12. fThis is shown with respect to source image data 50 that depicts the subject. As will be discussed in various embodiments, the disclosure may provide a display of modified image data 36 as one or more subsets 52 that can depict the detailed portion of the source image data 50. In the embodiments shown, the subset 52 may include a first subset 52a and a second subset 52b of the source image data 50, which can be modified and / or remapped by the controller 30 to generate a first modified image data 36a and a second modified image data 36b, respectively. As shown in Figure 2A, the subset 52 of the source image data 50 used to generate the modified image data 36 may be consistently aligned with the rotation angle γ of the scope 16 relative to the camera 18. In this configuration, the tilt angle θ and the simulated viewpoint angle θ s1 θ s2 This can be consistently aligned with the rotation angle γ of the scope 16 and communicated to the user via one or more of the visual cues 42. In this way, the imaging system 10 can communicate the simulated viewpoint angle P s Furthermore, it may provide selective output and display of modified image data 36 representing any of the source image data 50. The field of view 12 and various viewpoints P of the subset 52. n , P s1 , P s2 Such selective display of associated image data can provide improved flexibility in the operation of the camera device 14.

[0013] During operation, the controller 30 selectively outputs the source image data 50 and the first or second modified image data 36a, 36b, to display the full field of view φ f The entire field of view 12, or a simulated scope or viewpoint angle θ s1 θ s2 This may indicate either subset 52a or 52b corresponding to φ. For example, the controller 30 has a full field of view φ f The first view 54a shows the first simulated viewpoint angle θ. s1 A second view 54b showing source image data 50 of the corresponding region, and a second simulated viewpoint angle θ. s1A third view 54c showing source image data 50 of the corresponding region may be repeated. Each of views 54a, 54b, and 54c may be selectively output to the display device 40 in response to input to the user interface 20 of the camera device 14. In this way, the controller 30 may present a first view 54a showing a wide field of view encompassing both the first subset 52a and the second subset 52b. In addition, the controller 30 may selectively generate and output a second view 54b or a third view 54c corresponding to a subset 52 of the first view 54a.

[0014] Referring further to Figures 2A-2C, the generation and display of views 54 may involve various image processing, masking, cropping, or similar procedures. In various embodiments, the generation of visual cues 42 (e.g., orientation cues 42a and relative position cues 42b) may involve the generation of virtual masks 56a extending around each of views 54a, 54b, and 54c. The virtual masks 56a may be superimposed on a field stop mask that can define the periphery of the source image data 50. For example, in various cases, a field stop mask 56b associated with the scope 16 of the camera device 14 may include various features 56c (e.g., shapes, pointers, irregularities, etc.) that indicate the rotation angle γ of the scope 16 relative to the camera body 18 and can correspond to additional changes around the periphery of the source image data 50. These changes and features 56c associated with the field stop mask 56b may distract the user of the system 10. Therefore, the controller 30 may generate a virtual mask around the source image data 50 to create a smooth surrounding boundary that extends from the edges of the source image data 50 or the modified image data 36a, 36b to the edges of the window 56d output to the display device 40.

[0015] In addition to generating a virtual mask 56a around the source image data 50, the controller may similarly generate a virtual mask 56a around views 54b, 54c that show modified image data 36a, 36b corresponding to subsets 52a, 52b. For each of views 54a, 54b, 54c, the virtual mask 56a may include visual cues 42 (e.g., orientation cue 42a and relative position cue 42b). During operation, the controller 30 may detect one or more of the features 56c of the source image data 50 to detect the rotation angle γ of the scope 16. In this way, the controller 30 can pinpoint the position around the source image data 50 to position the visual cues 42 and accurately determine the rotation angle γ. Similar to the source image data 50, the virtual mask 56a may be generated and applied in each of the second view 54b and the third view 54c to frame and encapsulate subsets 52 corresponding to the modified image data 36. Therefore, the controller 30 can selectively output each of the views 54a, 54b, and 54c and apply the virtual mask 56a and / or visual cue 42 to assist the operation of the system 10.

[0016] To generate views 54a, 54b, and 54c, the controller 30 may further apply one or more image processing techniques, filters, and / or algorithms, referred herein to as image correction algorithms, to improve the illumination, contrast, etc., of the source image data 50. When outputting the source image data 50, the controller 30 considers the full field of view φ fImage data corresponding to a field of view 12 spanning a wide area can be processed. In addition, when applying an image correction algorithm to a second view 54b and a third view 54c, the controller 30 may limit the scope of the image correction algorithm to the corresponding first subset 52a or second subset 52b. For example, when applying an automatic exposure algorithm to generate a second view 54b, the controller 30 may limit the image data processed from the source data 50 to pixels or information located within the first subset 52a. In this way, the image correction algorithm and / or filter applied to generate the second view 54b may be limited to the range and attributes contained in the first subset 52a so that the relevant illumination and features are optimized for the subset 52a rather than the entire source image data 50.

[0017] Referring here to Figures 2A-2D, flowcharts are shown illustrating exemplary methods 60 for generating image data, as shown in subsets 52a and 52b. As provided in the following steps, the controller 30 may process the image data through procedures including dewarping of the source image data 50, subsampling of portions of the image data corresponding to rotation angles γ, and rewarping of the image data to generate modified image data 36. As previously mentioned, the modified image data 36 is generated from the simulated viewpoint P shown in Figure 1. s This can correspond to one of the following. As will be discussed in more detail with reference to Figures 3A-3C, the generation of the corrected image data 36 is generally done by digitally manipulating the source image data 50 to cover the entire field of view φ f The process may include removing distortion associated with the field of view 12 across the field of view P and generating normalized image data. In addition, subsets 52a, 52b may be sampled from the normalized source data and further distorted or manipulated to generate corrected image data 36. In this way, the corrected image data 36 is a simulated viewpoint P s It may be displayed in a way that imitates one of them.

[0018] As discussed herein, image warping may include various steps that can modify the position or proportion of the source image data 50 through coordinate or pixel mapping or various forms of geometric transformations. For example, the source image data may be a simulated viewpoint P s The image data may be distorted, tilted, rotated, and / or translated to simulate the visual aspects or features of the viewpoint Ps. The warping or dewarping algorithms described herein may include forward warping, reverse warping, spline warping, mesh warping, and other image processing methods to normalize the distortion of the source image data 50 and / or simulate the characteristic distortion or features of the simulated viewpoint Ps. In this way, the controller 30 can selectively generate image data at the simulated viewpoint Ps to suit various applications.

[0019] During operation, method 60 may be started in response to the activation of the camera device 14 and the reception of source image data 50 (62). Once activated, the controller 30 may start processing the source image data 50 to display one of views 54a, 54b, or 54c (64). In the following embodiments, the selected view is the full field of view φ f This will be explained with reference to the first view 54a, which is shown. However, naturally, view 54 may correspond to any of views 54a, 54b, or 54c at startup. In the shown embodiment, input to the user interface 20 may cause the controller 30 to generate and display a first subset 52a of the source image data 50 via an image processing routine (66). During operation, the image processing routine 66 may process the source image data 50 frame by frame or selectively by applying a distortion correction algorithm (68). The distortion correction algorithm may correct distortion, or otherwise normalize the source image data 50 to the full field of view φ fThe controller 30 may compensate for any irregularities or variations in magnification of one or more lenses or optical systems used to capture the source image data 50. In this way, the controller 30 controls the simulated viewpoint P s For additional processing to generate corrected image data 36, ​​normalized image data may be generated.

[0020] Simultaneously with, or sequentially with, the generation of normalized image data in step 68, the controller 30 may determine the portion of source image data 50 from which to sample a first subset 52a based on the rotation angle γ of the scope 16 (70). As previously stated, the rotation angle γ of the scope 16 may be determined in response to one or more of the features 56c of the source image data 50, which may remain in a fixed relationship with respect to the camera 18 as the scope 16 rotates. Once the rotation angle γ of the scope 16 is determined based on the features 56c, the controller 30 may continue the image processing routine 66 by selecting a subset 52a of the source image data 50 in step 70. As shown in Figure 2A, the subset 52a is positioned tangentially to the periphery of the source image data 50, or otherwise offset from the center of the source image data 50 to simulate the viewpoint angle θ s1 θ s2 The field of view of scope 16 in one of the subsets may be simulated. After the selection of the first subset 52a, the image processing routine 66 may continue by processing the first subset 52a with a rewarping algorithm, as will be further discussed with reference to Figures 3A-3C (72). For example, the rewarping algorithm processes the modified image data in subset 52a to simulate the viewpoint angle θ s1 θ s2 The inherent magnification, zoom, and / or irregularity associated with one of the selected ones can be simulated. Method 60 can continue processing the image data for display throughout the operation of the camera device 14, as described in steps 62-74.

[0021] During operation, method 60 can operate as a continuous image feed via an image processing pipeline. Thus, in response to a change in the rotation angle γ of scope 16, a subset 52 of source image data 50 can be reselected or updated to correspond to the portion of source image data 50 aligned with the rotation angle γ. As shown by the rotation arrow 61 in Figure 2A, the subset 52 of source image data 50 presented in each image frame can be updated based on the rotation angle γ of scope 16 at the time of capture of each frame. Thus, rotation of scope 16 relative to camera device 14 can result in a progressive rotation of the position of subset 52 in source image data 50 and the corresponding generation of modified image data 36 for display. In the shown embodiment, the position of subset 52 is updated along the position represented by the rotation arrow 61 over a period corresponding to the change in rotation angle γ from a first subset 52a to a second subset 52b.

[0022] In some implementations, the source image data 50 may be selectively displayed in various arbitrary formats for each of the views 54a, 54b, and 54c. For example, as described, the source image data 50 may be distortion-corrected, normalized, or flattened, as discussed in step 68, to correct for distortion, irregularity, or magnification variations of the optical element 28. In some cases, the inherent P n Source image data 50, or simulated viewpoint P s Instead of displaying the corrected image data 36, ​​the controller 30 may be configured to display the normalized image data from one of the views 54a, 54b, or 54c. In addition, the corrected image data 36 may be corrected in other ways (e.g., different levels of distortion, magnification, etc.) and presented in a custom display format. Generally, the custom display format is a simulated viewpoint P sThe image may be modified using techniques similar to those used to generate it. However, the custom display format may adjust, distort, and / or enlarge the normalized image data to suit the user's preferences or a variety of pre-set viewpoints and views. Thus, the controller 30 may be configured to selectively display normalized or modified image data in a custom display format for each of the views 54 or corresponding subsets 52 of the field of view 12 captured by the camera device 14.

[0023] Referring here to Figures 3A-3C, exemplary methods and processing steps used by the controller 30 to generate corrected image data 36 will be described in more detail. As previously mentioned, source image data 50 can be captured by a camera device 14, which is represented as an image sensor 90. As previously mentioned, the source image data 50 defines a central ray CR that passes through the center or focal center of the optical element 28 and reaches the image sensor 90 without being deflected, from the native viewpoint P n It may be captured by the controller 30. In some embodiments, the central ray CR may vary based on tolerances and alignments related to manufacturing and assembly, and the controller 30 may detect the actual position of the optical element 28 by identifying or calculating it. By detecting the actual position of the central ray CR, the controller 30 can determine the simulated viewpoint P regardless of variations between the scopes 16. s It can consistently generate the following. The detection of the actual position of the central ray CR, based on the position where light passes through the optical element without being deflected, is performed by the virtual mask 56a, which is the native viewpoint P n The central ray CR and the simulated viewpoint P s It may be even more beneficial to position the virtual mask 56a so that it is arranged radially around the corrected central ray MCR.

[0024] Simulated viewpoint P sTo generate the corrected image data 36, ​​the controller 30 offsets the central ray CR relative to the corrected central ray MCR, and adjusts the native viewpoint P to correspond to the angular offset of the corrected central ray MCR. n Each of the multiple light rays 92 incident on the optical element 28 can be remapped. As shown in the embodiments in Figures 3B and 3C, the native viewpoint P n Multiple rays 92 associated with this view P are simulated along the modified central ray MCR. s Each of these can be modified to align with the respective. In this way, the modified image data 36 corresponding to the subset 52 from the source image data 50 corresponds not only to the cropped and enlarged portion of the source image data 50, but also to the simulated viewpoint P. s Further simulated representations of the source image data 50 can be provided, as if captured along one of the following. As described herein, the focal center may correspond to the central portion of the field of view 12, which may not correspond to the geometric center of the lens. For example, the focal center may be offset from the geometric center to limit distortion or to provide the use of an asymmetric lens for various applications.

[0025] In the embodiment shown in Figure 3B, the first modified image data 36a offsets the first modified central ray MCR1 to obtain the first simulated viewpoint angle θ. s1 By aligning them, the first subset 52a can be generated from the source image data 50. As shown, θ s1 This is from a native speaker's perspective. n From the first simulated viewpoint angle θ s1 This is expressed as an offset Δθ1 to the first simulated viewpoint P. The offset Δθ1 remaps each of the multiple correction rays 94 represented by the correction image data 36, ​​which are aligned with the first correction central ray MCR1 and distributed around it, to the source image data 50, to the first simulated viewpoint P. s1 This may represent a calculated offset that needs to be applied by the controller 30 in order to remap so that it appears to be captured along the specified line.

[0026] Similar to the procedure discussed with reference to Figure 3B, Figure 3C shows the procedure for generating the second modified image data 36b along the second modified central ray MCR2. Second simulated viewpoint P s2 Here too, from a native perspective P n and the second simulated viewpoint P s2 It is represented based on the angular offset Δθ2 between them. During operation, the controller 30 may remap the source image data 50 associated with the second subset 52b to align with the second modified central ray MCR2. In addition, the native viewpoint P of the second subset 52b n Multiple rays 92 associated with the second modified central ray MCR2 can be remapped to align with the second modified central ray MCR2 to generate multiple modified rays 94. In this way, the controller 30 can map the source image data 50 to the second simulated viewpoint P s2 It can be modified to appear as if it were captured along those lines.

[0027] To more clearly explain the computational aspects related to the generation of the corrected image data 36, ​​a general consideration is provided here with reference to Figures 3A-3C. Conceptually, the multiple rays 92 shown in Figure 3A may correspond to a unit sphere that defines the central ray CR passing through the center of the optical element 28 so that the central ray CR passes through it without being deflected. As shown in Figures 3B and 3C, the lens or optical element 28 is the first simulated viewpoint P s1 and the second simulated viewpoint P s2 The first offset orientation 28a and the second offset orientation 28b are presented aligned with the viewpoint P s This shows the simulated orientation of the lens associated with each of the following. In addition, the multiple correction rays 94 represent the alignment of the resulting corrected image data 36, ​​which is aligned with the corresponding simulated light and the respective correction central rays MCR1 and MCR2. During operation, the controller 30 corrects the corresponding source image data 50 of the subset 52 corresponding to each of the multiple correction rays 94 to simulate the viewpoint P s1 , P s2The field of view 12 captured along each of these can be simulated.

[0028] More specifically, in the exemplary implementation, the pixel values ​​associated with source image data 50 are the simulated viewpoint P s Each of the multiple corrected rays 94 associated with the simulated viewpoint P can be corrected through a complex series of spherical, polar, and Cartesian calculations. s For all the Cartesian pixels necessary to create the corrected image data 36, ​​the corresponding position of the pixel relative to the corrected central ray MCR can be calculated in two-dimensional polar coordinates. This position can then be scaled according to the proportion of the subset 52 that forms the portion of the source image data 50 shown in the corrected image data 36. The mapping function is then applied to the polar coordinates of each pixel to obtain the simulated viewpoint angle P. s Pixels can be rearranged according to the associated transformation view. Such transformations may depend on specific properties of the optical element 28. For example, if the operation of the optical element 28 corresponds to a tangent mapping function, the inverse tangent function can be applied to the pixel position and polar coordinates relative to the corrected central ray MCR to flatten the image. This operation can similarly be applied to distortion-free methods used to flatten image data. In addition, similar functions can be applied to achieve the same effect for various types of lenses and corresponding lens mapping functions.

[0029] When each pixel associated with the modified image data 36 is transformed, the two-dimensional polar coordinate origin corresponds to the simulated viewpoint P in three-dimensional spherical coordinates. s The modified central ray MCR can be assigned to align with the simulated viewpoint P. sOnce the corrected image data 36 is assigned or assumed, the corresponding pixel data can be calculated for each of the multiple corrected rays 94 aligned with the corrected central ray MCR. The pixel data corresponding to the multiple corrected rays 94 can then be mapped to two-dimensional polar coordinates relative to the source image data 50 using a polar coordinate origin aligned with the central ray CR of the optical element 28. With the pixel data corresponding to the corrected rays 94 normalized in two-dimensional polar coordinates, lens distortion correction may be further applied to adjust the representation of the corresponding pixel data that may arise from distortions associated with the optical element 28. After lens distortion correction, the Cartesian XY position of the source pixels relative to the corresponding portion of the source image data 50 can be calculated by deformalizing the position by the focal length of the optical element 28 and converting each source pixel from polar coordinates to Cartesian coordinates from the source image data 50. Since the desired pixel positions for generating the corrected image data 36 may fall between pixel positions in the source image data 50, the pixel values ​​associated with the source image data 50 can be interpolated to output the corrected image data 36 through various interpolation methods (e.g., bilinear interpolation of the four nearest neighbor pixels). By repeating this process for all pixels in the corresponding subset 52, the simulated viewpoint P is generated. s This may provide remapped pixel information corresponding to the modified image data. While specific computational operations are described in the embodiments described above, alternative methods may be employed to generate simulated viewpoints without departing from the spirit or scope of this disclosure.

[0030] Using the modified image data 36 generated by the controller 30, the video console 32 can output the modified image data 36 to the display screen 40 in various ways. Here, various examples of the display configurations and methods associated with the source image data 50 and the modified image data 36 will be described with reference to FIGS. 4 to 7. As shown in FIGS. 4 and 5, the modified image data 36 can be simultaneously shown on the display screen 40 using one or more visual cues 42. In the illustrated embodiment, the visual cue 42 includes an orientation cue 42a that can specify the rotation angle γ of the scope 16. Additionally, a relative position cue 42b that specifies the positional relationship between the subset 52 of the source image data 50 and the modified image data 36 being displayed may be included. As shown in FIG. 4, the first subset 52a can be shown as a marker, outline, and / or overlay superimposed on the source image data 50 shown on the display screen 40. Similarly, in FIG. 5, the relative position cue 42b can indicate the position of the second subset 52b within the source image data 50 via a similar marker, outline, and / or overlay. In this configuration, the imaging system 10 can provide an intuitive visual cue 42 that ensures that the selected depiction of the source image data 50 and / or the modified image data 36 can be easily determined by viewing the display screen 40.

[0031] As described above, the mapping function can be applied to each of the pixels to relocate the pixels according to the transformed view associated with the simulated viewpoint angle Ps. In some cases, the system 10 can store calibration data in the memory 118 (FIG. 8) that can define the mapping function based on the position and transformation means for one or more types, classes, or specific models of the scope associated with the simulated viewpoint angle Ps. Further, the mapping function generally relates to the native viewpoint P nWhile this can be applied to map simulated viewpoint angles Ps, the controller 30 of system 10 may further identify a specific model of scope 16 and corresponding calibration data. For example, based on the model information detected for scope 16, the controller 30 may determine whether scope 16 is suitable for generating image data at one or more of the simulated viewpoint angles Ps. In addition, the controller 30 may access model-specific calibration data for the identified scope 16 model. In response to the model-specific calibration data, scope 16 updates the mapping function based on the specific calibrations (e.g., viewpoint, magnification, distortion, etc.) for the scope's particular model, thereby updating the mapping function to the native viewpoint P n It is possible to accurately generate one or more simulated viewpoint angles Ps from this.

[0032] As shown in Figures 6A and 6B, the orientation cue 42a is similarly shown. In these embodiments, the relative position cue 42b may correspond to a graphic 100 that can be positioned along the periphery of the depiction of the modified image data 36, ​​corresponding to a portion of the field of view that can be extended from the subset 52 depicted in the modified image data 36. For example, the graphic 100 may correspond to a band surrounding a portion of the periphery 102 of the display window 104. The band formed by the graphic 100 may extend around the periphery 102 along the region in which the source image data 50 can be extended to increase the extent shown within the display window 104. For example, a first simulated viewpoint P s1 If shown in the display window 104, the controller 30 may position the band formed by the graphic 100 to indicate that the rest of the source image data 50 can be extended along the periphery 102 in the direction of the graphic 100. For the visual representation of the extension direction identified by the graphic 100 in Figures 6A and 6B, the corresponding simulated viewpoint P s1 and P s2 However, the entire field of view φ of the field of view 12 captured in the source image data 50 fis shown in FIG. 2A. Thus, the graphic 100 shown in FIGS. 6A and 6B provides a visual representation of the direction with respect to the region of the field of view 12 that can be extended from the subset 52 depicted at the simulated viewpoint P in various embodiments. s can provide a visual representation of the direction with respect to the region of the field of view 12 that can be extended from the subset 52 depicted at the simulated viewpoint P in various embodiments.

[0033] As shown in FIGS. 7A and 7B, the relative position cue 42b can correspond to a preview window 106 positioned adjacent to the display window 104 or outside the periphery 102 showing a portion of the source image data 50 outside the display window 104. Similar to the previous embodiments considered, the relative position cue 42b provided in the embodiments of FIGS. 7A and 7B extends along a portion of the periphery 102 of the display window 104 if the field of view 12 captured by the source image data 50 over the full field angle φ extends outside the subset 52 depicted within the display window 104. Such a depiction of the visual cue 42 not only enables the user of the imaging system 10 to recognize the view presented as corresponding to one of the simulated viewpoints P provided by the modified image data 36 or the source image data 50, but also ensures that the availability and associated content of views not currently presented on the display screen 40 can be intuitively determined by the user. f extends along a portion of the periphery 102 of the display window 104 if the field of view 12 captured by the source image data 50 over the full field angle φ extends outside the subset 52 depicted within the display window 104. Such a depiction of the visual cue 42 not only enables the user of the imaging system 10 to recognize the view presented as corresponding to one of the simulated viewpoints P provided by the modified image data 36 or the source image data 50, but also ensures that the availability and associated content of views not currently presented on the display screen 40 can be intuitively determined by the user. s not only enables the user of the imaging system 10 to recognize the view presented as corresponding to one of the simulated viewpoints P provided by the modified image data 36 or the source image data 50, but also ensures that the availability and associated content of views not currently presented on the display screen 40 can be intuitively determined by the user.

[0034] Referring here to Figure 8, a block diagram of the imaging system 10 is shown. As discussed throughout this disclosure, the system 10 may include a camera device 14 that communicates with a controller 30. The camera device 14 may include a camera controller 110, a light source 112, an image sensor 90, and a user interface 20. In various implementations, the camera device 14 may correspond to an endoscope, laparoscope, arthroscope, etc., formed by a scope 16 in the form of an elongated probe with a narrow distal end 16b suitable for various non-invasive surgical techniques. For example, the distal end 16b may include a diameter of less than 2 mm. As shown, the camera device 14 may communicate with the controller 30 via a communication interface. Although shown to be connected via a conductive connection, the communication interface may correspond to a wireless communication interface operating via one or more wireless communication protocols (e.g., Wi-Fi, 802.11b / g / n, etc.).

[0035] The light source 112 can correspond to various light emitters configured to produce light in the visible and / or near-infrared regions. In various implementations, the light source 112 may include light-emitting diodes (LEDs), laser diodes, or other lighting technologies. The image sensor 90 or image sensor can correspond to various sensors and configurations, including, for example, charge-coupled device (CCD) sensors, complementary metal-oxide-semiconductor (CMOS) sensors, or similar sensor technologies. In various implementations, the camera controller 110 may correspond to a control circuit configured to control the operation of the image sensor 90 and the light source 112, and to process the source image data 50 and / or transmit it to the controller 30 or system controller. In addition, the camera controller 110 may communicate with a user interface 20 which may include one or more input devices, indicators, display devices, etc. The user interface 20 may provide control of the camera device 14, including the invocation of one or more routines, as discussed herein. The camera controller 110 may be implemented by various forms of controllers, microcontrollers, application-specific integrated controllers (ASICs), and / or various control circuits or combinations thereof.

[0036] The controller 30 or imaging controller may include a processor 116 and memory 118. The processor 116 may include one or more digital processing units, such as a central processing unit (CPU) having one or more processing cores, a graphics processing unit (GPU), a digital signal processor (DSP), a field-programmable gate array (FPGA), or an application-specific integrated circuit (ASIC). In some configurations, multiple processing units are combined into a system-on-a-chip (SoC) configuration, while in other configurations, the processing units may correspond to individual components. During operation, the processor 116 executes program instructions stored in memory 118 to perform the operations described herein.

[0037] Memory 118 may comprise one or more data storage devices, including, for example, a magnetic or solid-state drive and a random-access memory (RAM) device for storing digital data. Memory 118 may also comprise one or more stored program instructions, object detection templates, image processing algorithms, etc. As shown, memory 118 may comprise one or more modules that may comprise instructions for processing source image data 50 and generating modified image data 36. For example, processor 116 may access instructions in the memory modules to perform various processing tasks on the image data, including preprocessing, filtering, masking, trimming, and various enhancement techniques, to simulate viewpoint P s1 This can improve the visibility and generation of [the data].

[0038] In some implementations, the controller 30 may correspond to a display controller. In such applications, the controller 30 may include one or more formatting circuits 122 that process image data received from the camera device 14, communicate with the processor 116, and process the image data according to one or more of the operating methods discussed herein. The formatting circuits 122 may include one or more signal processing circuits, analog-to-digital converters, digital-to-analog converters, etc. The display controller may also include a user interface 124, which may be in the form of an integrated interface (e.g., a touchscreen, input buttons, an electronic display, etc.) or may be implemented by one or more connected input devices (e.g., a tablet), peripheral devices (e.g., a keyboard, a mouse, etc.), a foot pedal, a remote switch, etc.

[0039] As shown, the controller 30 also communicates with an external device or server 126, which may be a network, a local or cloud-based server, a device hub, a central controller, or various devices that can communicate with the controller 30, more generally, the imaging system 10 via one or more wired (e.g., Ethernet) or wireless (e.g., Wi-Fi, 802.11b / g / n, etc.) protocols. For example, the controller 30 may receive updates for various modules and routines, and may also communicate sample image data from the camera device 14 to a remote server for operational improvements, diagnostics, and updates to the imaging system 10. The user interface 124, the external server 126, and / or the surgical control console 128 may communicate with the controller 30 via one or more I / O circuits 130. The I / O circuits 130 may support a variety of communication protocols, including but not limited to Ethernet / IP, TCP / IP, Universal Serial Bus, ProfiBus, ProfiNet, ModBus, and serial communication.

[0040] According to certain aspects of the present disclosure, a surgical imaging system comprises a scope including an optical element aligned to a first tilt angle defining a native viewpoint, an image sensor configured to capture source image data in a field of view transmitted through the optical element, and a controller. The control is configured to control the capture of source image data in the native viewpoint at the first tilt angle, to select a first subset of the source image including a first portion of the field of view that simulates a second tilt angle, and to selectively output modified source image data that simulates the second tilt angle to a display screen.

[0041] Depending on the circumstances, this disclosure may implement one or more of the following features or configurations in various combinations: - Based on the difference between the first and second tilt angles, the first subset is offset from the focal center of the field of view of the source image data. - The controller is further configured to detect one or more features of the source image data that indicate the rotation of the scope relative to the camera body, and a first subset selection is made in response to the rotation of the scope. - The controller is further configured to correct distortion in a first subset of source image data that generates normalized image data, and to warp or modify the normalized image data of the first subset that generates modified source image data that simulates a second tilt angle. - Distortion correction for the first subset is applied to correct for one or more distortions or magnifications of the native viewpoint at the tilt angle. - Warping of normalized image data is distorted, simulating the scaling or distortion at the second tilt angle. - The controller is further configured to generate a simulated mask surrounding the first image, the simulated mask including an orientation cue that identifies the direction of rotation relative to the first image data, which represents the first portion of the field of view simulating a second tilt angle. - The controller is configured to generate modified source image data by creating a virtual mask that is superimposed on the scope's field stop mask around the field of view. - The controller is further configured to selectively generate a second set of image data showing a second subset of the source image data that includes the second portion of the field of view that simulates a third tilt angle. - The first tilt angle is offset by approximately 45° from the scope axis of the scope. - The controller is further configured to selectively output source data, first image data, and second image data to the display screen. - The second tilt angle is 30°, and the third tilt angle is 70°. - One or more features include the field stop mask of the scope shown in the source image data. - The generation of the first image data includes applying an image correction algorithm, and the controller is further configured to restrict the scope of the image correction algorithm to a first subset of the source image data for the first image data. - The image correction algorithm includes at least one of the following: an automatic exposure algorithm and a high dynamic range algorithm, and / or - A method for operating a surgical imaging system.

[0042] According to another aspect of the present disclosure, a surgical imaging system comprises a scope including optical elements aligned to an inclination angle defining a native viewpoint, an image sensor configured to capture source image data in a field of view transmitted through the optical elements, and a controller. The controller is configured to control the capture of source image data in the native viewpoint at the inclination angle and to selectively generate modified image data from source image data having a simulated view relative to the field of view in the native viewpoint, wherein the simulated view is modified to represent a simulated viewpoint different from the native viewpoint, and the modified image data is output to a display screen.

[0043] Depending on the circumstances, this disclosure may implement one or more of the following features or configurations in various combinations: - The native viewpoint is offset by approximately 45° from the scope axis of the scope. - The simulated view has a simulated viewpoint angle ranging from approximately -30° to 45° from the native viewpoint. - The source image data contains a set of pixels, and the simulated viewpoint is generated by remapping the rays aligned with the set of pixels in the source image using an offset central ray aligned to the simulated viewpoint angle. - The source image data contains a set of image data, the modified image data forms a subset of the source image data, the subset of the modified image data is processed to adjust lighting or exposure, and the processing of the subset of the source image data is performed within a simulated view to mask the rest of the source image data from the lighting or exposure adjustment. - The source image data includes a unit sphere of light rays captured within the field of view around the central ray of the lens. - The native viewpoint defines the central ray received at the center of the field of view, and the central ray passes through the focal center of the lens without deflection. - The corrected image data is generated relative to the simulated viewpoint with a corrected central ray offset from the lens's focal center. - The corrected image data is simulated to appear as if the corrected central ray passes through the lens in the field of view without being deflected. - The corrected image data is calculated based on the relative positions of multiple corrective rays distributed around the central corrective ray. - The controller is further configured to detect the actual position of the central ray passing through the lens that defines the focal center of the lens. - The controller is further configured to generate a virtual mask radially around the actual position of the central ray. - The corrected image data is generated based on the actual position of the central ray detected by the controller. - The modified image data is generated by remapping the source image data to fit the simulated viewpoint by adjusting the pixel values ​​in the field of view so that they correspond to light rays entering the lens with an angle offset adjusted based on the simulated viewpoint angle. - Source image data is captured across multiple source image frames, and corrected image data is generated as corrected image frames that form a corrected image stream. - The controller is further configured to selectively output source image data or at least one modified image data in response to input from a device communicating with the controller. - The user interface includes at least one of the following: a camera interface connected to the scope, auxiliary input accessories (e.g., foot pedal, hand switch, etc.), a tablet, a computer terminal, a surgical tool, or a tool control console (e.g., a shaver console, ablation console, pump, etc.). - Modified image data is selectively generated with simulated viewpoint angles of 30° and 70°, while the native viewpoint is aligned to a 45° tilt angle. - Source image data is captured across the entire field of view, and modified image data depicts a subset of the source image data. - The controller is further configured to generate visual cues indicating regions within the source image data that are depicted by the modified image data. - The visual cue includes a graphic that is presented simultaneously with the modified image data and indicates the region in the source image data where the subset is located. - The graphics include symbols that identify the positional relationship between the subset and the source image data, and / or - The graphics include markers, outlines, and / or overlays superimposed on the source image data to indicate a subset of areas within the source image data.

[0044] Naturally, any described process or step within a described process may be combined with other disclosed processes or steps to form a structure within the scope of this device. The exemplary structures and processes disclosed herein are for illustrative purposes only and should not be construed as limitations.

[0045] Furthermore, naturally, the aforementioned structure and method can be modified and altered without deviating from the concept of this device, and unless otherwise explicitly stated in the language of those claims, these concepts are intended to be covered by the following claims.

[0046] The above description shall be considered to relate only to the exemplary embodiments. Modifications of the device would be conceivable to those skilled in the art and to those who manufacture or use the device. Accordingly, the embodiments shown in the drawings and described above are for illustrative purposes only and are not intended to limit the scope of the device, which is understood to be defined by the following claims, which shall be interpreted in accordance with the principles of patent law, including the doctrine of equivalents. [Explanation of Symbols]

[0047] 10 Imaging System 12 field of view 14 Camera equipment 16 Scope 16a Proximal end portion 16b Distal end portion 20 User Interface 28 Optical elements 28a First offset orientation 28b Second offset orientation 30 controllers 32 Video Consoles 36. Modified image data 36a First corrected image data 36b Second corrected image data 40 Display device 42 Visual cues 42a Oriented Cue 42b Relative position queue 50 Source image data 52 subsets 52a First subset 52b Second subset 54 views 54a First View 54b Second View 54c Third View 56a Virtual Mask 56b Field Stop Mask 56d window 60 ways 61 Rotating Arrows 62~74 steps 90 Image Sensors 92 Rays of light 94 Correction Ray 100 Graphics 102 Surroundings 104 Display Window 106 Preview Window 110 Camera Controller 112 Light source 116 processors 118 memory 122 Formatting Circuit 124 User Interface 126 External Servers 128 Surgical Control Console 130 I / O circuit

Claims

1. A surgical imaging system, A scope including optical elements aligned to a first tilt angle that defines the native viewpoint, An image sensor configured to capture source image data within the field of view transmitted through the optical element, It is a controller, To control the capture of the source image data at the native viewpoint at the first tilt angle, To select a first subset of the source image, including the first portion of the field of view that simulates a second tilt angle, and An imaging system including a controller configured to selectively output corrected source image data simulating the second tilt angle to a display screen.

2. The imaging system according to claim 1, wherein the first subset is offset from the focal center of the field of view of the source image data based on the difference between the first tilt angle and the second tilt angle.

3. The aforementioned controller The imaging system according to claim 1, further configured to detect one or more features of the source image data indicating the rotation of the scope relative to the camera body, wherein the selection of the first subset corresponds to the rotation of the scope.

4. The aforementioned controller The first subset of the source image data is distorted to generate normalized image data, and The imaging system according to any one of claims 1 to 3, further configured to warp or modify the normalized image data of the first subset to generate modified source image data that simulates a second tilt angle.

5. The imaging system according to claim 4, wherein the distortion correction of the first subset is corrected for one or more distortions or magnifications of the native viewpoint at the tilt angle.

6. The imaging system according to claim 4 or 5, wherein the warping of the normalized image data is distorted to simulate the magnification or distortion at the second tilt angle.

7. The aforementioned controller The imaging system according to any one of claims 1 to 6, further configured to generate a simulated mask surrounded around the first image, wherein the simulated mask includes an orientation cue that identifies the direction of rotation relative to the first image data showing the first portion of the field of view that simulates a second tilt angle.

8. The imaging system according to any one of claims 1 to 7, wherein the controller is configured to generate the modified source image data by generating a virtual mask superimposed on the field stop mask of the scope around the field of view.

9. The aforementioned controller The imaging system according to any one of claims 1 to 8, further configured to selectively generate a second image data showing a second subset of the source image data, including a second portion of the field of view that simulates a third tilt angle.

10. The imaging system according to any one of claims 1 to 9, wherein the first inclination angle is offset by approximately 45° from the scope axis of the scope.

11. The aforementioned controller The imaging system according to claim 10, further configured to selectively output the source data, the first image data, and the second image data to the display screen.

12. The imaging system according to claim 11, wherein the second tilt angle is 30° and the third tilt angle is 70°.

13. The imaging system according to any one of claims 1 to 12, wherein one or more of the features include a field stop mask of the scope shown in the source image data.

14. The generation of the first image data includes applying an image correction algorithm, and the controller The imaging system according to any one of claims 1 to 13, further configured to limit the range of the image correction algorithm to the first subset of the source image data with respect to the first image data.

15. The imaging system according to claim 14, wherein the image correction algorithm includes at least one of an automatic exposure algorithm and a high dynamic range algorithm.

16. A method for operating the surgical imaging system described in claims 1 to 15.

17. A surgical imaging system, A scope including optical elements aligned at an inclination angle that defines the native viewpoint, An image sensor configured to capture source image data within the field of view transmitted through the optical element, It is a controller, To control the capture of the source image data at the native viewpoint at the aforementioned tilt angle, To selectively generate modified image data from source image data having a simulated view for the field of view at the native viewpoint, wherein the simulated view is modified to represent a simulated viewpoint different from the native viewpoint, and An imaging system comprising a controller configured to output the corrected image data to a display screen.

18. The imaging system according to claim 17, wherein the native viewpoint is offset by approximately 45° from the scope axis of the scope.

19. The imaging system according to claim 18, wherein the simulated view has a simulated viewpoint angle in the range of approximately -30° to 45° from the native viewpoint.

20. The imaging system according to claim 19, wherein the source image data includes a set of pixels, and the simulated viewpoint is generated by remapping rays aligned with the set of pixels in the source image using offset central rays aligned with the simulated viewpoint angle.

21. The source image data includes a set of image data, and the modified image data forms a subset of the source image data. The subset of the modified image data is processed to adjust the lighting or exposure. The imaging system according to any one of claims 17 to 20, wherein the processing of the subset of the source image data is performed within the simulated view to mask the remaining portion of the source image data from the adjustment of illumination or exposure.

22. The imaging system according to any one of claims 17 to 21, wherein the source image data includes a unit sphere of light rays captured within the field of view around the central ray of the lens.

23. The imaging system according to claim 22, wherein the native viewpoint defines the central ray received in the center of the field of view, and the central ray passes through the focal center of the lens without deflection.

24. The imaging system according to claim 23, wherein the corrected image data is generated with respect to the simulated viewpoint in a corrected central ray offset from the focal center of the lens.

25. The imaging system according to claim 24, wherein the corrected image data is simulated to appear as if the corrected central ray passes through the lens in the field of view without being deflected.

26. The imaging system according to claim 24, wherein the corrected image data is calculated based on the relative positions of a plurality of corrected rays distributed around the corrected central ray.

27. The aforementioned controller The imaging system according to claim 23, further configured to detect the actual position of a central ray passing through the lens that defines the focal center of the lens.

28. The aforementioned controller The imaging system according to claim 27, further configured to generate a virtual mask radially with respect to the actual position of the central ray.

29. The imaging system according to claim 27, wherein the corrected image data is generated based on the actual position of the central ray detected by the controller.

30. The imaging system according to any one of claims 17 to 29, wherein the modified image data is generated by remapping the source image data to fit the simulated viewpoint by adjusting the pixel values ​​in the field of view so that they correspond to light rays incident on the lens with an angle offset adjusted based on the simulated viewpoint angle.

31. The imaging system according to any one of claims 17 to 30, wherein the source image data is captured across a plurality of source image frames, and the modified image data is generated as modified image frames forming a modified image stream.

32. The aforementioned controller The imaging system according to any one of claims 17 to 31, further configured to selectively output the source image data or the at least one modified image data in response to input from a device communicating with the controller.

33. The imaging system according to claim 32, wherein the user interface comprises at least one of the following: a camera interface connected to the scope, auxiliary input accessories (e.g., a foot pedal, a hand switch, etc.), a tablet, a computer terminal, a surgical tool, or a tool control console (e.g., a shaver console, an ablation console, a pump, etc.).

34. The imaging system according to claim 33, wherein the modified image data is selectively generated to have simulated viewpoint angles of 30° and 70°, and the native viewpoint is aligned to the inclination angle of 45°.

35. The imaging system according to any one of claims 17 to 34, wherein the source image data is captured over the entire field of view, and the modified image data depicts a subset of the source image data.

36. The aforementioned controller The imaging system according to claim 35, further configured to generate visual cues indicating regions in the source image data that are depicted by the modified image data.

37. The imaging system according to claim 36, wherein the visual cue is presented simultaneously with the modified image data and includes a graphic indicating the region in the source image data where the subset is located.

38. The imaging system according to claim 37, wherein the graphic includes symbols that identify the positional relationship between the subset and the source image data.

39. The imaging system according to claim 38, wherein the graphic includes markers, outlines, and / or overlays superimposed on the source image data, indicating a subset of the region within the source image data.

Images