Medical devices and related systems and methods for automatic image brightness control

Abstract

The medical device system (100) may include a control unit (175) with one or more processors that implement an algorithm for enhancing images acquired by the medical device (101). The one or more processing boards perform the following steps: receiving a first image from a first view element, determining a current irradiance value of the first image, determining a first difference between the current irradiance value and the high irradiance target value if the current irradiance value is greater than the high irradiance target value, and determining a second difference between the current irradiance value and the low irradiance target value if the current irradiance value is less than the low irradiance target value, generating a new irradiance value using at least one of the first difference and the second difference, and converting the new irradiance value to a first voltage value for application to one or more illuminators of the medical device.

Description

[Technical Field]

[0001] Various aspects of the present disclosure generally relate to systems, devices, and methods for automatic image brightness control. Specifically, embodiments of the present disclosure relate, among other aspects, to imaging catheters, such as endoscopes or other medical devices, configured to automatically control illuminators and related systems and methods. This application claims the benefit of priority to U.S. Provisional Patent Application No. 63 / 377,433, filed September 28, 2022, which is incorporated herein by reference in its entirety. [Background technology]

[0002] During endoscopic procedures, medical professionals operating endoscopes often utilize one or more illuminators to illuminate the field of view of a camera at the distal end of the endoscope. Most imaging catheters, such as endoscopes, utilize a constant illumination output, with each illuminator in the imaging catheter providing a constant illumination output, for example, from one or more light-emitting diodes (LEDs). Such imaging catheters with constant illumination output control image brightness by varying the exposure and / or gain of the image sensor. This results in a noticeable step response in image brightness and a delayed response to changing scenes. Step response refers to the change in the system's output when the system's input is a unit step function. The step response is due to the limited number of exposure steps available to the image sensor, and the response delay is at least partially due to the exposure value being written in a single step at the end of each image frame. For example, if the exposure needs to be adjusted by 10 steps to increase or decrease the exposure of the image sensor, a noticeable delay (approximately 300 milliseconds for a 30 frames per second (fps) image sensor) can typically occur due to the minimum 10 image frames required to adjust the image brightness.

[0003] When users experience image delay in imaging catheter systems, procedures can be extended and procedural tasks can become more difficult or delayed. Alternative methods for adjusting illumination in imaging catheters and other medical devices are needed to reduce image delay and address other issues related to medical device illumination and imaging systems. Summary of the Invention

[0004] Aspects of the present disclosure relate to, among other aspects, systems, devices, and methods for assisting in reducing image latency in medical device imaging systems. The systems, devices, and methods of the present disclosure may reduce the time required to focus and / or properly illuminate the field of view of a camera or other imaging device of an endoscope or other medical device. Endoscopes and other medical devices incorporating the systems and methods of the present disclosure may help address image latency, shorten the time required for procedures, and address other issues. Each of the aspects disclosed herein may include one or more of the features described with respect to any of the other disclosed aspects.

[0005] According to one aspect, a medical device system may include a control unit configured to be operably coupled to a medical device. The control unit may include one or more processors that implement an algorithm for enhancing an image acquired by a first view element of the medical device. The one or more processing boards perform the following steps: receiving a first image from the first view element; determining a current irradiance value of the first image; if the current irradiance value is greater than a high irradiance target value, determining a first difference between the current irradiance value and the high irradiance target value; if the current irradiance value is less than the low irradiance target value, determining a second difference between the current irradiance value and the low irradiance target value; generating a new irradiance value using at least one of the first difference and the second difference; and converting the new irradiance value to a first voltage value for application to one or more illuminators of the medical device.

[0006] In other aspects, the medical device system may include one or more of the following features: the high illuminance target value and the low illuminance target value may together define a tolerance band around the target illuminance value stored by the control unit; the one or more processing boards may further perform the steps of determining whether the current illuminance value is below a first illuminance threshold and, if the current illuminance value is below the first illuminance threshold, generating a new illuminance value using a scaling factor; the one or more processing boards may further perform the steps of determining whether the new illuminance value is greater than a maximum illuminance value and, if the new illuminance value is greater than the maximum illuminance value, converting the maximum illuminance value to a second voltage value for application to one or more illuminators of the medical device and increasing a gain of the one or more imaging devices; the one or more processing boards may further perform the steps of determining whether the new illuminance value is lower than the current illuminance value and, if the new illuminance value is lower than the current illuminance value, decreasing a gain of the one or more imaging devices; the medical device may be an endoscope. Determining a current illumination value of the first image may include accumulating and summing pixel values ​​of the first image. The one or more processing boards may further perform the steps of determining a current frame rate of the first view element, generating a new frame rate using at least one of the first difference and the second difference, and applying the new frame rate to the first view element.

[0007] In another aspect, the medical device system may include one or more of the following features. The medical device may include the first view element and at least one illuminator. The one or more processors may further perform the steps of determining a current exposure time of the first view element, generating a new exposure time using at least one of the first difference and the second difference, and applying the new exposure time to the first view element. The one or more processors may further perform the steps of determining whether the current illuminance value is below or above the new illuminance value before converting the new illuminance value to the first voltage value, converting the new illuminance value to the first voltage value if the current illuminance value is below the new illuminance value, and increasing a frame rate of the first view element if the current illuminance value is above the new illuminance value. Generating a new illuminance value using at least one of the first difference and the second difference may include determining a first error coefficient of the first image and a second error coefficient of a second image received by the control unit prior to the first image. wherein the first error coefficient is a first difference when the current irradiance value is greater than the high irradiance target value, and the second error coefficient is a second difference when the current irradiance value is less than the low irradiance target value. The step of generating the new irradiance value may further include determining proportional, integral, and derivative tuning constants each associated with the medical device.

[0008] In other aspects, the medical device system can include one or more of the following features: the current irradiance value can be a first current irradiance value, the high irradiance target value can be a first high irradiance target value, the low irradiance target value can be a first low irradiance target value, and the new irradiance value can be a first new irradiance value. The one or more processing boards may further perform the following steps: receiving a second image from a second view element; determining a second current irradiance value of the second image; if the second current irradiance value is greater than a second high irradiance target value, determining a third difference between the second current irradiance value and the second high irradiance target value; if the second current irradiance value is less than the second low irradiance target value, determining a fourth difference between the second current irradiance value and the second low irradiance target value; generating a second new irradiance value using at least one of the third difference and the fourth difference; and converting the second new irradiance value to a second voltage value for application to one or more illuminators of the medical device. The one or more processing boards may further perform the following steps: displaying the second image received from the first view element via at least one electronic display; the second image is illuminated by the one or more illuminators that received the first voltage value.

[0009] In another aspect, a method for enhancing an image acquired by a medical device system is disclosed. The medical device system may include (a) one or more processors and (b) a medical device operably coupled to the one or more processors. The medical device is configured for insertion into a patient's body and includes a first view element and one or more illuminators. The method includes receiving a first image from the first view element, determining a current illuminance value of the first image, and, if the current illuminance value is greater than a high illuminance target value, determining a first difference between the current illuminance value and the high illuminance target value, and, if the current illuminance value is less than a low illuminance target value, determining a second difference between the current illuminance value and the low illuminance target value, generating a new illuminance value using at least one of the first difference and the second difference, and converting the new illuminance value to a first voltage value for application to one or more illuminators of the medical device.

[0010] In other aspects, the method may include one or more of the following features. The method may further include determining whether the new irradiance value is greater than a maximum irradiance value, and if the new irradiance value is greater than the maximum irradiance value, converting the maximum irradiance value to a second voltage value for application to one or more illuminators of the medical device, and increasing a gain of the one or more imaging devices. The method may further include determining a current exposure time of the first view element, generating a new exposure time using at least one of the first difference and the second difference, and applying the new exposure time to the first view element. The method may further include determining a current frame rate of the first view element, generating a new frame rate using at least one of the first difference and the second difference, and applying the new frame rate to the first view element.

[0011] In another aspect, a non-transitory computer-readable medium may include program instructions for causing a processor to execute a method for enhancing an image acquired by a first view element of a medical device system. The medical device system may include a processor configured to perform processing and a medical device operably coupled to the processor. The medical device is configured to be insertable into a patient's body and includes a first view element and one or more illuminators. The method may include receiving a first image from the first view element, determining a current illuminance value of the first image, and, if the current illuminance value is greater than the high illuminance target value, determining a first difference between the current illuminance value and the high illuminance target value, if the current illuminance value is less than the low illuminance target value, determining a second difference between the current illuminance value and the low illuminance target value, generating a new illuminance value using at least one of the first difference and the second difference, and converting the new illuminance value to a first voltage value for application to one or more illuminators of the medical device.

[0012] Both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention as claimed. The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the disclosure and, together with the description, serve to explain the principles of the disclosure. [Brief explanation of the drawings]

[0013] [Figure 1A] FIG. 1A is a perspective view of an exemplary endoscopic system according to an embodiment of the present disclosure. [Figure 1B] FIG. 1B is a perspective view of an exemplary endoscopic system according to an embodiment of the present disclosure. [Figure 2] FIG. 2 illustrates an exemplary method for automatically adjusting lighting in a medical device according to an embodiment of the present disclosure. [Figure 3] FIG. 3 illustrates an optional additional portion of the method of FIG. 2 according to an embodiment of the present disclosure. [Figure 4] FIG. 4 illustrates another exemplary method for automatically adjusting lighting in a medical device, according to an embodiment of the present disclosure. [Figure 5] FIG. 5 is an exemplary graph illustrating illuminance values ​​applied to one or more illuminators and exposure time values ​​applied to one or more imagers in a PID control system according to an embodiment of the present disclosure. [Figure 6] FIG. 6 illustrates another exemplary method for automatically adjusting lighting in a medical device, according to an embodiment of the present disclosure. [Figure 7] FIG. 7 is a simplified functional block diagram of a computer and / or server that may be configured as an apparatus or system for performing any of the methods described herein, according to aspects of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0014] Several aspects of the present disclosure are described in detail below, and examples of these aspects are illustrated in the accompanying drawings. Wherever possible, the same or similar reference numbers are used throughout the drawings to refer to the same or similar parts. The term "distal" refers to the part of the device that is furthest from the user when the device is introduced into a patient. In contrast, the term "proximal" refers to the part of the device that is closest to the user when the device is positioned within a patient. In FIGS. 1A and 1B, arrows marked "P" and "D" are used to indicate the proximal and distal directions in the figures. As used herein, the terms "comprises," "comprising," or other variations thereof are intended to cover non-exclusive inclusions; thus, a process, method, article, or device comprising a list of elements may include not only those elements, but also other elements not expressly listed or inherent in such process, method, article, or device. The term "exemplary" is used to mean "example" rather than "ideal." Additionally, terms such as "about," "substantially," and "approximately" are used to indicate a possible variation of + / - 10% of the stated value.

[0015] Embodiments of the present disclosure seek to improve the illumination and imaging of medical devices, such as endoscopes, during medical procedures. Among other non-limiting exemplary advantages, aspects of the present disclosure can reduce delays experienced in imaging systems and / or make it easier to view the field of view of one or more cameras on a medical device.

[0016] 1A and 1B show perspective views of an exemplary endoscopic system 100. The endoscopic system 100 may include an endoscope 101. While the term "endoscope" may be used herein, other devices may also be used in connection with the systems and methods of the present disclosure. The other devices include, but are not limited to, a duodenoscope, a colonoscope, a ureteroscope, a bronchoscope, a laparoscope, a sheath, a catheter, or any other suitable delivery device or other type of medical device. The systems and methods described below may be incorporated into any of these or other medical devices.

[0017] The endoscope 101 may include a handle assembly 106 and a flexible tubular shaft 108. The handle assembly 106 may include one or more of a biopsy port 102, a biopsy cap 103, an image capture button 104, an elevator actuator 107, a locking lever 109, a locking knob 110, a first control knob 112, a second control knob 114, a suction button 116, an air / water button 118, a handle body 120, and an umbilicus 105. Any of the actuators, elevators, knobs, buttons, levers, ports, or caps of the endoscope system 100, as listed above, may serve any purpose and are not limited by any particular use that may be suggested by the respective names of each component used herein. The umbilicus 105 may extend from the handle body 120 to auxiliary devices, such as a control unit 175, a water / fluid supply, and / or a vacuum source. The umbilicus 105 may transmit signals between the endoscope 101 and the control unit 175 to control the illumination and imaging components of the endoscope 101 and / or receive image data from the endoscope 101. The umbilicus 105 may also provide fluid for irrigation and / or aspiration from a water / fluid supply to the distal tip 119 of the shaft 108. Buttons 116 and 118 may control valves for aspiration and fluid supply (e.g., air and water), respectively. The shaft 108 may terminate at the distal tip 119. The shaft 108 may include an articulation portion 122 for deflecting the distal tip 119 upward, downward, leftward, and / or rightward. Knobs 112 and 114 may be used to control such deflection. A locking lever 109 and a locking knob 110 may lock the knobs 112 and 114 in a desired position, respectively.

[0018] The distal tip 119 may include one or more imaging devices 125, 126 and light sources 127-130 (e.g., one or more LEDs, optical fibers, and / or other illuminators). For example, the imaging devices (or view elements) 125, 126 may include one or more cameras, one or more image sensors, an endoscopic view element, an optical assembly including one or more image sensors and one or more lenses, and any other imaging device known in the art. As shown in FIG. 1A, the distal tip 119 may include a forward-facing imaging device 125 and a side-facing imaging device 126. However, in other embodiments, the distal tip 119 may include only one imaging device 125 (126), which may be forward-facing or side-facing. In other examples, the distal tip 119 may include three or more imaging devices 125, 126 oriented in different directions. Also, in some examples, the fields of view of each imaging device 125, 126 may overlap. The distal tip 119 may include one or more illuminators 127-130. One or more of the illuminators 127, 128 may be forward-facing illuminators (or may face distally). One or more of the illuminators 129, 130 may be side-facing illuminators. The side-facing imager 126 and the side-facing illuminators 129, 130 may face radially outward, perpendicular, approximately perpendicular, or laterally relative to the longitudinal axis of the shaft 108 and distal tip 119. The forward-facing or forward-facing imager 125 and the front-facing illuminators 127, 128 may face approximately along the longitudinal axis of the distal tip 119 and shaft 108.

[0019] The endoscopic system 100 of the present disclosure may also include a control unit 175, as shown in FIGS. 1A and 1B. The control unit 175 may be capable of interfacing with the endoscope 101 to provide power and commands to the imaging devices 125, 126 and the illuminators 127-130. The control unit 175 may also control other features of the endoscope 101, such as the application of suction, the deployment or delivery of fluids, and / or the movement of the distal tip 119. The control unit 175 may be powered by an external power source, such as an electrical outlet. The control unit 175 may also include buttons, knobs, a touchscreen, or other user interface for controlling the imaging devices 125, 126, the illuminators 127-130, and other features of the endoscope 101. The control unit 175 may be housed within the handle body 120 itself or in a separate device.

[0020] Control unit 175 may be configured to allow a user to set or control one or more lighting and imaging parameters. For example, control unit 175 may allow a user to set or control the illumination level of each illuminator 127-130, the gain level of each imager 125, 126, the exposure time of each imager 125, 126, the frame rate of each imager 125, 126, maximum or target values ​​for any illumination and imaging parameters, and / or any other parameters associated with imagers 125, 126 and illuminators 127-130. In some examples, control unit 175 may be configured to execute one or more algorithms using one or more illumination and imaging parameters, for example, to automatically adjust the illumination level of one or more of illuminators 127-130 and / or to automatically adjust one or more parameters of imagers 125, 126. For example, the control unit 175 may set or select the illumination level of one or more of the illuminators 127-130 based on data received from one or more of the imagers 125,126.

[0021] The control unit 175 may include electronic circuitry configured to receive, process, and / or transmit data and signals between the endoscope 101 and one or more other devices. For example, the control unit 175 may be in electronic communication with a display configured to display images based on image data and / or signals generated by the imaging devices 125, 126 of the endoscope 101 and processed by the control unit 175. The control unit 175 may be in electronic communication with the display in any suitable manner, either wired or wireless. The display may be manufactured in any suitable manner, may include touchscreen input, and / or may be connected to various input and output devices, such as, for example, a mouse, an electronic stylus, a printer, a server, and / or other electronic portable devices. The control unit 175 may include software and / or hardware to support operations as described above. For example, the control unit 175 may include one or more algorithms, models, etc. for performing any of the methods and / or systems described in this disclosure. The control unit 175 may be configured to automatically adjust the illumination values ​​applied to one or more of the illuminators 127-130, as well as automatically adjust the gain and frame rate applied to one or more of the imagers 125,126.

[0022] When operating endoscopic system 100, a user may use their left hand to hold handle assembly 106, while using their right hand to hold accessories and / or operate one or more of the handle assembly 106's actuators, such as first and second control knobs 112, 114 and locking lever 109 and locking knob 110. While grasping handle body 120, the user may use the fingers of their left hand to operate (by pressing) image capture button 104, suction button 116, and / or air / water button 118. During a procedure, the user may view one or more fields of view of imaging devices 125, 126 on an electronic display operably connected to control unit 175. One or more illuminators 127-130 may provide illumination to one or more fields of view of imaging devices 125, 126.

[0023] 2-4 are process flow diagrams illustrating various control loop algorithms that may be implemented by endoscopy system 100 or any other medical device system having one or more imaging devices and one or more illuminators. While the algorithms described herein are described in the context of an endoscopy system, they are not limited thereto and may be implemented using any medical device system known in the art that includes imaging and illumination components. Generally, the algorithms described herein vary illumination levels based on changing data received from one or more imaging devices, e.g., a changing field of view of the imaging devices.

[0024] FIG. 2 illustrates an illumination control method 200 that may be automatically performed by the control unit 175 of the endoscope system 100. The method 200 utilizes proportional, integral, and derivative (PID) coefficients to control the speed and accuracy of illumination in the endoscope system 100. In an initial step 201, a target image brightness is stored in the control unit 175 and an initial illumination value is set by the control unit 175. In some examples, a user may select the target image brightness. In other examples, the control unit 175 may automatically determine the target image brightness. For example, the control unit 175 may use previous treatment data to determine the target image brightness for use in the method 200. In some examples, the target image brightness may be a low (T low ) brightness target to high (T high ) a range of brightness values ​​up to a brightness target value. For example, the range of brightness values ​​used for the target image brightness may be a tolerance band set around a particular target brightness value. Also, during step 201, the user or control unit 175 may set an initial illumination value to apply to one or more illuminators 127-130.

[0025] In a next step 202, the control unit 175 may determine an actual illumination value by accumulating and summing values ​​of pixels in a current image frame received from one or more imagers 125, 126. In some examples, only a single image frame from a single imager 125, 126 may be used to determine the initial illumination value, while in other examples, multiple image frames from one or more imagers 125, 126 may be used.

[0026] Step 203 involves determining the error coefficients and PID coefficients for the current image frame during the time between frames received from one or more imagers 125, 126 (e.g., during vertical blanking). To determine the error coefficients for the current image frame, the control unit may perform the following calculations:

[0027] B current >T highIf , error = T high -B current B current <T low If , error = T low -B current B current is the calculated illumination value of the current image frame. low is the low light target value, and T high is the high illuminance target value. In some embodiments, control unit 175 may determine the target luminance range (or range of illuminance values) to be a tolerance band around the target illuminance value set by the user. To determine the PID coefficients, control unit 175 may perform the following calculations:

[0028] P=K P *(error) I=K I *(error + old error) D=K D *(error - old error) Then, old error = (error) K P is the proportional tuning constant. K I is the integral tuning constant, and K D is a differential tuning constant. Error is the error coefficient calculated for the current image frame, and OldError is the error coefficient calculated for the previous image frame. The tuning parameters (e.g., tuning constant K P ,K I ,K D ) is determined experimentally and depends on the type of lighting and driving circuitry used. In some examples, the tuning constant K P ,K I ,K D can be experimentally determined to achieve a target response time. The speed of the PID loop is determined by the tuning constant K P ,K I ,K D This is directly related to the tuning constant K P adjusts the output in proportion to the current error. Iis what controls the static error. The tuning constant K D is based on the rate of change of the error and has a damping effect on the output.

[0029] Once the error coefficients and PID coefficients for the current image frame have been determined using the calculations described above in step 204, the control unit 175 may determine new intensity values ​​to apply to one or more illuminators 127-130, for example, using the old intensity values, PID coefficients, and hardware scaling coefficients. The new intensity values ​​may be determined by the following calculation:

[0030] Provisional value=(P+I+D) / F Scaling B New =B current + Provisional value Then, B current =B New F Scaling is a scaling factor based on the hardware used to drive the lighting, such as the type of illuminator (LED, fiber optic, etc.) and the circuitry connected to the illuminator. The scaling factor F Scaling may be based on the driver circuit and may depend on the particular hardware implementation and the desired range of acceptable illumination. New i.e., an intermediate value used by the control unit to determine a new illuminance value to apply to one or more illuminators 127-130. The calculated illuminance value is a digital value that is converted to an analog voltage, current, or any other digital value that controls the illuminance applied to one or more illuminators 127-130, thereby enabling the control unit 175 to control the illumination of the endoscope 101. In some examples, the control unit 175 may determine when step 204 is complete and the illuminance value applied to one or more illuminators 127-130 is B. New Once applied, steps 202-204 are repeated for the next image frame from one or more of the imaging devices 125, 126. Steps 201-204 are thus an example of a control loop algorithm for automatically adjusting the illumination of the endoscope system 100.

[0031] In some instances, the control loop algorithm of Figure 2 generates a new illuminance value, B New may include an additional step 205 of determining whether the value of illuminance is near the upper or lower end of a range of illuminance values ​​that are accepted by a particular illuminator, or that the illuminator hardware will accept. In some embodiments, the control loop algorithm may not include step 205 and may proceed from step 204 to step 202 (shown in dotted lines) to continue cycling through the loop algorithm.

[0032] Figure 3 shows B New If is near the lower or upper end of the range of illuminance values ​​accepted by one or more illuminators, B New is at the maximum value of the range of illuminance values ​​accepted by one or more illuminators, and B New 3 shows different steps 301-303 that the control unit 175 may perform when illuminance is at the minimum of the range of illuminance values ​​accepted by one or more illuminators.

[0033] In step 301, B New If F is near the lower or upper end of the range of illuminance values ​​accepted by one or more illuminators, a different scaling factor (F Scaling ) is used. This different scaling factor (F Scaling ) is the scaling factor (F) previously used to force the change in illuminance value to be smaller than the previous change in illuminance value. Scaling ) is considered to be smaller than B New is near the low or high end of the range of illuminance values, the scaling factor (F Scaling ) may be able to reduce or eliminate illumination oscillations caused by nonlinearities in one or more illuminator hardware.

[0034] In step 302, B Newis at the maximum value of the range of illuminance values ​​accepted by the one or more illuminators (e.g., the illuminance value is at the maximum value). Because the illuminance value cannot be increased beyond the maximum value of the range of illuminance values ​​accepted by the one or more illuminators, the control unit 175 may adjust the digital gain of one or more image sensors associated with one or more image capture devices 125, 126. For example, B New Under normal conditions, when the illuminance or brightness level of the current image frame is below a minimum illuminance target value, the gain of one or more image sensors of one or more image capture devices 125, 126 is increased. The gain is then increased (e.g., in steps) until the minimum illuminance target value in the current image frame is reached or the maximum gain of one or more image sensors is reached. In some examples, if a saturation value (e.g., a value representing color intensity) in the current image frame falls from the target saturation value when the gain of one or more image sensors is being increased in steps, the gain may be increased to the maximum gain while the illuminance value in the current image frame is reduced.

[0035] In step 303, B New is at the minimum of the range of illuminance values ​​accepted by the one or more illuminators (e.g., the illuminance value is at a minimum). Because the illuminance value cannot be reduced beyond the minimum of the range of illuminance values ​​accepted by the one or more illuminators, the control unit 175 may adjust the digital gain of one or more image sensors associated with one or more image capture devices 125, 126. For example, B NewUnder normal conditions, when the illuminance or brightness level of the current image frame is at the minimum of a range of illumination values ​​and the illumination or brightness level of the current image frame exceeds a maximum illumination target value, the gain of one or more image sensors of one or more image capture devices 125, 126 is reduced. The gain is then reduced (e.g., in steps) until either the maximum illumination target value for the current image frame is reached or the minimum gain of one or more image sensors is reached. In some examples, if the saturation value in the current image frame falls from the target saturation value as the gain of one or more image sensors is being reduced in steps, the gain may be reduced to the minimum gain while the illumination value in the current image frame is increased. After completing any of steps 301, 302, and 303, the control unit may continue another cycle of the control loop algorithm of FIG. 2. For example, the control unit initiates step 202 of determining the actual image brightness for the next image frame received from one or more image capture devices 125, 126. When the control loop algorithm executed by the control unit 175 incorporates steps 205, 301-303, the control unit 175 can automatically switch between (i) adjusting the illumination values ​​applied to one or more illuminators 127-130 and (ii) adjusting the gain of one or more image sensors of one or more imaging devices 125, 126.

[0036] In some instances, B current To speed up the response of the control unit 175 to extreme differences between the image sensor and the target illumination value, an extreme image brightness guard may be used to change the gain of one or more image sensors by a value greater than 1. The extreme image brightness guard is current The minimum difference between the target illuminance value and the image brightness limit is met (or B current and the target illumination level is met), the control unit 175 adjusts the gain to B current and the target illuminance value. For example, if the gain is at the low end of the range of gain values, and B current If the illuminance suddenly drops significantly below the target illuminance value, the control unit 175 adjusts the gain to B currentand the target illuminance value. This scaling factor can be 2, 5, 10, or any value appropriate to reach the target illuminance value more quickly. By adjusting the rate at which the gain is increased or decreased, the delay time to reach the target illuminance value is reduced. If the extreme image brightness guard is not reached, the gain is increased or decreased by 1 as needed to reach the target illuminance value.

[0037] FIG. 4 illustrates another illumination control method 400 that may be automatically performed by the control unit 175 (or another control unit) of the endoscope system 100. The method 400 of FIG. 4 utilizes proportional, integral, and derivative (PID) coefficients to control the speed and accuracy of illumination in the endoscope system 100. In an initial step 401, a target image brightness (e.g., a target irradiance value) is stored in the control unit 175, and initial irradiance values ​​(e.g., preset irradiance values) for one or more illuminators 127-130 and initial exposure times for one or more imagers 125, 126 are determined by the control unit 175. The initial irradiance value applied by the control unit 175 in the method 400 may be a specific irradiance value that provides sufficient light so that the image brightness is appropriate for the average imaged volume at the maximum exposure available in the imagers 125, 126 for the desired frame rate. This specific irradiance value may be set by a user or applied automatically by the control unit 175. For example, the default illuminance value may be one that achieves an average image brightness of approximately 40-50%. The user may adjust the illuminance value in the range of 25-70% image brightness based on the user's preferences and clinical needs. In some examples, the user may select a target image brightness (e.g., a target illuminance value), while in other examples, the control unit 175 may automatically determine the target image brightness. For example, the control unit 175 may use previous treatment data to determine the target image brightness for use in the method of FIG. 2. In some examples, the target image brightness may be a low (T low ) brightness target to high (T high ) can be a range of brightness values ​​up to a brightness target value.

[0038] 2, the control unit 175 may accumulate and sum the values ​​of pixels in a current image frame received from one or more imagers 125, 126. In some examples, only a single image frame from a single imager 125, 126 may be used to determine the initial illumination value, while in other examples, multiple image frames from one or more imagers 125, 126 may be used.

[0039] Step 403 involves determining error coefficients and PID coefficients for the current image frame during the time between frames received from one or more imagers 125, 126 (e.g., during vertical blanking). The error coefficients and PID coefficients are determined in the same manner as described above with respect to FIG. 2. Scaling may vary when using a PID loop with exposure time. For example, to have finer control over illumination, a user may want to scale the output to have smaller steps. Once the error coefficients and PID coefficients for the current image frame are determined, control unit 175 may determine a new illumination value and adjust the exposure time based on the new illumination value in step 404. For example, if the image is too bright (e.g., B New >B current ) the exposure time is shortened, and if the image is too dark (e.g. B current >B New ) the exposure time is increased. The exposure time may be increased by a single unit or multiple units after each execution of the PID algorithm loop for the current frame. If the exposure time reaches its maximum value and the image remains too dark (e.g., B current >B New ) and the same illuminator control algorithm as described above with respect to FIG. 2 may be used to increase the illuminance values ​​supplied to one or more of the illuminators 127-130.

[0040] Sensor manufacturers typically have controllable registers for exposure, where the value written to one of the registers is a fraction of a line. For example, a sensor with 480 lines running at 30 frames per second has a line time of approximately 65 microseconds. If the exposure number in the register corresponds to 1 / 16 of a line, writing the value 16 to the register provides an exposure time of approximately 65 microseconds. In this example, the maximum allowable exposure for a particular image sensor is approximately 30 microseconds. This is because exposures longer than 30 microseconds reduce the frame rate. Knowing the exposure in fractions of a line allows a PID-based algorithm, such as method 400 of FIG. 4, to be used to control the exposure of the image sensor. Using method 400 of FIG. 4 to adjust the exposure time of one or more imagers 125, 126, as well as the irradiance values ​​applied to one or more illuminators 127-130, can increase the spectral stability of the light used to illuminate the target anatomy. Method 400 can also help minimize color shifts resulting from different lighting conditions.

[0041] FIG. 5 illustrates an exemplary graph of illuminance values ​​applied to one or more illuminators 127-130 in a PID control system, such as the system described with respect to FIG. 2, and exposure time values ​​applied to one or more imagers 125, 126 in a PID control system, such as the system described with respect to FIG. 4. By using the slopes of the illuminance and exposure time values, a complex method of alternating between (i) adjusting the illuminance values ​​of one or more illuminators 127-130 and (ii) adjusting the exposure time of one or more imagers 125, 126 can be utilized to adjust the brightness of an image. For example, if the demand for a change in brightness is large (e.g., above a certain threshold), the illuminance values ​​applied to the illuminators 127-130 can be adjusted to modify the image brightness. Adjusting the illuminance values ​​applied to the illuminators 127-130 reflects an increase or decrease in brightness in a shorter time than adjusting the exposure time. If the demand for a change in brightness is small (e.g., below a certain threshold), the brightness of the image can be changed by adjusting the exposure time of one or more imagers 125, 126. Increasing or decreasing brightness by increasing or decreasing exposure time in a single step has less effect on image brightness than increasing or decreasing illuminance values ​​in a single step. By switching between adjusting the illuminance values ​​applied to illuminators 127-130 and adjusting the exposure times applied to image capture devices 125, 126, image brightness can be controlled with greater precision, especially near the lower end of illumination control.

[0042] FIG. 6 illustrates another illumination control method 600 that may be automatically performed by the control unit 175 (or another control unit) of the endoscopic system 100. The method 600 of FIG. 6 utilizes proportional, integral, and derivative (PID) coefficients to control the speed and accuracy of illumination in the endoscopic system 100. The method 600 of FIG. 6 also controls the frame rate of the imagers 125, 126 in combination with any of the other methods described above with reference to FIGS. 2-5 as an additional way of controlling the brightness of a received image. As shown in FIG. 6, in an initial step 601, a target image brightness (e.g., a target illuminance value) is set (e.g., stored in the control unit 175), and initial illuminance values ​​for one or more illuminators 127-130 are determined by the control unit 175. Also during step 601, an initial frame rate is set (e.g., stored in the control unit 175) for one or more imagers 125, 126.

[0043] In the next step 602, in the same manner as described above in connection with step 202 of FIG. 2, the control unit 175 may determine the actual image brightness by accumulating and summing the values ​​of pixels in the current image frame received from one or more imaging devices 125, 126.

[0044] During the time between frames received from one or more image capture devices 125, 126 (e.g., during vertical blanking), in steps 603, 605, control unit 175 determines whether the actual illumination value of the current image frame is below (step 603) or above (step 605) the target illumination value.

[0045] If the actual illumination value of the current image frame is below the target illumination value (step 603), control unit 175 (i) first adjusts the illumination value applied to one or more illuminators 127-130 until a maximum illumination value is reached for one or more illuminators 127-130 or the target illumination value is reached, and then (ii) adjusts the gain value applied to one or more imagers 125, 126 until a maximum gain value is reached or the target illumination value is reached. As shown in step 604, if both the illumination value applied to one or more illuminators and the gain value applied to one or more imagers 125, 126 are at their respective maximum values, control unit 175 allows an increase in exposure time by decreasing the frame rate of one or more imagers 125, 126 until the target illumination value or minimum frame rate is reached for the received image.

[0046] If the actual illumination value of the current image frame exceeds the target illumination value (step 605), control unit 175 increases the frame rate applied to one or more image capture devices 125, 126 until the target illumination value is reached or the maximum frame rate is reached for the received image. Then, if the actual illumination value is still above the target illumination value and the maximum frame rate is reached (step 606), control unit 175 (i) adjusts the illumination value applied to one or more illuminators 127-130 until the minimum illumination value is reached or the target illumination value is reached for one or more illuminators 127-130, and then (ii) adjusts the gain value applied to one or more image capture devices 125, 126 until the minimum gain value is reached or the target illumination value is reached. The combination of automatic adjustment of the illumination values ​​applied to one or more illuminators 127-130, the gain values ​​applied to one or more imagers 125, 126, and the frame rate applied to one or more imagers 125, 126 allows for more efficient adjustment of image brightness, e.g., to minimize color shifts due to different lighting conditions, and higher frame rates can reduce video latency and provide a "smoother" video.

[0047] In various embodiments, any of the systems and methods described herein may include a control unit 175 and a medical device (e.g., endoscope 101). The control unit 175 may include a processor in the form of one or more processors or central processing units (“CPUs”) for executing program instructions. In some examples, the one or more processors may be one or more processing boards. The control unit 175 may include an internal communication bus and a storage unit (e.g., ROM, HDD, SDD, etc.) capable of storing data on a computer-readable medium, although the control unit 175 may also receive programming and data via network communications. The control unit 175 may also have memory (e.g., RAM) that stores instructions for executing the techniques presented herein, although the instructions may be stored temporarily or permanently within other modules of the control unit 175 (e.g., the processor and / or computer-readable medium) or may be stored remotely, such as on a cloud server electronically connected to the control unit 175. Various system functions of the control unit 175 may be implemented in a distributed manner across multiple similar platforms to distribute the processing load. Alternatively, the system described herein may be implemented by appropriate programming of one computer hardware platform in the control unit 175 .

[0048] Figure 7 presents a functional block diagram of a general-purpose computer hardware platform. Figure 7 illustrates a network or host computer platform that may typically be used to implement a server 700 or browser, or any other device that performs features of the methods and systems described herein. Those skilled in the art are familiar with the structure, programming, and general operation of such computer equipment, and therefore, the drawings are considered self-explanatory.

[0049] A platform for a server 700 or the like may include, for example, a data communication interface 760 for packet data communication. The platform may also include a central processing unit (CPU) 720 in the form of one or more processors for executing program instructions. While the server 700 often receives programming and data via network communication 770, the platform typically includes an internal communication bus 710, program storage, and data storage for various data files processed and / or communicated by the platform, such as ROM 730 and RAM 740. The hardware elements, operating systems, and programming languages ​​of such equipment are conventional in nature and are presumed to be sufficiently familiar to those skilled in the art. The server 700 may also include input and output ports 750 for connecting input and output devices such as a keyboard, mouse, touchscreen, monitor, display, etc. Of course, various server functions may be implemented in a distributed manner across multiple similar platforms to distribute the processing load. Alternatively, the server may be implemented by appropriate programming of a single computer hardware platform.

[0050] Program aspects of the technology described herein may be considered a "product" or "article of manufacture," typically in the form of executable code and / or associated data carried on or embodied in some type of machine-readable medium. "Storage"-type media includes tangible memory of a computer or processor, or any or all of its associated modules, such as various semiconductor memories, tape drives, disk drives, etc., that may provide non-transitory storage for software programming at any time. All or portions of the software may, in some cases, be communicated over the Internet or various other telecommunications networks. Such communication may enable, for example, loading of software from one computer or processor to another, e.g., from a management server or host computer of a mobile communications network to a server computing platform, and / or from a server to a mobile device. Accordingly, other types of media that may carry software elements include optical waves, radio waves, and electromagnetic waves, such as those used across physical interfaces between local devices, over wired and optical landline networks, and via various wireless links. Physical elements that carry such waves, such as wired or wireless links, optical links, etc., may also be considered software-bearing media. As used herein, unless limited to non-transitory tangible "storage" media, terms such as computer or machine "readable medium" refer to any medium that participates in providing instructions to a processor for execution.

[0051] Although the methods, apparatus, and systems of the present disclosure are described with exemplary reference to control unit 175, embodiments of the present disclosure may be applicable to any environment, such as a desktop or laptop computer, and may be applicable to any type of Internet protocol.

[0052] In the foregoing description of exemplary embodiments of the invention, various features of the invention may be grouped together in a single embodiment, drawing, or description for the purpose of streamlining the disclosure and facilitating understanding of one or more of the various inventive aspects. However, this method of disclosure is not to be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as the claims reflect, inventive aspects may comprise fewer than all features of a single foregoing disclosed embodiment. Accordingly, the claims are expressly incorporated into this Detailed Description, with each claim standing on its own as a separate embodiment of the invention.

[0053] Furthermore, although some embodiments described herein may include some features included in other embodiments but not others, as will be understood by those skilled in the art, combinations of features from different embodiments are also intended to form different embodiments within the scope of the present invention. For example, in the claims, any of the claimed embodiments may be used in any combination.

[0054] Thus, while particular embodiments have been described, those skilled in the art will recognize that other and further modifications may be made thereto without departing from the spirit of the invention, and it is intended to claim all such modifications and variations that fall within the scope of the invention. For example, functions may be added or deleted from the block diagrams, and operations may be interchanged among steps shown in the figures. Steps may be added or deleted to methods described within the scope of the invention.

[0055] The subject matter disclosed above should be considered illustrative and not restrictive, and the claims are intended to encompass all such modifications, extensions, and other implementations that fall within the true spirit and scope of the present disclosure. Thus, to the maximum extent permitted by law, the scope of the present disclosure should be determined by the broadest permissible interpretation of the claims and their equivalents, and should not be limited or constrained by the above detailed description. While various implementations of the present disclosure have been described, it will be apparent to those skilled in the art that many more implementations are possible within the scope of the present disclosure. Accordingly, the present disclosure should not be limited except in light of the claims and their equivalents.

Claims

1. 1. A medical device system, comprising: a control unit configured to be operably coupled to the medical device, the control unit comprising: one or more processors implementing an algorithm for enhancing an image acquired by a first view element of the medical device, the one or more processors comprising: receiving a first image from the first view element; determining a current illumination value of the first image; if the current illuminance value is greater than the high illuminance target value, determining a first difference between the current illuminance value and the high illuminance target value; if the current illuminance value is less than the low illuminance target value, determining a second difference between the current illuminance value and the low illuminance target value; generating a new illumination value using at least one of the first difference and the second difference; converting the new illuminance value into a first voltage value for application to one or more illuminators of the medical device; A medical device system.

2. The medical device system of claim 1 , wherein the high and low target illumination values ​​together define a tolerance band around a target illumination value stored by the control unit.

3. the one or more processors: determining whether the current illuminance value is below a first illuminance threshold; if the current illumination value is below the first illumination threshold, generating the new illumination value using a scaling factor; 10. A medical device system according to any one of the preceding claims, further comprising:

4. the one or more processors: determining whether the new illumination value is greater than a maximum illumination value; if the new irradiance value is greater than a maximum irradiance value, converting the maximum irradiance value to a second voltage value for application to one or more illuminators of the medical device and increasing a gain of one or more imaging devices of the medical device; 10. A medical device system according to any one of the preceding claims, further comprising:

5. the one or more processors: determining whether the new illumination value is lower than the current illumination value; If the new illumination value is lower than the current illumination value, decreasing a gain of one or more imaging devices of the medical device; 10. A medical device system according to any one of the preceding claims, further comprising:

6. 10. A medical device system according to any one of the preceding claims, wherein the medical device is an endoscope.

7. 10. The medical device system of claim 9, wherein determining a current illumination value of the first image comprises accumulating and summing pixel values ​​of the first image.

8. the one or more processors: determining a current frame rate of the first view element; generating a new frame rate using at least one of the first difference and the second difference; applying the new frame rate to the first view element; 10. A medical device system according to any one of the preceding claims, further comprising:

9. The medical device system of claim 6 further comprising the medical device including the first viewing element and at least one illuminator.

10. the one or more processors: determining a current exposure time of the first view element; generating a new exposure time using at least one of the first difference and the second difference; applying the new exposure time to the first view element; 10. A medical device system according to any one of the preceding claims, further comprising:

11. the one or more processors: determining whether the current illumination value is below or above the new illumination value before converting the new illumination value to the first voltage value; if the current illumination value is less than the new illumination value, converting the new illumination value to the first voltage value; increasing the frame rate of the first view element if the current illumination value is greater than the new illumination value; 10. A medical device system according to any one of the preceding claims, further comprising:

12. generating a new illumination value using at least one of the first difference and the second difference includes determining a first error coefficient for the first image and a second error coefficient for a second image received by the control unit prior to the first image; if the current illuminance value is greater than the high illuminance target value, the first error coefficient is the first difference; 2. The medical device system of claim 1, wherein if the current illumination value is less than the low illumination target value, the first error coefficient is the second difference.

13. The medical device system of claim 12 , wherein generating the new illumination value further comprises determining proportional, integral, and derivative tuning constants each associated with the medical device.

14. the current illuminance value is a first current illuminance value, the high illuminance target value is a first high illuminance target value, the low illuminance target value is a first low illuminance target value, and the new illuminance value is a first new illuminance value; the one or more processors: receiving a second image from a second view element; determining a second current illumination value of the second image; if the second current illuminance value is greater than a second high illuminance target value, determining a third difference between the second current illuminance value and the second high illuminance target value; if the second current illuminance value is less than a second low light target value, determining a fourth difference between the second current illuminance value and the second low light target value; generating a second new illuminance value using at least one of the third difference and the fourth difference; converting the second new illuminance value into a second voltage value for application to one or more illuminators of the medical device; 10. A medical device system according to any one of the preceding claims, further comprising:

15. the one or more processors:

10. The medical device system of claim 9, further comprising the step of displaying a second image received from the first view element by at least one electronic display, the second image being illuminated by the one or more illuminators that received the first voltage value.

Images