Medical devices for automatic image brightness control and related systems and methods

Abstract

A medical device system (100) may include a control unit (175) that includes one or more processors that implement an algorithm to enhance an image obtained by a medical device (101). The one or more processing boards perform the steps of: receiving a first image from a first viewing element; determining a current illumination value of the first image; when the current illumination value is greater than a target high illumination value, determining a first difference value between the current illumination value and the target high illumination value; under the condition that the current illumination value is smaller than a target low illumination value, determining a second difference value between the current illumination value and the target low illumination value; generating a new illumination value by using at least one of the first difference value and the second difference value; and converting the new illumination value into a first voltage value for application to one or more illuminators of the medical device.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of priority to U.S. Provisional Application No. 63 / 377,433, filed on September 28, 2022, which is incorporated herein by reference in its entirety. Technical Field

[0002] Various aspects of the present disclosure generally relate to systems, devices and methods for automatic image brightness control. More specifically, embodiments of the present disclosure relate to imaging catheters (such as endoscopes or other medical devices that are configured to automatically control illuminators), and related systems and methods, among others. Background Art

[0003] During endoscopic surgery, the medical professional operating the endoscope typically relies on one or more illuminators to illuminate the field of view of the camera at the distal end of the endoscope. Most imaging catheters (such as endoscopes) rely on fixed illumination output, where each of the imaging catheter illuminators outputs constant illumination, such as from one or more light emitting diodes (LEDs). Such imaging catheters with constant illumination output control image brightness by changing the exposure and / or gain of the image sensor. This results in a significant step response in image brightness, as well as a slow response to changing scenes. The step response refers to the change in the output of the system when its input is a unit step function. The step response is due to the limited number of exposure steps available to the image sensor, and the slow response is at least in part because the exposure value is written in a single step at the end of each image frame. For example, if the exposure needs to be adjusted by 10 steps to increase or decrease the exposure of the image sensor, then at least 10 image frames will typically be required to adjust the brightness of the image, resulting in a significant lag (approximately 300 milliseconds for a 30 frame per second (fps) image sensor).

[0004] When users experience image lag with imaging catheter systems, procedures can be prolonged and procedures can risk being more difficult and delayed.Alternative methods of illumination adjustment for imaging catheters and other medical devices exist to reduce image lag and address other issues with medical device illumination and imaging systems. Summary of the invention

[0005] Aspects of the present disclosure relate to systems, apparatus, methods, etc. to help reduce imaging lag in medical device imaging systems, etc. The systems, apparatus, and methods of the present disclosure can reduce the time required to focus and / or properly illuminate the field of view of a camera or other imaging device of an endoscope or other medical device. Endoscopes and other medical devices incorporating the systems and methods of the present disclosure can help address image lag, can help reduce the time required for surgery, and can help address other problems. Each of the aspects disclosed herein can include one or more of the features described in combination with any of the other disclosed aspects.

[0006] According to one aspect, a medical device system may include a control unit configured to be operatively coupled to a medical device. The control unit may include one or more processors that implement an algorithm to enhance an image obtained by a first viewing element of the medical device. One or more processing boards perform the following steps: receiving a first image from the first viewing element; determining a current illumination value of the first image; determining a first difference between the current illumination value and the target high illumination value when the current illumination value is greater than a target high illumination value; determining a second difference between the current illumination value and the target low illumination value when the current illumination value is less than a target low illumination value; generating a new illumination value using at least one of the first difference and the second difference; and converting the new illumination value into a first voltage value for application to one or more illuminators of the medical device.

[0007] In other aspects, the medical device system may include one or more of the following features. The target high illumination value and the target low illumination value together may define a tolerance band around the target illumination value stored by the control unit. The one or more processing boards may also perform the following steps: determining a case where the current illumination value is below a first threshold illumination value; and generating a new illumination value using a scaling factor when the current illumination value is below the first threshold illumination value. The one or more processing boards may also perform the following steps: determining a case where the new illumination value is greater than a maximum illumination value; and converting the maximum illumination value into a second voltage value for application to one or more illuminators of the medical device and increasing a gain of one or more imaging devices when the new illumination value is greater than the maximum illumination value. The one or more processing boards may also perform the following steps: determining a case where the new illumination value is lower than the current illumination value; and reducing the gain of one or more imaging devices when the new illumination value is lower than the current illumination value. The medical device may be an endoscope. Determining the current illumination value of the first image may include accumulating and summing pixel values ​​of the first image. The one or more processing boards may also perform the following steps: determining a current frame rate of the first viewing element; generating a new frame rate using at least one of the first difference and the second difference; and applying the new frame rate to the first viewing element.

[0008] In other aspects, the medical device system may include one or more of the following features. The medical device may include a first viewing element and at least one illuminator. The one or more processors may also perform the following steps: determining a current exposure time for the first viewing element; generating a new exposure time using at least one of the first difference and the second difference; and applying the new exposure time to the first viewing element. The one or more processors may also perform the following steps: determining whether the current illumination value is below or above the new illumination value before converting the new illumination value to the first voltage value; converting the new illumination value to the first voltage value when the current illumination value is below the new illumination value; and increasing the frame rate of the first viewing element when the current illumination value is above the new illumination value. Generating the new illumination value using at least one of the first difference and the second difference may include determining a first error coefficient for a first image and a second error coefficient for a second image, wherein the second image is received by the control unit before the first image; wherein the first error coefficient is the first difference when the current illumination value is greater than the target high illumination value; and wherein the first error coefficient is the second difference when the current illumination value is less than the target low illumination value. Generating the new illumination value may also include determining a proportional adjustment constant, an integral adjustment constant, and a derivative adjustment constant, each of which is associated with the medical device.

[0009] In other aspects, the medical device system may include one or more of the following features. The current illumination value may be a first current illumination value, the target high illumination value may be a first target high illumination value, the target low illumination value may be a first target low illumination value, and the new illumination value may be a first new illumination value; and one or more processing boards may also perform the following steps: receiving a second image from a second viewing element; determining a second current illumination value of the second image; if the second current illumination value is greater than the second target high illumination value, determining a third difference between the second current illumination value and the second target high illumination value; if the second current illumination value is less than the second target low illumination value, determining a fourth difference between the second current illumination value and the second target low illumination value; using at least one of the third difference and the fourth difference, generating a second new illumination value; and converting the second new illumination value into a second voltage value for application to one or more illuminators of the medical device. One or more processing boards may also perform the following steps: displaying the second image received from the first viewing element via at least one electronic display, and the second image is illuminated by one or more illuminators receiving the first voltage.

[0010] In other aspects, methods of enhancing images are disclosed, which are obtained by a medical device system. The medical device system may include (a) one or more processors, and (b) a medical device, which is operatively connected to the one or more processors; wherein the medical device is configured to be inserted into a patient's body and includes a first viewing element and one or more illuminators. The method includes the following steps: receiving a first image from the first viewing element; determining a current illumination value of the first image; when the current illumination value is greater than a target high illumination value, determining a first difference between the current illumination value and the target high illumination value; when the current illumination value is less than a target low illumination value, determining a second difference between the current illumination value and the target low illumination value; generating a new illumination value using at least one of the first difference and the second difference; and converting the new illumination value into a first voltage value for application to one or more illuminators of the medical device.

[0011] In other aspects, the method may include one or more of the following features. The method may also include the following steps: determining a situation where the new illumination value is greater than the maximum illumination value; in the case where the new illumination value is greater than the maximum illumination value, converting the maximum illumination value into a second voltage value for application to one or more illuminators of the medical device and increasing the gain of one or more imaging devices. The method may also include the following steps: determining a current exposure time of the first observation element; generating a new exposure time using at least one of the first difference and the second difference; and applying the new exposure time to the first observation element. The method may also include the following steps: determining a current frame rate of the first observation element; generating a new frame rate using at least one of the first difference and the second difference; and applying the new frame rate to the first observation element.

[0012] In other aspects, a non-transitory computer readable medium may include program instructions for causing a computer to perform a method for enhancing images obtained through a first viewing element in a medical device system; and the medical device system may include a processor configured to implement a process, and a medical device operatively connected to the processor, the medical device configured to be inserted into a patient's body and including a first viewing element and one or more illuminators. The method may include the following steps: receiving a first image from the first viewing element; determining a current illumination value of the first image; if the current illumination value is greater than a target high illumination value, determining a first difference between the current illumination value and the target high illumination value; if the current illumination value is less than a target low illumination value, determining a second difference between the current illumination value and the target low illumination value; generating a new illumination value using at least one of the first difference and the second difference; and converting the new illumination value into a first voltage value for application to one or more illuminators of the medical device.

[0013] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate several exemplary aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0015] Figure 1A and Figure 1B A perspective view of an exemplary endoscope system according to aspects of the present disclosure;

[0016] Figure 2 According to aspects of the present disclosure, exemplary methods are shown for automatically adjusting lighting of a medical device;

[0017] Figure 3 According to various aspects of the present disclosure, Figure 2 Optional additional parts of the method;

[0018] Figure 4 Another exemplary method is shown according to aspects of the present disclosure for automatically adjusting lighting of a medical device;

[0019] Figure 5 An exemplary graph of illumination values ​​and exposure time values ​​applied to one or more illuminators and one or more imaging devices of a PID controlled system is shown according to aspects of the present disclosure;

[0020] Figure 6 Another exemplary method is shown according to aspects of the present disclosure for automatically adjusting lighting of a medical device;

[0021] Figure 7 According to aspects of the present disclosure, there is a simplified functional block diagram of a computer and / or server that can be configured as an apparatus or system to perform any of the methods described herein. DETAILED DESCRIPTION

[0022] Reference will now be made in detail to various aspects of the present disclosure, examples of which are illustrated in the accompanying drawings. Wherever possible, the same or similar reference numerals will be used in the drawings to refer to the same or similar parts. When the device is introduced into a patient's body, the term "distal" refers to the portion farthest from the user. In contrast, when the device is placed in a patient's body, the term "proximal" refers to the portion closest to the user. Figure 1A and Figure 1B, arrows marked "P" and "D" are used to illustrate the proximal and distal directions in the figure. As used herein, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements, but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. The term "exemplary" is used in the sense of "example" rather than "ideal." In addition, relative terms such as, for example, "about," "substantially," "approximately," etc., are used to indicate possible variations of ±10% of the stated numerical value or range.

[0023] Embodiments of the present disclosure are directed to improving the illumination and imaging of medical devices (such as endoscopes) during medical procedures. As non-limiting exemplary benefits, aspects of the present disclosure can reduce the lag experienced by the imaging system, and / or can facilitate viewing of the field of view of one or more cameras of the medical device, etc.

[0024] Figure 1A and Figure 1B A perspective view of an exemplary endoscope system 100 is shown. Endoscope system 100 may include endoscope 101. Although the term endoscope may be used herein, it should be understood that other devices (including, but not limited to, duodenoscopes, colonoscopes, ureteroscopes, bronchoscopes, laparoscopes, sheaths, catheters, or any other suitable delivery device or other type of medical device) may be used in conjunction with the systems and methods of the present disclosure, and the systems and methods discussed below may be incorporated into any of these or other medical devices.

[0025] Endoscope 101 may include a handle assembly 106 and a flexible tubular shaft 108. Handle assembly 106 may include one or more of the following: biopsy port 102, biopsy cap 103, image capture button 104, lifter actuator 107, locking lever 109, locking knob 110, first control knob 112, second control knob 114, suction button 116, air / water button 118, handle body 120, and umbilicus 105. All actuators, lifters, knobs, buttons, levers, ports, or caps of endoscope system 100 (such as those listed above) may be used for any purpose and are not limited by any particular use, which may be implied by the corresponding nomenclature of each component used herein. Umbilicus 105 may extend from handle body 120 to auxiliary devices, such as control unit 175, water / fluid supply, and / or vacuum source. The umbilicus 105 may transmit signals between the endoscope 101 and the control unit 175 to control the illumination and imaging components of the endoscope 101, and / or receive image data from the endoscope 101. The umbilicus 105 may also provide fluid from a water / fluid supply for flushing and / or provide suction to the distal end 119 of the shaft 108. Buttons 116 and 118 may control valves for suction and fluid supply (e.g., air and water), respectively. The shaft 108 may terminate at the distal end 119. The shaft 108 may include an articulated section 122 for deflecting the distal end 119 in an upward, downward, leftward, and / or rightward direction. Knobs 112 and 114 may be used to control such deflection. Locking lever 109 and locking knob 110 may lock knobs 112 and 114 in desired positions, respectively.

[0026] The distal tip 119 may include one or more imaging devices 125, 126 and illumination sources 127-130 (e.g., one or more LEDs, optical fibers, and / or other illuminators). Examples of imaging devices (or viewing elements) 125, 126 include one or more cameras, one or more image sensors, an endoscope viewing element, an optical assembly (including one or more image sensors and one or more lenses), and any other imaging device known in the art. Figure 1AAs shown, the distal tip 119 may include a forward-pointing imaging device 125 and a side-pointing imaging device 126. However, in other embodiments, the distal tip 119 may include only one imaging device 125, 126, which may be forward-pointing or side-pointing. In other examples, the distal tip 119 may include three or more imaging devices 125, 126 pointed in different directions; and in some examples, the field of view of each imaging device 125, 126 may overlap. The distal tip 119 may include one or more illuminators 127-130; and one or more illuminators 127, 128 may be forward-pointing illuminators (or facing in the distal direction), and one or more illuminators may be side-pointing illuminators 129, 130. The side-pointing imaging device 126 and the side-pointing illuminators 129, 130 may face radially outward, vertically, substantially vertically, or otherwise transverse to the longitudinal axis of the shaft 108 and the distal tip 109. The forward or forward-facing imaging device 125 and forward-pointing illuminators 127 , 128 may face generally along the longitudinal axis of the distal tip 119 and shaft 108 .

[0027] The disclosed endoscope system 100 may also include a control unit 175, such as Figure 1A and Figure 1B As shown. The control unit 175 can be coupled to the endoscope 101 to provide power and instructions to the imaging devices 125, 126 and the illuminators 127-130. The control unit 175 can also control other aspects of the endoscope 101, such as, for example, the application of suction, the deployment or delivery of fluids, and / or the movement of the distal tip 119. The control unit 175 can be powered by an external power source (such as an electrical outlet). In addition, the control unit 175 may include buttons, knobs, a touch screen, or other user interface to control the imaging devices 125, 126, the illuminators 127-130, and other features of the endoscope 101. The control unit 175 can be housed in the handle body 120 itself or in a separate device.

[0028] The control unit 175 may be configured to allow a user to set or control one or more lighting and imaging parameters. For example, the control unit 175 may allow a user to set or control the lighting level of each of the illuminators 127-130, the gain level of each of the imaging devices 125, 126, the exposure time of each of the imaging devices 125, 126, the frame rate of each of the imaging devices 125, 126, the maximum or target value of any of the lighting and imaging parameters, and / or any other parameters associated with the imaging devices 125, 126 and the illuminators 127-130. In some examples, the control unit 175 may be configured to execute one or more algorithms using one or more lighting and imaging parameters, such as to automatically adjust the lighting level of one or more of the illuminators 127-130 and / or to automatically adjust one or more parameters of the imaging devices 125, 126. For example, the control unit 175 may set or select the lighting level of one or more illuminators 127-130 based on data received from one or more imaging devices 125, 126.

[0029] The control unit 175 may include electronic circuitry configured to receive, process, and / or transmit data and signals between the endoscope 101 and one or more other devices. For example, the control unit 175 may be in electronic communication with a display configured to display an image based on image data and / or signals processed by the control unit 175, which image data and / or signals may have been generated by the imaging devices 125, 126 of the endoscope 101. The control unit 175 may be in electronic communication with the display in any suitable manner, either via wires or wirelessly. The display may be manufactured in any suitable manner and may include a touch screen input and / or may be connected to various input and output devices, such as, for example, a mouse, an electronic stylus, a printer, a server, and / or other portable electronic devices. The control unit 175 may include software and / or hardware that facilitates operations such as those discussed above. For example, the control unit 175 may include one or more algorithms, models, etc. for executing any of the methods and / or systems discussed in the present disclosure; and may be configured to automatically adjust the illumination value applied to one or more illuminators 127-130, and automatically adjust the gain and frame rate applied to one or more imaging devices 125, 126.

[0030] When operating the endoscope system 100, the user can use his / her left hand to hold the handle assembly 106, while the right hand is used to hold auxiliary devices and / or operate one or more of the actuators of the handle assembly 106, such as the first and second control knobs 112, 114 and the locking lever 109 and the locking knob 110. While grasping the handle body 120, the user can use the fingers of the left hand to operate the image capture button 104, the suction button 116, and / or the air / water button 118 (each by pressing). During the procedure, the user can observe the field of view of one or more of the imaging devices 125, 126 on an electronic display that is operably connected to the control unit 175. One or more illuminators 127-130 can provide illumination to the field of view of one or more of the imaging devices 125, 126.

[0031] Figures 2 to 4 Flowcharts for illustrating various control loop algorithms that may be implemented by the endoscope system 100 or any other medical device system having one or more imaging devices and one or more illuminators. Although the algorithms described herein are discussed with respect to an endoscope system, the algorithms are not so limited and may be implemented using any medical device system known in the art that includes an imaging component and an illumination component. In general, the algorithms discussed herein vary illumination levels based on varying data received from one or more imaging devices, such as a varying field of view of an imaging device.

[0032] Figure 2 An illumination control method 200 is shown that may be automatically performed by the control unit 175 of the endoscope system 100. The method 200 utilizes proportional, integral, and derivative (PID) coefficients to control the illumination speed and accuracy of the endoscope system 100. In an initial step 201, a target image brightness is stored in the control unit 175, and an initial illumination value is set by the control unit 175. In some instances, a user may select the target image brightness; and in other instances, the control unit 175 may automatically determine the target image brightness. For example, the control unit 175 may utilize previously program data to determine the target image brightness for use in the method 200. In some instances, the target image brightness may be a range of brightness values ​​applied to one or more illuminators 127-130, ranging from a target low (T 低 ) brightness value to target high (T 高 ) range of brightness values. For example, the range of brightness values ​​for the target image brightness may be a tolerance band set around a specific target brightness value. Additionally, during step 201, a user or control unit 175 may set an initial illumination value to be applied to one or more illuminators 127-130.

[0033] In a next step 202, the control unit 175 may determine the actual illumination value by accumulating and summing the pixel values ​​of the current image frame received from the one or more imaging devices 125, 126. In some examples, only a single image frame of a single imaging device 125, 126 may be used to determine the initial illumination value; and in other examples, multiple image frames from the one or more imaging devices 125, 126 may be used.

[0034] During the time between frames received from one or more imaging devices 125, 126 (e.g., during vertical blanking), step 203 includes determining the error and PID coefficients for the current image frame. To determine the error coefficients for the current image frame, the control unit may perform the following calculations: When B 当前 >T 高 When, error = T 高 –B 当前； When B 当前 <T 低 When, error = T 低 –B 当前。 B 当前 is the calculated illumination value of the current image frame. 低 is the target low illumination value; and T 高 is the target high illumination value. In some examples, the control unit 175 may determine the target brightness range (or illumination value range) as a tolerance band around the target illumination value set by the user. To determine the PID coefficient, the control unit 175 may perform the following calculation: P=K P *(error); I=K I *(error + old error); D=K D *(error-olderror); Then, old error = (error). K P K is the proportional adjustment constant. I is the integral adjustment constant, and K D is the derivative adjustment constant. "Error" is the calculated error coefficient of the current image frame, and "Old Error" is the calculated error coefficient of the previous image frame. Adjustment parameters (e.g., adjustment constant K P , K I and K D ) is determined experimentally and depends on the type of lighting used and the drive circuit. In some examples, the adjustment constant K P , K I and K DThe speed of the PID loop is directly related to the control constant K P , K I and K D . Adjustment constant K P The output is adjusted in proportion to the current error. Adjustment constant K I Control static error. Adjust constant K D Based on the rate of change of the error, and provides a damping effect on the output.

[0035] At step 204, once the error and PID coefficients for the current image frame are determined using the calculations described above, the control unit 175 may determine new illumination values ​​to apply to one or more illuminators 127-130, for example, using the old illumination values, PID coefficients, and hardware scaling factors. The new illumination values ​​may be determined by the following calculation: Temporary_Value=(P+I+D) / F 缩放 ; B 新 =B 当前 +Temporary_Value; Then, B 当前 =B 新 . F 缩放 is a scaling factor based on the hardware used to determine the lighting, such as the type of luminaire (LED, fiber optic, etc.) and the circuitry connected to the luminaire. 缩放 It can be based on the driver circuit and can depend on the specific hardware implementation and the desired allowed lighting range. Temporary_Value is used by the control unit to determine B 新 The calculated illumination values ​​are digital numbers that are converted into analog voltages, currents, or any other digital values ​​that control the illumination applied to the one or more illuminators 127-130, thereby allowing the control unit 175 to control the illumination of the endoscope 101. In some examples, once step 204 is completed and B 新 Applied to one or more illuminators 127-130, the control unit 175 will repeat steps 202-204 for the next image frame of one or more imaging devices 125, 126. Thus, steps 201-204 are an example of a control loop algorithm to automatically adjust the illumination of the endoscope system 100.

[0036] In some instances, Figure 2 The control loop algorithm may include an additional step 205: determining B 新Whether (the new lighting value) is near the top or bottom of the range of lighting values ​​accepted by the particular luminaire or the range of lighting values ​​that the luminaire's hardware is capable of accepting. In some examples, the control loop algorithm may not include step 205, and may proceed from step 204 to step 202 (shown in dashed lines) to allow the ring algorithm to continue looping.

[0037] Figure 3 It shows that if B 新 If B is near the bottom or top of the range of illumination values ​​accepted by one or more luminaires, 新 is the maximum value in the range of illumination values ​​accepted by one or more luminaires, and if B 新 At the minimum value of the range of illumination values ​​accepted by one or more luminaires, the control unit 175 may respectively perform different steps 301 - 303 .

[0038] In step 301, if B 新 Near the bottom or top of the range of illumination values ​​received by one or more luminaires, Figure 2 During the next cycle of the algorithm, a different scaling factor (F 缩放 ). The different scaling factors (F 缩放 ) is smaller than the previously used scaling factor (F 缩放 ) to force the change in illumination value to be smaller than the previous change in illumination value. 新 When you are near the bottom or top of the lighting value range, adjust the scaling factor (F 缩放 ) can facilitate the reduction or elimination of any illumination oscillations caused by nonlinearities in the hardware of one or more illuminators.

[0039] In step 302, B 新 The illumination value is at a maximum value of a range of illumination values ​​accepted by the one or more illuminators (e.g., the illumination value is at a maximum value). Because the illumination value cannot be increased beyond the maximum value of the range of illumination values ​​accepted by the one or more illuminators, the control unit 175 can adjust the digital gain of one or more image sensors associated with the one or more imaging devices 125, 126. For example, in which B 新 Under normal conditions where the illumination or brightness level of the current image frame is at a maximum value of the range of illumination values ​​and is below a minimum target illumination value, the gain of one or more image sensors of the one or more imaging devices 125, 126 is increased. The gain is then increased (e.g., in a stepwise manner) until the minimum target illumination value of the current image frame is reached or the maximum gain of the one or more image sensors is reached. In some examples, if the saturation value (e.g., a numerical value representing color intensity) of the current image frame decreases from the target saturation value as the gain of the one or more image sensors is being increased in a stepwise manner, the gain may alternatively be increased to the maximum gain while the illumination value of the current image frame is decreasing.

[0040] In step 303, B 新 The illumination value is at a minimum value of a range of illumination values ​​accepted by the one or more illuminators (e.g., the illumination value is at a minimum value). Because the illumination value cannot decrease beyond the minimum value of the range of illumination values ​​accepted by the one or more illuminators, the control unit 175 can adjust the digital gain of one or more image sensors associated with the one or more imaging devices 125, 126. For example, in which B 新 Under normal conditions where the illumination or brightness level of the current image frame is at a minimum value of the illumination value range and is above a maximum target illumination value, the gain of one or more image sensors of one or more imaging devices 125, 126 is reduced. The gain is then reduced (e.g., in a stepwise manner) until the maximum target illumination value of the current image frame is reached or the minimum gain of the one or more image sensors is reached. In some examples, if the saturation value of the current image frame decreases from the target saturation value as the gain of the one or more image sensors is being reduced in a stepwise manner, the gain may alternatively be reduced to the minimum gain while the illumination value of the current image frame is increased. After completing any of steps 301, 302, or 303, the control unit may continue Figure 2 Another cycle of the control loop algorithm of the control unit 175, for example, begins at step 202: determining the actual image brightness of the next image frame received from the one or more imaging devices 125, 126. When the control loop algorithm executed by the control unit 175 incorporates steps 205 and steps 301-303, the control unit 175 can automatically switch between (i) adjusting the illumination value applied to the one or more illuminators 127-130 and (ii) adjusting the gain of the one or more image sensors of the one or more imaging devices 125, 126.

[0041] In some examples, to speed up the control unit 175 for B 当前 In response to an extreme difference between the image sensor and the target illumination value, the extreme image brightness protection can be used to change the gain of one or more image sensors by a value greater than 1. The extreme image brightness protection can be B 当前 and the target illumination value. Once the extreme image brightness protection is met (or B 当前 and the target lighting level), the gain will be increased by the control unit 175 with B 当前 A value that is proportional to the difference between the target illumination value and the gain. For example, if the gain is at the low end of the gain value range and B 当前 If the lighting value suddenly drops below the target lighting value, the gain is increased by the control unit 175 with B 当前A value that is proportional to the difference between the target illumination value and the gain. The scaling factor can be 2, 5, 10, or any number suitable to reach the target illumination value more quickly. By adjusting the rate at which the gain is increased or decreased, the lag time in reaching the target illumination value is reduced. When the extreme image brightness protection is not reached, the gain is increased or decreased by 1 as needed to reach the target illumination value.

[0042] Figure 4 Another illumination control method 400 is shown, which may be automatically performed by the control unit 175 (or another control unit) of the endoscope system 100 . Figure 4 The method 400 of the present invention utilizes proportional, integral and derivative (PID) coefficients to control the illumination speed and accuracy of the endoscope system 100. In an initial step 401, a target image brightness (e.g., a target illumination value) is stored in the control unit 175; and an initial illumination value (e.g., a preset illumination value) of one or more illuminators 127-130 and an initial exposure time of one or more imaging devices 125, 126 are determined by the control unit 175. The initial illumination value applied by the control unit 175 to the method 400 may be a specific illumination value that provides enough light so that the image brightness is sufficient to meet the average imaging volume at the maximum exposure available to the imaging device 125, 126 for the desired frame rate. The specific illumination value may be set by the user or automatically applied by the control unit 175. For example, the default illumination value may achieve an average image brightness of approximately 40% to 50%, and the user may be able to adjust the brightness value within a range between 25% and 70% image brightness based on user preference and clinical needs. In some examples, the user may select a target image brightness (e.g., a target illumination value); and in other examples, the control unit 175 may automatically determine the target image brightness. For example, the control unit 175 may utilize previous program data to determine the target image brightness for use in Figure 2 In some examples, the target image brightness can be a range of brightness values ​​applied to one or more illuminators 127-130, ranging from a target low (T 低 ) brightness value to target high (T 高 )The range of brightness values.

[0043] In the next step 402, the Figure 2 In the same manner as described in step 202 of , the control unit 175 may accumulate and sum the pixel values ​​of the current image frame received from the one or more imaging devices 125, 126. In some examples, only a single image frame from a single imaging device 125, 126 may be used to determine the initial illumination value; and in other examples, multiple image frames from the one or more imaging devices 125, 126 may be used.

[0044] During the time between frames received from one or more imaging devices 125, 126 (e.g., during vertical blanking), step 403 includes determining error and PID coefficients for the current image frame. The error and PID coefficients are similar to those described above with respect to Figure 2 When using the PID loop and exposure time, F 缩放 may change; for example, the user may desire to scale the output to have a smaller step size to more finely control the illumination. Once the error and PID coefficients for the current image frame are determined, the control unit 175 may determine a new illumination value, and at step 404, the exposure time may be adjusted based on the new illumination value. For example, when the image is too bright (e.g., B 新 >B 当前 ), the exposure time is reduced; and when the image is too dark (e.g., B 当前 >B 新 ), the exposure time increases. After each execution of the PID algorithm loop for the current frame, the exposure time can be increased by a single unit or by multiple units. Once the exposure time reaches the maximum value and the image remains too dark (e.g., B 当前 >B 新 ), then the illumination value provided to one or more illuminators 127-130 may be obtained using the above relative Figure 2 The same luminaire control algorithm described is added.

[0045] Sensor manufacturers typically set up controllable registers for exposure so that the number written to one or more of the registers is some fraction of a row. For example, a 480-row sensor running at 30 frames per second will have a row time of approximately 65 microseconds. If the exposure number in the register corresponds to 1 / 16 of a row, then writing the number 16 to the register will result in an exposure time of approximately 65 microseconds. In this example, the maximum exposure allowed for a particular image sensor will be approximately 30 microseconds, since any exposure longer than 30 microseconds will force the frame rate to be reduced. By knowing the exposure in fractions of a row, the exposure of the image sensor can be controlled using a PID-based algorithm such as Figure 4 Method 400. By utilizing Figure 4 By adjusting the exposure time of one or more imaging devices 125, 126 and adjusting the illumination values ​​applied to one or more illuminators 127-130 according to method 400, the spectral stability of the light used to illuminate the target anatomical structure can be increased. In addition, method 400 can be advantageous in minimizing color shift due to different illumination scenarios.

[0046] Figure 5 An exemplary graph of illumination values ​​and exposure time values ​​is shown, the illumination value being used in a PID control system (such as with respect to Figure 2The exposure time value is applied to one or more illuminators 127-130 in the PID control system (such as relative to Figure 4 126 . By utilizing the slope of the illumination value and the exposure time value, a more complex method can be used to adjust the brightness of the image, which switches between (i) adjusting the illumination value of one or more illuminators 127-130 and (ii) adjusting the exposure time of one or more imaging devices 125, 126. For example, if the need for a change in brightness is large (e.g., above a certain threshold), then the illumination value applied to the illuminators 127-130 can be adjusted to modify the image brightness. By adjusting the illumination value applied to the illuminators 127-130, the increase or decrease in brightness will take effect in less time than if the exposure time is adjusted. If the need for a change in brightness is small (e.g., below a certain threshold), then the exposure time of one or more imaging devices 125, 126 can be adjusted to modify the image brightness. Compared to a single step increase or decrease in the illumination value, an increase or decrease in brightness due to a single step increase or decrease in exposure time will have a smaller impact on the image brightness. By switching between adjusting the illumination values ​​applied to the illuminators 127-130 and adjusting the exposure times applied to the imaging devices 125, 126, the brightness of the image can be controlled with greater precision, particularly near the lower limit of illumination control.

[0047] Figure 6 Another illumination control method 600 is shown, which may be automatically performed by the control unit 175 (or another control unit) of the endoscope system 100 . Figure 6 The method 600 utilizes proportional, integral and derivative (PID) coefficients to control the illumination speed and accuracy of the endoscope system 100. Figures 2 to 5 Any of the other methods discussed, Figure 6 The method 600 incorporates frame rate control of the imaging devices 125, 126 as an additional aspect of controlling the brightness of the received images. Figure 6 As shown, in an initial step 601, a target image brightness (e.g., a target illumination value) is set (e.g., stored in the control unit 175); and initial illumination values ​​of one or more illuminators 127-130 are determined by the control unit 175. In addition, during step 601, an initial frame rate is set at one or more imaging devices 125, 126 (e.g., stored in the control unit 175).

[0048] In the next step 602, the Figure 2In the same manner as described in step 202 , the control unit 175 may determine the actual image brightness by accumulating and summing the pixel values ​​of the current image frame received from one or more imaging devices 125 , 126 .

[0049] During the time between frames received from one or more imaging devices 125, 126 (e.g., during vertical blanking), at steps 603 and 605, the control unit 175 determines whether the actual illumination value of the current image frame is below (step 603) or above (step 605) the target illumination value.

[0050] If the actual illumination value of the current image frame is below (step 603) the target illumination value, the control unit 175 will (i) first adjust the illumination value applied to the one or more illuminators 127-130 until the maximum illumination value of the one or more illuminators 127-130 is reached or the target illumination value is reached, and then (ii) adjust the gain value applied to the one or more imaging devices 125, 126 until the maximum gain value is reached or the target illumination value is reached. As shown in step 604, if the illumination value applied to the one or more illuminators and the gain value applied to the one or more imaging devices 125, 126 are both at their respective maximum values, the control unit 175 will continue to reduce the frame rate of the one or more imaging devices 125, 126 to allow for an increase in exposure time until the target illumination value of the received image is reached or the minimum frame rate is reached.

[0051] If the actual illumination value of the current image frame is higher than (step 605) the target illumination value, the control unit 175 will increase the frame rate applied to the one or more imaging devices 125, 126 until the target illumination value of the received image is reached or the maximum frame rate is reached. Then, if the actual illumination value is still higher than the target illumination value and the maximum frame rate is reached (step 606), the control unit 175 will (i) adjust the illumination value applied to the one or more illuminators 127-130 until the minimum illumination value of the one or more illuminators 127-130 is reached or the target illumination value is reached, and then (ii) adjust the gain value applied to the one or more imaging devices 125, 126 until the minimum gain value is reached or the target illumination value is reached. By combining the automatic adjustment of the illumination value (applied to the one or more illuminators 127-130), the gain value (applied to the one or more imaging devices 125, 126) and the frame rate (applied to the one or more imaging devices 125, 126), the image brightness can be adjusted more effectively, for example to minimize color shift due to different lighting scenes. Higher frame rates may also result in reduced video latency, and allow for the presentation of "smoother" video.

[0052] In various embodiments, any of the systems and methods described herein may include a control unit 175 and a medical device (e.g., endoscope 101), and the control unit 175 may include a processor (in the form of one or more processors or central processing units ("CPU")) for executing program instructions. In some instances, the one or more processors may be one or more processing boards. The control unit 175 may include an internal communication bus and a storage unit (such as ROM, HDD, SDD, etc.) that may store data on a computer-readable medium, but the control unit 175 may receive programming and data via network communications. The control unit 175 may also have a memory (such as RAM) that stores instructions for performing the techniques presented herein, but these instructions may be temporarily or permanently stored in other modules of the control unit 175 (e.g., processors and / or computer-readable media) or remotely stored, such as on a cloud server that is electronically connected to the control unit 175. The various system functions of the control unit 175 may be implemented in a distributed manner on multiple similar platforms to distribute the processing load. Alternatively, the system discussed herein may be implemented by appropriate programming of one computer hardware platform at the control unit 175.

[0053] Figure 7 A functional block diagram of a general computer hardware platform is provided. Figure 7 A network or host computer platform is shown, which may be typically used to implement a server 700 or a browser, or any other device that performs the features of the methods and systems described herein. It is believed that those skilled in the art are familiar with the structure, programming and general operation of such computer devices, and therefore, the drawings should be self-explanatory.

[0054] For example, a platform for server 700, etc. may include a data communication interface for packet data communication 760. The platform may also include a central processing unit (CPU) 720 in the form of one or more processors for executing program instructions. The platform typically includes an internal communication bus 710, a program storage device, and a data storage device for various data files, which will be processed and / or communicated by the platform (such as ROM 730 and RAM 740), but the server 700 typically receives programming and data via network communication 770. The hardware components, operating systems, and programming languages ​​of such devices are conventional in nature, and it is presumed that those skilled in the art are very familiar with them. The server 700 may also include input and output ports 750 to connect input and output devices, such as keyboards, mice, touch screens, monitors, displays, etc. Of course, various server functions can be implemented in a distributed manner on multiple similar platforms to distribute the processing load. Alternatively, the server can be implemented by appropriate programming of a computer hardware platform.

[0055] The program aspects of the technology discussed herein can be generally regarded as a "product" or "article" in the form of executable code and / or related data, which is carried on or implemented in a type of machine-readable medium. "Storage" type media include any or all of the tangible memories of computers, processors, etc. or their related modules, such as various semiconductor memories, tape drives, hard drives, etc., which can provide non-transient storage to software programming at any time. All or part of the content of the software may sometimes be communicated through the Internet or various other telecommunications networks. For example, such communications can allow the loading of software from one computer or processor to another, for example, from the management server or host of a mobile communication network to the computer platform of the server and / or from the server to the mobile device. Therefore, another type of medium that can carry software elements includes light waves, radio waves and electromagnetic waves, such as through wired and optical fixed telephone networks and through various air links on the physical interface between local devices. Physical elements that carry such waves (such as wired or wireless links, optical links, etc.) can also be regarded as the medium that carries the software. As used herein, unless restricted to non-transitory, tangible "storage" media, terms such as computer or machine "readable media" refer to any medium that participates in providing instructions to a processor for execution.

[0056] Although the disclosed methods, apparatuses, and systems are described with exemplary reference to the control unit 175, it should be understood that the disclosed embodiments are applicable to any environment, such as a desktop or notebook computer, etc. In addition, the disclosed embodiments are applicable to any type of Internet protocol.

[0057] It should be understood that in the above description of exemplary embodiments of the present invention, various features of the present invention are sometimes combined together in a single embodiment, figure, or description thereof for the purpose of streamlining the present disclosure and assisting in the understanding of one or more of the various inventive aspects. However, this method of disclosure should not be interpreted as reflecting an intent that the claimed invention requires more features than are expressly recited in each claim. On the contrary, as the following claims reflect, multiple inventive aspects lie in less than all the features of a single preceding disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into the detailed description, with each claim existing on its own as an independent embodiment of the present invention.

[0058] In addition, although some embodiments described herein include some features and do not include other features included in other embodiments, the combination of features of different embodiments is intended to be within the scope of the present invention and form different embodiments, as will be understood by those skilled in the art. For example, in the following claims, any of the claimed embodiments may be used in any combination.

[0059] Thus, while certain embodiments have been described, those skilled in the art will recognize that other and additional modifications may be made thereto without departing from the spirit of the invention, and it is intended to claim all such changes and modifications that fall within the scope of the invention. For example, functionality may be added to or deleted from the block diagrams, and operations may be interchanged between the steps shown in the drawings. Steps may be added to or deleted from the methods described within the scope of the invention.

[0060] The subject matter disclosed above should be considered illustrative, not restrictive; and the appended claims are intended to cover all such modifications, enhancements, and other embodiments that fall within the true spirit and scope of the present disclosure. Therefore, to the maximum extent permitted by law, the scope of the present disclosure should be determined by the broadest interpretation of the following claims and their equivalents, and should not be limited or constrained by the foregoing specific embodiments. Although various embodiments of the present disclosure have been described, it will be apparent to those skilled in the art that more embodiments are possible within the scope of the present disclosure. Therefore, the present disclosure is not limited except by the appended claims and their equivalents.

Claims

1. A medical device system, comprising: A control unit configured to be operatively coupled to a medical device, wherein the control unit comprises: one or more processors implementing an algorithm to enhance an image acquired by a first viewing element of the medical device, wherein the one or more processors perform the following steps: receiving a first image from the first viewing element; determining a current illumination value of the first image; In a case where the current lighting value is greater than the target high lighting value, determining a first difference between the current lighting value and the target high lighting value; In a case where the current lighting value is less than the target low lighting value, determining a second difference between the current lighting value and the target low lighting value; generating a new illumination value using at least one of the first difference and the second difference; and The new illumination value is converted to a first voltage value for application to one or more illuminators of the medical device.

2. The system according to claim 1, wherein: The target high illumination value and the target low illumination value together define a tolerance band around a target illumination value stored by the control unit.

3. A system according to any one of the preceding claims, wherein: The one or more processors also perform the following steps: determining a condition where the current illumination value is below a first threshold illumination value; and In the event that the current illumination value is below the first threshold illumination value, the new illumination value is generated using a scaling factor.

4. A system according to any one of the preceding claims, wherein: The one or more processors also perform the following steps: determining a condition where the new illumination value is greater than a maximum illumination value; and In the event that the new illumination value is greater than the maximum illumination value, the maximum illumination value is converted to a second voltage value for application to one or more illuminators of the medical device and a gain of one or more imaging devices of the medical device is increased.

5. A system according to any one of the preceding claims, wherein: The one or more processors also perform the following steps: determining a condition where the new lighting value is less than the current lighting value; and In case the new illumination value is lower than the current illumination value, a gain of one or more imaging devices of the medical device is reduced.

6. A system according to any one of the preceding claims, wherein: The medical device is an endoscope.

7. A system according to any one of the preceding claims, wherein: Determining the current illumination value of the first image includes accumulating and summing pixel values ​​of the first image.

8. A system according to any one of the preceding claims, wherein: The one or more processors also perform the following steps: determining a current frame rate of the first viewing element; generating a new frame rate using at least one of the first difference and the second difference; and The new frame rate is applied to the first viewing element.

9. The system of claim 6, further comprising the medical device comprising the first viewing element and at least one illuminator.

10. A system according to any one of the preceding claims, wherein: The one or more processors also perform the following steps: determining a current exposure time of the first viewing element; generating a new exposure time using at least one of the first difference and the second difference; and The new exposure time is applied to the first viewing element.

11. A system according to any one of the preceding claims, wherein: The one or more processors also perform the following steps: Before converting the new lighting value to the first voltage value, determining whether the current lighting value is lower or higher than the new lighting value; In a case where the current lighting value is lower than the new lighting value, converting the new lighting value into the first voltage value; and In case the current illumination value is higher than the new illumination value, the frame rate of the first viewing element is increased.

12. A system according to any one of the preceding claims, wherein: Generating a new illumination value using at least one of the first difference value and the second difference value comprises determining a first error coefficient for the first image and a second error coefficient for a second image, wherein the second image is received by the control unit before the first image; Wherein, when the current illumination value is greater than the target high illumination value, the first error coefficient is the first difference; and Wherein, when the current lighting value is less than the target low lighting value, the first error coefficient is the second difference.

13. The system according to claim 12, wherein: Generating the new illumination value also includes determining a proportional adjustment constant, an integral adjustment constant, and a derivative adjustment constant, each associated with the medical device.

14. A system according to any one of the preceding claims, wherein: The current lighting value is a first current lighting value, the target high lighting value is a first target high lighting value, the target low lighting value is a first target low lighting value, and the new lighting value is a first new lighting value, and The one or more processors further perform the following steps: receiving a second image from a second viewing element; determining a second current illumination value of the second image; determining a third difference between the second current lighting value and the second target high lighting value when the second current lighting value is greater than the second target high lighting value; determining a fourth difference between the second current lighting value and the second target low lighting value when the second current lighting value is less than the second target low lighting value; generating a second new illumination value using at least one of the third difference value and the fourth difference value; and The second new illumination value is converted into a second voltage value for application to one or more illuminators of the medical device.

15. A system according to any one of the preceding claims, wherein: The one or more processors also perform the following steps: A second image received from the first viewing element is displayed via at least one electronic display, wherein the second image is illuminated by the one or more illuminators receiving the first voltage value.

Images