Information management and inventory tracking for medical devices

Abstract

During the manufacture of a medical device, if the device is measured, the parameters and the results of the measurements are stored in a database. For example, the parameters of an endoscope's image sensor may be measured and stored in a database. The medical device has a machine-readable serial number. A computer reads the serial number and records a decrease in on-hand inventory of the medical device. When the on-hand inventory indicates a reason for a reorder to replenish the on-hand inventory, the computer places an order with a supplier of the class of medical device to replenish the inventory. A computer controlling the medical device may read the database to obtain the measured characteristics. The obtained data may be used to calibrate or adjust the device's functions. For example, the white balance and color correction of the endoscope may be used to calculate normalized video.

Description

[Background technology]

[0001] This application claims priority to U.S. Application No. 18 / 370,375, filed September 19, 2023, entitled "Image Enhancement for Endoscope"; U.S. Provisional Application No. 63 / 538,485, filed September 14, 2023, entitled "Endoscope"; U.S. Provisional Application No. 63 / 534,855, filed August 27, 2023, entitled "Endoscope"; U.S. Provisional Application No. 63 / 531,239, filed August 7, 2023, entitled "Endoscope"; and U.S. Provisional Application No. 63 / 437,115, filed January 4, 2023, entitled "Endoscope with Identification and Configuration Information." The priority applications are incorporated by reference.

[0002] This application relates to endoscopes, laparoscopes, arthroscopes, colonoscopes, and similar surgical devices or instruments specially adapted or intended to be used for medical purposes or in surgical procedures on the body, or in preparation for surgical procedures, to evaluate, inspect, measure, monitor, survey, or test the human and animal body, living or dead, along with devices designed to assist in surgery.

[0003] Endoscopes may be arthroscopes (for joint surgery), laparoscopes (for abdominal surgery), colonoscopes (rectum, colon, and lower small intestine), cystoscopes (bladder and urethra), encephaloscopes (brain), hysteroscopes (vagina, cervix, uterus, and fallopian tubes), sinusoscopes (ear, nose, throat), thoracoscopes (chest outside the lungs), tracheoscopes (trachea and bronchi), esophagoscopes (esophagus and stomach), etc. Endoscopes may have a rigid shaft or a flexible insertion tube. Summary of the Invention

[0004] In general, in a first aspect, the invention features a method. A computer reads a machine-readable serial number of a medical device when the medical device is used. Based on reading the serial number, the computer records a decrease in the level of on-hand inventory of a class of medical devices of which the medical device is a component. The computer calculates when the decrease in on-hand inventory of the class indicates a level that warrants a reorder to replenish the on-hand inventory. When warranted, the computer places an order with a supplier of medical devices of the class to replenish the on-hand inventory of medical devices of the class.

[0005] In general, in a second aspect, the invention features a computer system. The system has one or more processors and machine-readable non-transitory memory having instructions stored therein. The instructions are programmed to cause the processor to read a machine-readable serial number of the medical device when the medical device is used and to record a decrease in the level of on-hand inventory of a class of medical devices of which the medical device is a component based on the reading of the serial number. The instructions are programmed to cause the processor to calculate when a decrease in on-hand inventory of the class indicates a level that warrants a reorder to replenish on-hand inventory. The instructions are programmed to cause the processor to place an order with a supplier of medical devices of the class to replenish on-hand inventory of the class of medical devices.

[0006] In general, in a third aspect, the invention features an endoscope. The endoscope has an image sensor designed to collect photons and generate video. The image sensor is of a type that has characteristics that vary from sensor to sensor within manufacturing tolerances. The endoscope has a connector for connecting the endoscope to a computer image processor. The image processor is programmed to retrieve data from a database designed to store characteristics of the endoscope's image sensor and / or image sensor behavior, the database storing characteristics specific to a particular sensor or particular class of sensors. The retrieved data describes the characteristics of the image sensor from the database. The image processor is programmed to calculate a normalized video based on the retrieved characteristics and the video from the image sensor.

[0007] In general, in a fourth aspect, the invention features a method. A computer processor is programmed to receive image data from an image sensor of an endoscope and retrieve the data from a database designed to store characteristics of the image sensor of the endoscope and / or its behavior. The image sensor is of a type with different characteristics for each sensor, and the database stores characteristics specific to a particular sensor or class of sensors. The computer calculates a normalized video based on the retrieved characteristics and the video from the image sensor.

[0008] In general, in a fifth aspect, the invention features a method. A database stores information about a particular individual medical device, including its physical location and / or ownership. An optical reader reads a 2D identification code on the packaging of the medical device. A database record is updated to indicate the current physical location and / or ownership based on the read 2D identification code.

[0009] In general, in a sixth aspect, the invention features a computer system. The system has one or more processors and machine-readable non-transitory memory having instructions stored therein. The instructions are programmed to cause the processor to store information about a particular individual medical device, including its physical location and / or ownership. An optical reader reads a 2D identification code on the packaging of the medical device. A database record is updated to indicate the current physical location and / or ownership based on the read 2D identification code.

[0010] Embodiments may include one or more of the following features: A computer may read a machine-readable serial number of an endoscope when the endoscope is used. Based on the reading of the serial number, an on-hand inventory level for the class of endoscope may be decremented and recorded. A reorder level may be calculated based on the decrement in on-hand inventory level. The computer may calculate when the on-hand inventory level warrants a reorder to replenish the on-hand inventory. Based on the calculated basis for reordering, the computer may place an order with a supplier of endoscopes for the class to replenish the on-hand inventory for the class of endoscopes. The medical device may be an endoscope. The endoscope may have an image sensor. The image sensor may be designed to collect photons to generate video. The image sensor may be of a type that has different characteristics from sensor to sensor within manufacturing tolerances. A program may be programmed to connect one of the medical devices or the endoscope to a computer processor. The processor may be programmed to retrieve characteristics of the connected medical device from a database. The database may be designed to store information about specific individual medical devices, endoscopes, or image sensors. The database may be accessed based on the read serial number. The database may store characteristics of the image sensor of the endoscope. The database may store characteristics of the behavior of the image sensor. The database may store characteristics specific to a particular sensor. The database may provide characteristics of the image sensor of one endoscope from the database for calculating normalized video based on the acquired characteristics and video from the image sensor. The database may store the physical location of individual medical devices, endoscopes, or components. The database may store the physical location and / or ownership of individual medical devices, endoscopes, or components. The optical reader may read an identification code on the packaging of the medical device. The database record of the physical location and / or ownership may be updated to indicate the current physical location and / or ownership based on the read identification code.When a medical device or endoscope can be connected to its control computer, the computer can be programmed to retrieve characteristics of the connected device from a database. The identification code can be a machine-readable 2D optical code. The identification code can be stored in a machine-readable electronic memory. The database can store information about device models of manufactured classes of medical devices compatible with each other. The database can be designed to index to device model information based on the read identification code of an individual medical device. The database can be designed to store data describing the color spectral performance of the image sensor. The database can be designed to store data describing the image plane resolution of the image sensor. The database can be designed to store calibration characteristics of the endoscope's image sensor specific to a particular image sensor of a particular endoscope. The image processor can be programmed to calculate normalized video based on the acquired calibration characteristics and video from the image sensor. The calibration characteristics can describe the white balance of the image sensor, or the color correction gamma curve of the image sensor, or the distortion correction of the image sensor, or any two or more of these characteristics, or any three or more of these characteristics. The computer may calculate a delivery date forecast based on the read 2D identification code. The computer may update a database record of physical location and / or ownership to indicate the current physical location based on reading the read 2D identification code or machine readable memory.

[0011] The above advantages and features are of exemplary embodiments only and are presented solely to aid in understanding the invention. It should be understood that they are not to be construed as limiting the invention as defined by the claims. Additional features and advantages of embodiments of the invention will become apparent in the following description, from the drawings, and from the claims. [Brief explanation of the drawings]

[0012] [Figure 1A] 1 is a perspective view or perspective cutaway view of an endoscope and / or endoscope-related devices. [Figure 1B] FIG. 1 is a block diagram of a computer or processor. [Figure 2] 1 is a perspective view or perspective cutaway view of an endoscope and / or endoscope-related devices. [Figure 3] 1 is a perspective view or perspective cutaway view of an endoscope and / or endoscope-related devices. [Figure 4] 1 is a perspective view or perspective cutaway view of an endoscope and / or endoscope-related devices.

[0013] [Figure 5A] FIG. 1 is a block diagram of a computer or processor. [Figure 5B] FIG. 1 is a block diagram of a computer or processor. [Figure 5C] FIG. 1 is a block diagram of a computer or processor. [Figure 5D] FIG. 1 is a block diagram of a computer or processor.

[0014] [Figure 5E] A time series of video frames. [Figure 5F] FIG. 1 is a block diagram of a computer or processor.

[0015] [Figure 6A] This is a screenshot. [Figure 6B] This is a screenshot. [Figure 6C] This is a screenshot. [Figure 6D] This is a screenshot. [Figure 6E] FIG. 1 is a block diagram of a computer or processor. [Figure 6F] This is a screenshot. [Figure 6G] This is a screenshot. [Figure 6H] FIG. 1 is a block diagram of a computer or processor. DETAILED DESCRIPTION OF THE INVENTION

[0016] This description is organized as follows. I. Overview IA endoscopic surgery IB Overall Architecture IC Integrated Sterilization Package ID Single Use Handpiece II. Partially reusable, partially disposable / replaceable endoscopes and coupling joints therebetween III. Extendable, Bendable, or Articulating Camera Tips IV. Additional Features of Endoscopes V. Endoscope tip Molding and assembly of VA endoscope tip components VB Single-Use Fiber Optic Illumination for Scope Tip Advanced design with VC light guide VD diffusion terminal surface VI. Antifouling and antifogging VI.A.Heating VI.B. Vials of fluid for protecting the coating on the endoscope lens / window for endoscope delivery packages VII. Lens Cap with Optical Correction for Use When Inserting an Offset Field-of-View Endoscope VIII. Avoidance of fasteners, springs, and other small components VIII.A. Buttons with Recessed Springs VIII.B. Overmolding of Cases on Circuit Boards VIII.C. Avoiding Internal Fasteners VIII. Rotational resistance caused by DO ring VIII.E. Ultrasonic welding of the two halves of the outer handle shell VIII.F. Thermoplastic Elastomer Coated Handle IX.Liquid flow IX.A. Liquid-tight seal IX.B. Inducing Spiral Flow X. Molding and joining XA Angled slots for connecting dissimilar materials Joining of XB Obturator component parts Twist lock XC components together XI. Endoscope tip XI.A. Molding and Assembly of Endoscope Tip Components XII. Image Processing Unit XII.A. Image Processing XII.B. HDR Exposure Fusion to Maintain Frame Rate XII.C. Auto Exposure XII.D. Video Processing for Super-Resolution XII.E. Diagnosis and Lesion Detection XII.F. Scope Control XII.G. Flex Boards and Electronics in Endoscope Handles XII.H. Cable XII.I. Wireless Communication as an Alternative to Cables XII.J. Insulation XII.K. Other Peripherals XII.K.1. Monitor XII.K.2.USB Port XII.K.3. Connecting to Cloud Storage XII.K.4. USB Connection for Keyboard and Mouse XII.K.5.Mike XII.K.6.Insufflation tube XIII. Machine-Readable Serial Number XIII.A. Machine-Readable Serial Number XIII.B. Use of machine-readable serial numbers to reduce errors, ensure non-expiration, and ensure sterile single-use XIII.C. Use of Machine-Readable Serial Numbers to Communicate Patient Data to Electronic Health Records XIII.D. Use of Machine-Readable Serial Numbers for Inventory Control, Location Tracking, Reordering, and Stock Management XIII.E. Configuration / Performance Information XIII.F. Calibration Information XIII.G. Implementation XIV. Embodiments I. Overview IA endoscopic surgery

[0017] Referring to FIG. 1A, an endoscope 100 (such as an arthroscope, laparoscope, or other), a trocar 102, and an obturator 104 may be used in joint surgery, joint access, or other minimally invasive surgery.

[0018] Various endoscope tip designs may have the following characteristics: The overall tip may be small enough to match the dimensions of the endoscope, typically the dimensions in the table below. In some cases, the tip may be slightly larger or smaller in diameter than the shaft. The tip may mechanically stabilize and hold within its diameter a camera 410, lighting, fluid infusion or evacuation ports, procedural tools, etc. The tip may be sealed against the high pressures typically used to deflect tissue out of the scope's field of view to prevent the intrusion of bodily tissue and fluids, as well as insufflation fluids. The tip may deliver or enable the delivery of illumination light, either via a tip-mounted LED 418 or by using an optical fiber 430 to transmit light from the handle or controller. Opaque portions of the tip assembly may eliminate stray light from the illumination fiber / LED / light guide, from undesired light paths within the tip, and reflected light from the surgical cavity. The tip may be manufacturable at desired volumes and costs. The tip may have a configuration that is atraumatic to surrounding tissue, e.g., lacking sharp points or edges. The scope may be formed from biocompatible materials, such as stainless steel and / or certain plastics. In some cases, the tip may have a perforating tip. The tip may be designed to prevent fogging or smearing. The tip may preferably allow for cleaning in situ while still in place at the surgical site. IB Overall Architecture

[0019] 1B, endoscope 100 may be part of an overall system designed to deliver high-definition video for use in endoscopic surgery. The system may provide live high-definition video displayed on a video monitor and captured as stored video and still images; illumination of the surgical cavity; irrigation and / or distension (insufflation) of the surgical site; and image refinement such as zoom, rotation, removal or reduction of hot spots and other artifacts.

[0020] The system may include an endoscope including an insufflation tube, communication / control / power / illumination cables, a cannula, and an obturator. The image processing unit (IPU) or master controller may be reusable across multiple procedures. If illumination is provided by fiber optics, there may also be a light box, typically near the IPU, to align the fiber optic fibers with other necessary cords and hoses. One or more of the endoscope, tube, cable, cannula, and obturator may be designed for disposable single use or sold together as an integrated kit.

[0021] Referring to Figures 5A and 5B, the endoscope may have electronics in the handle that control the camera and lighting (LED or fiber optic). The IPU may have a computer processor for various image processing functions and controllers for electromechanical devices within the endoscope, Wi-Fi or similar wireless communications, USB and cloud storage, and the like. Because the scope is single-use, sterility is easily provided. The connecting cables may also be single-use, allowing them to be delivered in sterile packaging. The IPU is more costly and cannot be easily sterilized, so it is outside the sterile field.

[0022] Referring to FIG. 5F, various isolation couplers may provide electrical isolation between the wall voltage components for the IPU and the patient. IC Integrated Sterilization Package

[0023] The endoscope, tubing, and cables may be designed for disposable, single-use use and packaged and sold together as an integrated kit. Additionally, one or more of the obturators and cannulas may be packaged and sold with the kit. The kit may be sold in sterile packaging. The packaging may be designed to be opened at the time of surgery in a sterile field surrounding the patient. The packaging cover may be made from Tyvek® or some similar film that is permeable to ethylene oxide or similar sterilants, so that the packaging and components may be sterilized together at the time of manufacture. The film cover remains in place until immediately before surgery, eliminating the need to disinfect or sterilize the scope immediately before surgery. The tray holding the components may be transparent, allowing the contents of the tray to be viewed before the Tyvek cover is opened.

[0024] Because the components are sold together, they can be calibrated to each other. Various characteristics of the lighting, image sensor, lenses, filters, and the like can be calibrated to each other as a set at the manufacturing plant. White balance can be one of the parameters calibrated at the factory. Because the components are single-use and sold as an integrated package, they can be calibrated to each other at the factory, and their joint calibration continues for the life of the product. In contrast, in traditional endoscopes, the light source and the endoscope are independent, and because the color temperature or balance of the illumination source varies from light source to light source and the color sensitivity of the image sensor pixels varies from scope to scope, white balancing must be performed by the user as part of preparation for each procedure. In configurations in which the scope is sold in a disposable, single-use configuration with an electronic serial number (see IV.A below and

[0088] -

[0095] ) that is traceable back to calibration factors measured at the factory, the scope may be calibrated by imaging a white surface, which provides a test surface with equal red, green, and blue pigments, resulting in intermediate levels of illumination, desaturated pixel values ​​from the image sensor, and a matrix of correction factors may be calculated to color balance the pixels of the image sensor signal. ID Single Use Handpiece

[0025] The endoscope itself may be designed for single-use disposability. The image sensor, lens, filter, and cover window, and illumination emitter (either the LED 418 or the distal end of the optical fiber lighting fiber or waveguide) may be located at the distal end of the insertion shaft. The sensor, lens, filter, cover window, and illumination emitter may be designed to interoperate with each other to allow insertion into the small diameter insertion shaft. Single-use ensures sterility even for components with complex geometries and materials that cannot be autoclaved (such as the endoscope's electronics). The endoscope may have electronic tracking to ensure single-use (see IV.B and

[0096] -

[0104] below).

[0026] Typical dimensions for various surgical specialties may be as follows (measured in millimeters): [Table 1] II. Additional Features of the Endoscope

[0027] 2, the disposable shaft portions 110, 120 may then be separable into an outer cannula 132 for protection and strength, and an inner shaft portion 134 that carries various illumination, optical, and fluid delivery components. Illumination may be provided from an illumination source in the handle, or from illumination in an external controller, by an LED 418 at or near the distal tip, or via optical fiber 430.

[0028] An endoscope may have handles 112, 114, 120 and a shaft 110 for insertion into the body. At or near the distal tip 116 of the shaft 110 may be a lens, electronic image sensor, filter, or other optical component 410. The camera orientation may be fixed within the scope or may be pannable. The camera 410 may be at the tip 116 and point outward from the shaft, or may be recessed a short distance behind the structural tip of the shaft. An illumination source, such as an LED 418, may also be at or near the tip. The tip 116 may have a rigid, pointed trocar tip, or may have a spoon-shaped portion that extends past the distal surface of the window in the tip 116, or may be flexible (like the tip of a colonoscope), in each case extending slightly beyond the distal surface of the window in the tip 116 to provide physical protection for the tip 410 during insertion or to protect the camera 410 from surgical cutting devices.

[0029] The illumination may be visible light, infrared light, and / or ultraviolet light. In some cases, illumination LEDs (light-emitting diodes) or other illumination sources may be located in the reusable handles 112, 114 or in the docking station / controller, the disposable shaft may have an optical fiber 430 for transmitting light to the tip, and the joint 130 may have an optical coupler. In other cases, the illumination LEDs 418 may be located at the tip 116 to directly illuminate the surgical cavity; in such cases, the joint 130 may have a power connector. In some cases, the LEDs 418 may be recessed from the tip, or may be located elsewhere in the shaft, or may be in an external controller, and the optical fiber 430 may carry the illumination light to the tip. The optical fiber 430 may be configured, for example, with a split, so that the light is arranged in a desired pattern around the image sensor to better distribute the light into the surgical cavity around the camera.

[0030] The shaft 110 itself is rigid and may be made from a non-bioreactive metal such as stainless steel or coated aluminum. In some cases, the surgical cavity around the endoscope tip 400 may be insufflated with a gas (typically carbon dioxide) or irrigated with saline. In either case, fluid inflow and outflow may be achieved by channels through the shaft.

[0031] The shaft 110 may also carry power wires to the illumination LEDs 418 and camera 410, and signal wires that carry video signals from the camera 410 back to the electronics in the reusable portions 112, 114 of the handle. Power to the camera 410 may be supplied via conductors in a flexible cable or on a (flexible or rigid) printed circuit board, which may be insulated by a conformal insulating coating such as Parylene. This same flexible circuit board 416 may have signal conductors for the video signal from the image sensor 410. The video signal may be transmitted from the image sensor 410 to the handle using any video signal protocol, for example, MIPI-CSI2 (Mobile Industry Processor Interface-Camera Serial Interface 2) or HDMI®. In some cases, the Parylene coating may improve biocompatibility.

[0032] The shaft 110 may also carry cables or other mechanical elements for controlling the panning of the camera 410 .

[0033] Referring again to Figure 2, the handle may have a rotation collar. The rotation collar may have various features to facilitate rotation. For example, indentations 302 may provide a good grip for the fingers for lighter roll torque. Fins may provide more leverage for greater roll torque and may also provide a fixed rotational reference point.

[0034] The proximal handle 114 may include a rotational sensor so that the angular orientation of the camera 410 may be ascertained. For example, the inner surface of the proximal handle 114 may have one or more magnets 320 mounted thereon, and a printed circuit board 322 (which rotates with the rotating collar 112 and disposable cap 120) may have a Hall Effect sensor 324 that detects the magnets. This may be used to calculate the rotational orientation, which in turn may be used to "right" the image from the camera 410 on the video display screen.

[0035] The distal tip of the shaft, the camera 410 mounted therein, and the mounting of components within the shaft 110 can be designed to be robust. At times during surgery, the tip of the endoscope may come into contact with a shaver, ablation probe, or cauterization probe, and it may be desirable for the tip to be robust against such contact. To reduce the risk of components migrating and remaining within the patient, the disposable shaft and its components can be designed to avoid joints with a high risk of mechanical failure. A disposable optical system can prevent image degradation that occurs when non-disposable optical devices are reused across multiple surgical procedures.

[0036] Endoscopes as a class include arthroscopes, laparoscopes, colonoscopes, and other specialized scopes for various body cavities. In arthroscopes for joint surgery, the shaft may be as small as 6 mm, 5 mm, 4.5 mm, 4 mm, 3.6 mm, 3.3 mm, 3 mm, 2.8 mm, 2.6 mm, 2.4 mm, 2.2 mm, 2 mm, or 1.8 mm and may be rigid. In other endoscopes, such as colonoscopes, the diameter may be larger and the shaft may be flexible.

[0037] The endoscope may be delivered as a handle and multiple tips, each tip individually sealed for sterilization.

[0038] Referring to FIG. 3 , hoses 160, 162 for inletting and outlet irrigation / insufflation fluids / gases and electrical connection cord 164 may be permanently affixed 340, 342 to disposable cap 120. This arrangement may allow hose 162, which carries water out of the surgical cavity and thus becomes contaminated, to be disposable, preventing contact with the reusable portion 114 of the handle. The hoses and cords 160, 162 may be routed by a channel 354 that runs the length of the reusable handle 112, 114. Channel 344 may have a large enough inner diameter to allow easy passage of hoses and cords 160, 162, 164 and connectors 350, 352, and have continuous, smooth walls to allow for easy sterilization and rapid replacement of replaceable components. Channel 354 may be offset from the central axis to allow for placement of printed circuit board 322 on the central axis. The connectors 350, 352 at the ends of the hoses and cords 160, 162 can be small enough to fit through the channel 354. Thus, replacement of the shaft 110, cap 120, and hoses and cords 160, 162 can be accomplished by passing the connectors 350, 352 and hoses and cords 160, 162 through the channel 344. The electrical cord 164 can have a connector 354 at or near the joint 130, and the hose 160 for delivering irrigation / insufflation fluid / gas to the surgical cavity can similarly have a connector at the joint 130 to make the hose reusable or can be permanently affixed 340 to reduce the chance of leaks. Having the hoses and cables 160, 162 approximately on axis reduces undesirable cable flop during use of the scope and reduces undesirable torque on the cap 120. Forming the shaft 120, cap 120, and hoses 160, 162 as an integral unit for replacement reduces the possibility of leaks and improves the sterility of the replacement operation. II.A. Tip Design with Light Guide

[0039] The optical fiber 430 may be extruded into a shape that improves light delivery, such as a rectangle, or may be a U-shaped light guide 450 that would extend the length of the endoscope from the illumination source to the U-shaped emission surface at the distal end of the endoscope. In some cases, the individual fibers 430 may be replaced for at least a portion of their length with a shaped light guide 450, such as a circular or U-shaped ring of clear light guide around the outer edge of the tip chassis 438, 480. The light guide 450 may be a two-part structure with two different refractive indices for internal reflection, similar to an optical fiber. In other cases, the light guide 450 may be formed from a clear, light-transmitting medium coated with a reflective coating, such as aluminum, gold, or silver. In some cases, the shaped light guide 450 may extend only a short distance, for example, the length of the inner tip piece, and a conventional round fiber may be used to bring light from the illumination source to the proximal end of the light guide 450 at the tip chassis 438, 480. III. Image Processing Unit

[0040] Referring to Figures 5A and 5B, the image processing unit (IPU) may use an interface board to drive and receive signals from the scope via a cable and a custom or commercially available motherboard. In some cases, the motherboard may be a commercially available motherboard with an Intel CPU and an NVIDIA GPU. The motherboard provides most of the external interface ports. The patient may be isolated from the line voltage (110 or 120V 60Hz in the US, 240V 50Hz in Europe) by a medical-grade AC / DC power supply and a separate interface board called the patient interface board. The patient interface board processes signals to convert them between the signal form used internally by the IPU and the signal form passed to and from the scope. III.A. Image Processing

[0041] The image processing computer may perform image processing. The GPU provides a well-documented API that can be utilized for graphical processing acceleration, and the software running on the motherboard may then have an internal API that can combine software processing components for image enhancement. A series of video chips in the scope handle and IPU (image processing unit) box may convert very small, high-speed video signals from the sensor (such as a Bayer-formatted MIPI-CSI2 interface) into signals suitable for transmission distances longer than a few centimeters and into protocols (such as YCbCr422 or MPEG) that are more easily processed and stored by the various stages of the imaging pipeline. The IPU processor may receive data (which may be video data, still images, telemetry data, etc.) from the scope via the handle board, cable, and patient interface board. The IPU may capture still images from the video and / or process the video with image correction and enhancement software to deliver high-quality images to a monitor or for storage on some storage medium or in the patient record.

[0042] Various video signal processing chips, image signal processors (ISPs), and graphics processing units (GPUs) may perform numerous video transformations on the video data received from the scope before the data is displayed on a monitor or saved to an output device. The IPU box may have multiple processors, including dedicated image signal processors (ISPs), general-purpose CPUs such as Intel Pentium®, graphics accelerators (GPUs), field programmable gate arrays (FPGAs), custom accelerator hardware, and possibly others. Video transformations may be performed in one or another of these processors, in software, or in some combination of hardware and software. The total amount of processing power may be selected to ensure that image processing can be performed within the image latency requirements. The following transformations may be performed: Receives raw image data from the endoscope's image sensor in a Bayer formatted MIPI-CSI2 stream and re-encodes it into a YCbCr422 or h.264 MPEG stream for improved processability. A video stream processor such as Cypress CX3 translates the MIPI-CSI2 video stream into a UVC-compliant USB 3.0 video stream. HDR or WDR processing (high dynamic range or wide dynamic range)—software (a) for expanding the dynamic range of a captured image by avoiding overexposed or underexposed areas of video. This is achieved by combining sequential overexposed and underexposed image frames from an image sensor, reducing the display intensity of unusually bright pixels to reduce hot spots, and increasing the display intensity of unusually dark pixels within a frame to improve image visibility. See Figure 5E. HDR / WDR processing may use the Mertens exposure fusion algorithm. Rotation and image erection based on a rotation sensor in the handle, including display and rotation of a position indicator that may be displayed as an arrow around the perimeter of a circular mask in the user interface. Correction of distortions (either systematic due to fisheye distortion or similar distortions in the lens specifications, or specific distortions measured on a specific scope to be corrected in the IPU by inverse transformation), removal of artifacts. Cropping a rectangular image received from the scope into a rectangle that can be rotated around a center point on the display. A circular mask is applied over this rectangular crop to provide the user with a circular image display. This may recreate the view surgeons have become accustomed to with rod lens scopes for decades. Also, outside the cone of view provided by the lens, the outer edges of the image may be so distorted or obscured by the edges of the lens housing that they communicate more confusion than information. Auto-exposure to target a desired average image brightness by adjusting exposure time and gain in the image capture pipeline. Demosaicing Black level correction Gain adjustment Color correction Defect correction Noise reduction Tone mapping Color correction and white balance correction -Zoom in / out within the target image Lens resolution correction Local contrast enhancement Edge enhancement Image enlargement (enlarging the circle displayed on the monitor, possibly losing the upper and lower rims of the circular display) Reformatting and compressing video data for storage on a storage device, and decompressing stored video data for display. Controlling transmission over a network connection for storage in the cloud or on a storage device local to the IPU or other non-cloud storage Super-resolution is discussed below in III.D from

[0051] to

[0060] . It involves upsampling from a lower resolution (e.g., 1280x720) to a resolution of 2160x2160 ("4K"). The Frame Writer is the final stage, which puts the video into the video system's frame buffer for display or storage. The fully processed video stream may be displayed on a video monitor or sent to a storage device or network interface.

[0043] Dividing the pipeline into phases enables parallel processing: for example, each phase may be assigned to one core of a multi-core CPU or to a different functional unit of a GPU. III.B. HDR Exposure Fusion to Maintain Frame Rate

[0044] Referring to Figure 5E, HDR exposure fusion may be performed on pairs of frames acquired simultaneously by two different cameras, and then the images are merged in pairs. Exposure fusion algorithms include Mertens-Kautz-Van Reeth or Hugin / Enfuse.

[0045] In other cases, a single image sensor may be programmed to overexpose frame n, then underexpose frame n+1, then overexpose frame n+2, and so on. This may be controlled by blinking the illumination LEDs 418 at the frame rate or by controlling the exposure time of the image sensor. Frames with short exposure times may reveal details in the overexposed portions of the image (“hot spots”), and overexposed frames may reveal details in the underexposed portions of the image (“dark areas”). By merging frames, both hot spots and dark areas are captured in the output image, increasing the dynamic range that can be captured.

[0046] The frames may then be merged in pairs using the same class of HDR exposure fusion algorithms, except applied to overlapping pairs of frames: frame n is merged with frame n+1, then frame n+1 is merged with frame n+2, then frame n+2 is merged with frame n+3, etc. This maintains the output frame rate at the input frame rate. III.C. Automatic Exposure

[0047] An auto-exposure algorithm may be used to adjust for variations in light intensity levels in a scene being captured by an image sensor to a target brightness level. With static gain, exposure, and illumination intensity, if the camera moves closer to an object, the overall scene will be brighter, and therefore the exposure time per frame, gain, and / or illumination intensity should be reduced to capture less light. Conversely, if the camera moves farther away from the object, the overall scene will be darker, and the exposure time, gain, and / or illumination intensity should be increased to capture more light.

[0048] An auto-exposure implementation may control both exposure time and gain to achieve a target intensity set point. The gain control may be either an analog gain at the image sensor pixel cell or a digital gain applied to the image sensor or digital image processing pipeline. The brightness set point may be set by a user "brightness" control or may be set automatically. An auto-exposure algorithm may perform the following steps: 1. Divide the frame into nxn pixel blocks. 2. Calculate the average intensity for each block. 3. The calculated intensity for each block is compared to an intensity set point (which may be set for each block or for the image as a whole) to obtain an error value for each block. Each block may be assigned a weight to scale its calculated error value. This weight allows certain blocks to be more important than others (i.e., blocks in the center of the grid are weighted more heavily than blocks further out). 4. Summing all weighted block errors to get an overall error value. 5. Evaluate the change. If the overall error value is below a defined change threshold, no change is made. b. If the overall error value is above a defined change threshold, scale the change for one update cycle and update the change threshold for the size of the overall error by the following formula: Maximum change threshold = Maximum change threshold + (Total error x multiplier Here the multiplier is less than 1 to allow for a damped response. c. The maximum threshold is set to minimize the user perception of discrete light level changes in similar usage environments, but allow for fast updates when changing quickly from dark to light or light to dark. The multiplier is used to tune this response to achieve the fastest response time to large changes in environmental conditions while preventing light level pulsation perceived by the user. 6. Input the overall error into either the exposure or gain PID control. a. If the scene is too bright: (i) If the gain is at its minimum value, exposure PID control is performed. (ii) Otherwise, gain PID control is performed. b. If the scene is too dark: (i) If the exposure is at its maximum, gain PID control is performed. (ii) Otherwise, exposure PID control is performed. c. Depending on the implementation, any two or more parameters may be substituted for gain and exposure, including illumination intensity, exposure time, etc. 7. Write the resulting exposure and gain to the ISP.

[0049] The auto-exposure algorithm may be a downstream, perhaps immediately subsequent, stage of the WDR algorithm. This reduces the sensitivity of the auto-exposure algorithm to frame-by-frame changes in the exposure time used by the WDR algorithm. The auto-exposure algorithm may be run every few frames (rather than every frame) to reduce processing bandwidth. The intensity calculations per block may be parallelized to run on a GPU.

[0050] The software may provide that many parameters of this algorithm may be tunable via a configuration file loaded as part of system startup, including the number of frames that can be run between recalculations of auto-exposure parameters, the block size in stage 1, the average intensity set point in stage 3, the map of block weights in stage 3, and the PID coefficients for the PID calculation in stage 5. III.D. Video Processing for Super-Resolution

[0051] 5C and 5D, the input to the super-resolution block may be a low-resolution video (e.g., a 720x720 pixel ("720p") or 1280x720 image), and the output may be a 2160x2160 pixel ("4K") image with enhanced quality. The "super-resolution" box may then have the block diagram shown in FIG. 5D. A machine learning model may be used to combine noise reduction, lens resolution correction, edge enhancement, local contrast enhancement, and upscaling as an integrated module. When these functions are performed alone, each is subject to various trade-offs, and image enhancement by one stage may interfere with and degrade the enhancement of another stage. For example, multiple noise reduction algorithms tend to result in blurry images. Traditional edge sharpening tends to amplify noise. By combining all these functions into a single machine learning model, those trade-offs may be reduced.

[0052] Various types of machine learning models can be used with the systems disclosed with respect to Figures 5C and 5D, including fully convolutional neural networks, generative adversarial networks, recurrent generative adversarial networks, or deep convolutional networks. Convolutional neural networks (CNNs) are particularly useful for image processing. A super-resolution CNN block may be formed by combining: A CNN upscaling module from NexOptic Technology Corp., Vancouver, BC, which can improve apparent resolution by allowing the processor to infer pixel-to-pixel interpolation based on local information, as well as previous and next frame information. NexOptic's noise reduction module, which can reduce noise from the image sensor, electronics, and stray photons. NexOptic's Lens Resolution Correction module. This stage can enhance the performance of a lens by understanding the transfer function of a fixed image through the lens. NexOptic's local contrast enhancement module, which can assist surgeons by increasing the contrast between light and dark, between different shades of red, etc. Dynamic range compensation - overexposed and washed-out portions of the image may be balanced against darkened and washed-out portions of the image. The total dynamic range may be adjusted to improve contrast and bring out details lost in overexposed or underexposed portions (see Figure 5E). NexOptic's edge enhancement module, which may reduce loss of resolution (blurring) that may be introduced by the lens system (e.g., due to limitations in lens size or complexity) or by camera motion of objects in the scene, and may assist the surgeon by improving edge extraction to make structures more apparent at the surgical site. High-entropy random noise interferes with data compression. CNNs can be trained to recognize and remove random pixel noise, which can improve data compression.

[0053] By combining all these functions into a single CNN, local contrast, edge enhancement, and noise reduction can all be improved simultaneously. Much like human neural networks are adept at optimizing multiple parameters simultaneously, computer CNNs can be trained to optimize for several properties simultaneously. Hardware contrast and edge enhancement can be disabled. In some cases, degradation and training may involve at least two of the parameters listed above, such as resolution and edge enhancement, or resolution and local contrast. In some cases, any three of these types of image degradation, such as resolution, local contrast, and edge enhancement, or resolution, image sensor noise, and lens correction, may be trained into a model. In some cases, the model may be trained on any four of these parameters. In some cases, it may be trained on all five.

[0054] In one exemplary implementation, an input sequence of low resolution frames

number

[0055] The video super-resolution model may be performed in two stages: a motion estimation and compensation procedure followed by an upsampling process. Alternatively, rather than explicitly calculating and compensating for motion between input frames, motion information may be implicitly utilized to generate a dynamic upsampling filter, and a super-resolution frame may be constructed directly by a local filter on a frame constructed at the center of a calculation window. The machine learning model may be trained by capturing a reference video at normal resolution and then degrading the reference video with transformations that simulate loss of resolution, introduction of noise, lens aberrations, and similar lens noise, degradation of contrast, and / or degradation of edges. The machine learning model may be trained to recover the original reference video at full resolution. The same training may be sufficient to enable upsampling of a video captured at normal resolution to a higher resolution. Lens models are created from a combination of design data and images captured from standard test patterns (e.g., checkerboard or Cartesian line arrays) for the lens design or to detect and measure lens imperfections specific to each scope, and may create generalized transformations or store registration corrections for specific scopes. In some cases, high-quality reference data may be displayed on a physical display and viewed by an endoscopic camera. Machine learning models may be trained to recreate the reference data from camera video. Training may utilize l1 loss with total variation (TV) normalization to reduce visual artifacts.

[0056] A lens correction model may address imperfections in the lens system that remain after balancing all constraints, for example, by creating a lens model, passing a large set of ultra-high resolution images captured with a camera with a very high quality lens through the lens model (to establish a baseline "perfect" image), and then training a CNN to correct the set of images passed through the lens model and transform each image into a "perfect" image.

[0057] The super-resolution CNN may produce better overall image quality (compared to raw data directly from the camera and compared to using all classical blocks independently). Combining a classical enhancement algorithm with an enhancement CNN may provide an opportunity to tune the parameters of the classical algorithm in parallel based on CNN training, when the classical algorithm requires tuning the parameters serially. The super-resolution CNN may enable tunable runtime performance through architectural choices, allowing for a trade-off between overall image quality and speed.

[0058] In some cases, the CNN may retrain itself on the fly. For example, alternating frames may be captured with deliberately underexposed (too dim) lighting and normal lighting during a moment when the camera and image are stationary relative to each other. The CNN may be retrained to recognize hotspots where detail is lost due to overexposure and where detail is lost in dark areas of underexposed frames. In some cases, several machine learning systems may be chained together, such as one to enhance dynamic range, one for blur reduction and edge sharpening, one to recognize frame-by-frame motion, one for contrast enhancement, and one for upsampling for super-resolution.

[0059] In some cases, the bypass feature may disable the super-resolution neural network and instead upsample the image to a resolution of 2160x2160 by conventional means such as bicubic interpolation.

[0060] NexOptic's components may be obtained under the product name Super Resolution, as described in U.S. Patent No. 11,076,103, Gordon, "Photographic Underexposure Correction Using a Neural Network," and U.S. Publication No. 2021 / 0337098A1, Gordon, "Neural Network Supported Camera Image or Video Processing Pipelines," both of which are incorporated by reference. III.E. Diagnosis and Lesion Detection

[0061] In some cases, the image processing pipeline of FIG. 5B may include processing for detecting various lesions. For example, during a colonoscopy, the image processing pipeline may include a processor for detecting polyps. During an esophagoscopy, the image processing pipeline may include a processor for detecting Barrett's esophagus. III.F. Scope Control

[0062] The scope may have several controls including push buttons on the scope, a touch screen on the face of the IPU, and a graphical user interface with a touch screen that can be accessed via the internet from an external computer.

[0063] One push button on the scope may control three things: (a) still frame capture, (b) video recording on / off, and (c) LED adjustment, high beam / low beam. For example, one press may capture the current view as a still frame; two presses may start or stop video recording; and three presses or a three-second press may adjust the LED brightness.

[0064] The IPU may have front panel controls for the scope, including image adjustment, color brightness, zoom, and the like. In either a user visible mode or a system setup / test mode, controls on the IPU's front panel or accessible by a computer via the internet may control the following: LED lighting - The buttons on the scope only provide a single, momentary connection toggle, so they do not provide fine-grained control, only coarse on / off control. An alternative user interface may provide finer lighting control. Sensor Control - Adjust tone or color balance, zoom, etc. Controlling the storage of images and videos in the non-volatile memory of the IPU - which parts of which videos should be stored etc.

[0065] Adjusting the LED brightness requires careful integration with the image sensor. If the brightness is controlled by conventional pulse width modulation (PWM) that is not synchronized with the frame sync of the image sensor, banding may occur in the image. Alternatively, a constant current source or a voltage-controlled current source may be used to adjust the LED brightness and avoid banding. III.G. Flex Boards and Electronics in Endoscope Handles

[0066] A flex circuit board 416 may carry signals and power from the handle to the components at the tip. At the tip, molded plastic sections (braces or chassis 412, 414, 438) may hold all component parts in the proper orientation. The components (image sensor, lens, filter, window, and mount) may be selected to ensure the desired offset angle (typically 0°, 30°, 45°, or 60° on-axis) and the desired field of view (typically 50°, 60°, 70°, 80°, 90°, 100°, 130°, or 180°).

[0067] The distance from the image sensor at the tip to the receiver on the circuit board in the handle may be approximately 115 mm to 330 mm, which is relatively long for a MIPI-CSI2 video connection. The flex circuit board may have a circuit layout and shielding selected to create an impedance-matched signal path for the video data from the video sensor with low radiated emissions, low loss, and low sensitivity to external interference. A connection from the inner insertion shaft to an isolated reference potential on the handle circuit board may protect against interference from RF ablation or coagulation devices by allowing the video signal from the image sensor to float with respect to the applied RF energy, minimizing interference induced on the signal conductors carrying the MIPI-CSI2 signals from the image sensor to the handle board.

[0068] A rigid circuit board (HB-PCBA - "handle board printed circuit board assembly") within the handle may contain a microprocessor, magnetic sensor, and transmitter chip. The transmitter chip may receive a low-power, high-bandwidth, high-speed signal, which may be transported using a MIPI-CSI2 stream from the image sensor received via the flex board, and convert the video signal into a serialized signal suitable for transmission to the IPU over a 3-meter cable. Because 3 meters is a relatively long distance, the cable may have low insertion loss and be carefully impedance-matched to ensure signal integrity. The serialized signal is received by the IPU, converted back to a MIPI-CSI2 interface, and passed to the Image Signal Processor (ISP) for processing. III.H. Cable

[0069] The IPU may be connected to the scope by a custom cable. The cable may be approximately 3 meters (10 feet) long, long enough to allow the surgeon freedom of movement and keep the non-sterile IPU an acceptably long distance from the patient. The connector may be customized to ensure that the scope cannot be connected to other devices that would not provide the necessary patient isolation.

[0070] The cable may use a USB Type A or C connector because this connector has good shielding and physical insertion characteristics, even though the cable in this application does not carry USB signals or utilize the USB protocol. The cable may have a protective hood that extends a few millimeters beyond the end of the USB connector (alternatively, the USB connector may be recessed below the end of the hood). The hood may provide insulation around the connector when the cable is disconnected from the IPU, providing the necessary creepage and clearance distance for patient electrical isolation if, for example, the end of the cable accidentally comes into contact with something electrically live or connected to ground. The hood may be keyed to ensure that it only connects to the correct port on the IPU and cannot (easily) be plugged into a generic USB connector, ensuring only the correct connection of the cable to the connector on the IPU. The cable ends and plugs on the IPU box may be color-coded from each other.

[0071] The cable may provide power to the scope, communicate command signals to the scope, retrieve configuration information stored in the scope's onboard memory, and carry video signals back from the scope to the IPU. The cable may also support a scheme for detecting when the scope is connected to the IPU. This is accomplished by sensing a voltage change on a pin on the scope cable, with the pin being pulled to a logic high voltage when the cable is disconnected and being forced to a logic low when the cable is connected. The pin on the cable may be connected to a pull-up resistor on the IPU side and pulled to GND on the handle board side, so that when a handpiece is connected to the IPU, the handle board pulls the pin down and the processor may detect that the handpiece is connected. III.I. Wireless Communication as an Alternative to Cables

[0072] The cable connection between the IPU and the handpiece may be replaced by wireless transmission, such as Bluetooth, Wi-Fi, or some other wireless protocol. In these cases, the handpiece may have a battery whose capacity can power the handpiece for the full length of the procedure. The wireless connection may provide an alternative architecture for implementing patient electrical isolation, as required by the IEC 60601-1 standard. III.J. Insulation

[0073] 5F, the patient interface board may electrically isolate the motherboard from the patient-facing cables and scope by providing optical connections or transformers to interrupt the copper signal path. Data isolation may be provided between the video stream processor (e.g., Cypress CX3) and the motherboard by a fiber optic cable driven by a USB 3.0 transceiver at each end of the cable, without power conductors, while allowing for the interruption of copper conductors, and communicating via the USB 3.0 communication protocol.

[0074] The physical interface between the scope and IPU may be a USB 3.0 cable consisting of three twisted pairs, a ground conductor, and a power pair of wires, although the physical layer communication is not USB 3.0. The patient interface board may electrically isolate the processing circuitry from the patient-facing cable and scope by providing an optical connection or transformer to interrupt the copper signal path. The isolation mechanism may insulate the patient from the possibility of electrical shock and prevent excessive leakage current.

[0075] The IPU box may include a transformer 1170 that steps down the 120 / 220V AC voltage to a secondary voltage that is used internally to operate the IPU box's processing circuitry 1172, and a second transformer 1180 may isolate the secondary circuitry 1172 from the patient and patient-facing circuits.

[0076] Two safety capacitors 1182 and 1184 may be provided in series across the primary and secondary sides of the isolation transformer. The purpose of capacitors 1182 and 1184 is to create a current shunt for the common-mode currents generated in the isolated switching power supply utilizing transformer 1180. The lower impedance of these capacitors relative to the parasitic capacitance between the patient isolation island 1174 containing the scope and ground may attract most of the common-mode current, reducing the common-mode current traveling between the patient isolation island 1174 containing the scope and ground, thereby reducing radiated emissions. The two capacitors may be surface-mount ceramic capacitors to minimize their impedance at higher frequencies. Capacitor 1186 may be arranged differently across the secondary side of transformer 1180 to create a low impedance across the transformer secondary at high frequencies. This low impedance allows the common-mode current traveling on the positive output of the transformer to pass through capacitor 1186, through capacitors 1182 and 1184, back into the transformer, and to the negative output of the transformer. The two capacitors 1182 and 1184 may be arranged in series and may be safety capacitors that are UL listed to meet the requirements of IEC 60601.

[0077] A second pair of two capacitors in series 1192, 1194 may connect the USB connector shell (the metal shielding jacket on the female side of the USB connector) to two mounting holes leading to the grounded IPU chassis to provide a short return path to ground for common-mode currents injected into the scope and / or scope cable. The capacitor pair 1192, 1194 may be placed symmetrically on each side of the USB connector shell and both may be connected to grounded chassis mounting points (e.g., to the housing 1196 of the IPU 1100) to improve shielding effectiveness against common-mode currents injected into the scope cable.

[0078] The values ​​of capacitors 1182, 1184, 1186, 1188, 1192, 1194 are selected to provide sufficient reduction in common mode current and to meet the leakage requirements of IEC 60601-1.

[0079] A fiber-only optical cable may be utilized to transport high-speed video data from the patient isolation circuit 1174 to the secondary circuit 1172 to meet IEC 60601-1 patient isolation requirements. The fiber optic cable may include a USB 3.0 transceiver at each end of the cable. High-speed video from the scope may be translated from the MIPI-CSI2 protocol used by the image sensor to the USB 3.0 protocol through an integrated circuit. USB 3.0 SuperSpeed ​​RX and TX data pairs may be converted to optical signals transported over the optical cable via the optical transceiver. The optical transceivers at each end of the cable may be locally powered to eliminate the need for electrical power and copper wires to be run over the optical cable, allowing the cable to maintain compliance with IEC 60601-1 isolation requirements.

[0080] The patient interface board may provide the scope interface including the insulation diagram and the required BF type patient isolation per IEC 60601-1. This includes the isolated power supply and isolation of any other interfaces with copper wires that can conduct electricity (such as a USB interface). III.K. Other Peripherals III.K.1. Monitor

[0081] The IPU may drive a video monitor so that the surgeon can have a real-time view of the procedure. III.K.2.USB Port

[0082] A USB port may be provided on the front of the unit for use with a USB flash drive, which may be cabled to the motherboard. Four USB ports may be provided on the rear of the unit for use with a USB mouse and keyboard. Ethernet and Wi-Fi interfaces may be provided from the motherboard for network connectivity to cloud storage (see IV.C and IV.D below,

[0105] -

[0112] ). An analog microphone input may be provided on the rear of the unit, as well as a Bluetooth interface that may be used for annotation during the procedure. A speaker may be provided for the IPU. An AC power plug may provide power for the IPU. The AC power may be controlled by a power switch. III.K.3. Connecting to Cloud Storage

[0083] The IPU and programming may allow video, images, metadata, and other data to be captured and stored. Programs on the IPU may allow for software updates of the IPU. This data may be uploaded or backed up to the cloud, for example, via Wi-Fi, Bluetooth, or a similar wireless connection, or may be stored on an external, removable USB flash drive connected to a USB port. This flash drive may then be used to transfer the data to patient records when needed by the facility, or uploaded to cloud storage from an external PC (see IV.C and IV.D, below,

[0105] -

[0112] ).

[0084] Video may be stored in two-minute increments. If there is a write error, the length of the lost video may be kept to that limit. Stored video and still images may be annotated with date, time, and location metadata, and the serial numbers of the scope and IPU. In the cloud, the serial numbers may be used to connect the video and images to the correct patient's medical record.

[0085] At the end of each surgical day, the day's case data may be stored either on a cloud server or on a USB drive. If the connection to the cloud fails, the USB storage may provide an easily accessible backup. The surgeon may then access the cloud storage or USB data, transfer it to the patient's medical record, and annotate it with the physician's notes. III.K.4. USB Connection for Keyboard and Mouse

[0086] During normal operation, the scope push buttons are the only available user input. A USB keyboard and mouse may be connected to the system to perform system configuration. The keyboard and mouse may allow entry to service or configuration screens. III.K.5.Mike

[0087] The IPU may have a connector for a wired microphone and may allow for the connection of a wireless microphone, which may allow real-time annotation of the video captured by the surgeon. System settings may allow the user to specify whether they want audio enabled and then a microphone connected to either the 3.5mm jack or the Bluetooth interface. IV. Machine-Readable Serial Number IV.A. Machine-Readable Serial Number

[0088] As shipped, each scope may have one or more scope-specific data encoded in machine-readable, scannable, and / or human-readable form. The data may include one or more of the scope's serial number, configuration data, manufacturing calibration data, tracking data, etc. These data may be used for multiple purposes.

[0089] The information may be encoded in a machine-readable code on the packaging for the scope, embedded in the packaging, or embedded in the scope as a scannable code. The scannable code may be a matrix (2D), linear bar, or any form of machine vision code that can be scanned by a smartphone. Examples include QR Code, Code 39, Code 49, Code 93, Code 128, Aztec Code, Han Xin Barcode, Data Matrix Code, JAB Code, MaxiCode, PDF417 Code, SPARQ Code, and any other variant. The scannable code may be an RFID or similar tag that can be scanned by a phone sensor. Scanning may be optical or use any IEEE 802 or related communication protocol, including Bluetooth, RFID (ISO 14443), or NFC (ISO 18092). The scannable code may be encoded on the packaging, the handle of the scope, or the nose cap at the insertion tip of a replaceable scope. Alternatively, it may be stored in the handset's EEPROM memory, connected via SPI (Serial Peripheral Interface), I2C (Inter-Integrated Circuit), USB, or one-wire protocol, and read when the scope is plugged into an image processing unit (IPU). The scope may have a small amount of non-volatile memory that can be read and written by the IPU during initial device manufacturing. The memory may store an electronically readable serial number written to the memory during manufacturing. This memory may also store configuration information for each scope, such as the scope model, serial number, white balance coefficients, lens characteristics correctable in the IPU, focus parameters, etc. This memory may also store usage information, such as a timestamp or time of use determined by the IPU, to prevent reuse of the scope after 24 hours.To ensure tamper resistance, information written to the handle memory may be written under a secure or encrypted protocol used between the IPU and the handle's microprocessor.

[0090] The information may be stored as a single piece of data (essentially a serial number, or some other data that semantically and equivalently uniquely identifies the scope), which may be used as an index key for a database on the server, which in turn contains the complete data for the scope. In some cases, various operating parameters of the scope may be stored in a database on the server, and either the model number or the serial number may be used as a lookup key to retrieve this collection of configuration data and parameters. In other cases, operating parameters may be individually tailored for each individual scope. For example, at the start of arthroscopic surgery on the shoulder, the IPU may verify that the scope being used is indeed an arthroscope of the appropriate diameter, length, and optical performance. The two approaches may be combined, whereby some parameters are stored based on the model number, and others are stored individually for each scope.

[0091] Data stored in on-board memory or in a remotely accessible database may include: A unique serial number or database lookup key Model number and version number (integer or ASCII) A text description of the component's model number / model identifier (typically a 32 character ASCII string) that can be displayed on the control display screen Calibration / normalization data Complete configuration specification - for example: ○ The manufacturer's part number for the image sensor, which allows a number of additional characteristics of the image sensor to be looked up in a table in the IPU, including: ○Image sensor size (rows x columns) Supported frame rates and frame reporting rates for sensors Minimum / maximum integration time and integration time resolution (e.g., 0.1 to 100 ms in 1 ms increments) Built-in illumination source identifiers for scopes - white, infrared, ultraviolet, individual colors, image plane resolution (720x480, 1280x720, etc.) Identifier for what sensor is in the image plane - e.g., 1 bit on / off for each of red, green, blue, ICG infrared, and other colors to be expanded in future software updates Information for establishing white balance, color correction gamma curve, distortion correction coefficients, sensor color sensitivity, lighting color, etc. ○ (Boolean) Provides / does not provide illumination source to handpiece ○ (Boolean) Provide / Do not provide anti-fog heater for handpiece ○ (Boolean) Provide / Do not provide handpiece rotation sensor ○(Boolean) Supports / does not support focus control in handpiece Calibration / normalization data – e.g. ○ Lens focus variation correction data Correction coefficients to compensate for image sensor color sensitivity, lighting color, white balance, and distortion correction ○LED lighting brightness coefficient Identifiers for enabling / disabling specific image enhancement parameters based on hardware image configuration - This may be used to pre-configure image processing settings based on the expected imaging application for this scope. For example, a bit vector may enable or disable optimization of resolution, contrast, smoothing, and other optical properties. · Various size and length characteristics, which can be important for controlling water pressure and the like. ·Manufacture date Date and time of first use Date and time of initial connection and disconnection Length of scope use during procedure

[0092] Storing data in onboard memory (rather than an offboard database) may improve field adaptability. Onboard data storage may reduce the need for software updates to the IPU and may improve robustness when the scope is used in parts of a hospital or facility that do not have reliable internet access. In some cases, particularly where the endoscope uses interchangeable replaceable components (e.g., interchangeable insertion shafts in a reusable handle), each individual replaceable component may have a machine-readable serial number.

[0093] Data stored on the handset may be encrypted and the decryption key may be stored in the IPU. Encryption may improve safety and security by preventing malicious actors from corrupting memory contents or otherwise disrupting the proper operation of the system.

[0094] Data may be communicated either in a fixed-field binary protocol or in a "keyword=" protocol (similar to the JSON protocol for web pages).

[0095] The connector may be a standard connector (e.g., USB-A) or an application-specific connector. The application-specific connector may ensure that incompatible devices are not plugged into each other. The application-specific connector may have additional pins that allow all necessary signals and video to be supported, such as video signals over twisted pair, higher current to power heaters in the handset, and optical connectors for illumination optical fibers. IV.B. Use of machine-readable serial numbers to reduce errors, ensure non-expiration, and ensure sterile single-use

[0096] A machine-readable serial number may be used to improve the efficiency of initiating new procedures. When the scope is about to be used, the serial number may be scanned either as a 2D optical barcode on the enclosed box, on the scope itself, or via remote sensing (e.g., an RFID tag), or it may be read from an EEPROM or similar non-volatile memory built into the scope itself, for example, when the scope is plugged into an IPU. Alternatively, the box or package may have printed information, such as the product model number, lot, manufacturing date, expiration date, and serial number, which allows for redundancy in case the machine-readable information cannot be read.

[0097] 6A and 6B as one example of pairing a scope with a particular patient and procedure, a surgical staff member may request a computer display of a set of names of patients, for example, patients scheduled for consultation within the next few hours. The staff member may click a button associated with the particular patient to be paired with the scope. The computer may respond by turning on the camera and requesting the staff member to present the matrix (2D) barcode on the scope package to the camera. Once the camera verifies the matrix (2D) barcode, the computer may pair the scope associated with that matrix (2D) barcode with the patient and the patient's medical record. Once the human patient, the patient's electronic medical record, and the scope are paired with each other, one or more of the following actions may occur: Patient and scope database records may be linked to each other. The database record for the scope may be marked to indicate that the scope is about to be used. The database may be queried to ensure that the scope has not been previously paired with a different patient (in some cases, the software may provide an emergency override only if the scope is not actually in use with another patient). The computer may impose a number of validation checks and usage constraints: o Based on the machine readable serial number and the database, the computer may check if this scope is assigned to the facility / location where it is being put into use. o Based on the machine readable serial number and the database, the computer may check if the scope used is appropriate for the procedure scheduled for the patient. For example, if the scope is an arthroscope for a joint and the surgeon is a GI doctor or the scheduled procedure is a colonoscopy, the computer may flag a warning to the human user and ask for confirmation. o The computer may check that all expiration dates have been met and that the scope is not subject to a recall or other product action. The scope's database record may be updated to indicate the storage location where the video, still images, or audio recordings are stored in association with the patient's medical record.

[0098] In another example, the computer may first read a 2D barcode or other machine-readable optical code and, based on that code, present a list of patients scheduled for treatment using that form of scope, or may request a patient identifier such as name, patient record number, or the like.

[0099] When a scope is plugged into an IPU for use, the steps and checks listed above may be performed or repeated. If the scope has not previously been paired with a patient, a check may be performed for the first time and the computer may prompt the human user to provide patient identification information. Additionally, in the database record of the scope, the initial connection to the IPU may cause the following updates to be made to the database of scope data: A flag may be set to indicate that the scope has been removed from its packaging and is no longer sterile. If the scope is sold for single use, the initiation of use may be recorded in the manufacturer's database and / or in the scope's own memory to ensure sterility and reliability. That single use may be recorded as a single binary flag that, when set, prohibits further use. The first use may be marked with a timestamp and / or location, which prevents the scope from being reused for some period of time (e.g., 2 or 4 hours). This allows the scope to be plugged in multiple times during a single procedure (e.g., to untangle cables or to reset after a power failure), but still be sufficient to prevent reuse.

[0100] When a procedure is initiated or a scope is plugged into the IPU, the IPU may execute a dialog to verify that the scope and procedure are appropriate for each other. For example, the IPU may query the patient's electronic medical record to verify the procedure to be performed and verify that the attached scope is appropriate for that procedure. If an incompatibility is detected, the IPU may issue an alert and request confirmation and override. The serial number of the exact scope used may be stored in the medical record for audit issues.

[0101] During a procedure, the scope or IPU may encounter an error or malfunction. In such an event, the IPU or scope may communicate with a computer or database to initiate a complaint or feedback process with the manufacturer, hospital, or appropriate reporting organization. Alternatively, the software may provide the user with the ability to initiate a feedback event. Error codes may be stored in the medical record for audit purposes.

[0102] The stored data may allow a single IPU to be used with multiple scope configurations, reducing the complexity of purchasing, supplying, and using different scopes for different purposes.

[0103] When a scope is refurbished, its flag may be cleared and it may be reused.

[0104] The machine-readable serial number may provide an index into the manufacturer's document library, providing near-instant access to information if needed during a surgical procedure. IV.C. Use of Machine-Readable Serial Numbers to Communicate Patient Data to Electronic Medical Records

[0105] Referring to FIG. 6C , pairing a scope with a patient may improve the efficiency of incorporating information into the patient's electronic medical record. During the procedure, the surgeon or assistant may mark the entire video or a marked portion, which may be permanently stored in the patient's electronic medical record or imported into a separate database maintained by the hospital / client or the scope manufacturer. In some cases, the IPU may compute a speech-to-text transcription of the physician's narration during the procedure. The IPU may connect to a cloud application via Wi-Fi or Ethernet. Images and videos may be sent to this cloud application in real time after each procedure or may be stored on a USB memory stick. Images and videos may be sent to the cloud application as a live stream or collected in the IPU's storage for periodic uploading, such as at the end of the day. Images and videos may be stored with the patient's electronic medical record, either in raw form or after some selection and editing. They may be annotated with text or dictation by the physician or assistant. Video or images of the procedure may be automatically routed to storage associated with the specific patient (e.g., the patient's electronic medical record or a patient portal hosted by a database separate from the hospital or surgeon's electronic medical record) based on the pairing established by the machine-readable serial number.

[0106] This video may be anonymized, edited, and / or delivered to the patient, possibly with a voice-over dictation, as described in patent application Ser. No. 16 / 278,112, filed Feb. 17, 2019, which is incorporated by reference. This video may improve the patient's post-operative rehabilitation and may provide a patient-specific report. IV.D. Use of Machine-Readable Serial Numbers for Inventory Control, Location Tracking, Reordering, and Stock Management

[0107] Referring to Figures 6D and 6E, the purchaser / hospital may interact with a database to set minimum inventory levels. Alternatively, a computer system accessible to the manufacturer may calculate reorder stock levels by checking average usage rates, time of delivery based on location, and any pending or in-transit inventory. As each scope is used, one or more computers may decrement the existing stock level, and if the decremented level indicates a reorder compared to the reorder stock level, the computer may automatically enter a reorder to maintain stock at the appropriate level. The computer may integrate with a distribution center or third-party connection via an API or similar EDI (electronic data interchange) to order the reordered quantity. A given user ID may have one or more billing locations and one or more delivery addresses.

[0108] The scope packaging may be scanned as needed, typically when the scope arrives at the hospital / purchaser site, allowing inventory to be registered and moved internally from one location to another (e.g., a storeroom on a different floor or wing). Additionally, the system may use tracking information from UPS or FedEx or another shipper / logistics manager to determine the location of inventory in transit from the manufacturer through the distribution network to the final hospital / purchaser. The system may use tracking proof of delivery as a signal that the product has been received by the customer site.

[0109] The system may issue an alert if it detects that a scope appears to be lost. For example, the system may calculate a typical inventory time for a given location (e.g., perhaps two weeks) and recognize if a scope has not been scanned or moved for several times that time. Similarly, the system may alert about unexpected inventory movements. The system may be programmed to filter out false positives and over-reports. For example, a scope may take an unexpected route due to movement to a shipping hub or through a hospital's internal distribution system, which should be suppressed to avoid over-reporting.

[0110] Referring to Figures 6F and 6G, the system may provide a dashboard to provide real-time data to surgeons, facility managers and staff, and the supply chain, providing instant information about inventory levels and feeding into the supply chain. For example, the dashboard may provide orders that have been placed but not yet received, shipment tracking information, delivery estimates, current on-hand inventory levels, and recent usage patterns. Figure 6F shows an inventory list of the current inventory of scopes on hand for immediate use, including model name, model description, lot number (typically useful in the event of a product recall), serial number, and expiration date. Figure 6G shows recently used scopes, including their serial number, where used, date and time, and by whom. This information may be presented at an aggregate or statistical level, or as a list of individual scopes and their current status.

[0111] This tracking may improve utilization and inventory management by ensuring "just-in-time" ordering.

[0112] The database may store information tracking the history of the scope. If the serial number is remotely scannable (e.g., RFID tag), the location of the scope may be tracked through the distribution channel and storage at the purchaser's hospital. This information may be used to ensure the scope has not exceeded any time limits, has not been stored in a location known to exceed temperature limits, etc. For example, a 24-hour limit after first use may be enforced by the IPU by reading the time of first use from non-volatile memory on the handle board PCBA. When a procedure is initiated, the IPU may query via the internet to verify that the scope has not exceeded the manufacturer's date and that it is within specifications and not subject to any safety recalls. IV.E. Configuration / Performance Information

[0113] Each machine-readable serial number may be associated with information describing specific configuration and / or performance information about the scope. The serial number may be used to trace back records of how the device was constructed and the components used in its construction. In some cases, reusable scope handles may be designed to be used with interchangeable parts, and each interchangeable or replacement component may have a machine-readable serial number, which may then be associated with a database record with a specific model number, which in turn may have a list of specific configuration and / or performance information. This information can be checked when a scope is paired with a patient or similar occasions where the specific performance of a particular scope or component should be compared to the patient's specific needs. The serial number may be used to associate with the patient and to track inventory. IV.F. Calibration Information

[0114] During manufacturing, certain measurements and calibration data may be measured, such as white balance, general level settings, color correction gamma curves, coefficients for distortion correction, sensor color sensitivity, illumination color, any anomalies in focus in different parts of the field of view, and the like. This data may be stored in a database indexed by the scope's serial number so that it may be available to the image processing pipeline for compensation. IV.G. Implementation

[0115] A database for implementing the above features may include tables that track information such as: [Table 3]

[0116] The database may have tables that describe scopes: one table may describe the characteristics of a model of a scope common to product IDs of a class of scopes; a second table may describe individual scopes. [Table 4] [Table 5]

[0117] The database may have tables that describe treatments. One table may describe characteristics of classes of treatments. A second table may describe individual treatments for individual patients. [Table 6] [Table 7]

[0118] The database may include tables that describe orders and inventory status. [Table 8] [Table 9]

[0119] The database may include tables that describe patients. [Table 10]

[0120] The database may have tables that establish relationships between scopes, procedures, patients, surgeons, inventory status, and the like.

[0121] Other tables may represent surgeons, surgeon specialties, surgeon offices, surgeon staff members, system administrators, addresses, passwords, and similar security information; links to linked videos, audio dictations, and images to be placed in the patient's medical record; procedure checklists, access permissions to allow patients to view only the patient information to which they are entitled, and the like.

[0122] Referring to FIG. 6H, the system architecture may be organized around a client-server principle. Users may interact with client software on their device, e.g., a browser HTML or downloaded app on either their phone or desktop computer. On the server side, the front end may be provided by commercially available components, e.g., Amazon AWS, an ALB (application load balancer), and / or a cloudfront content delivery network. The application server may include components for patient management, inventory management, user identity management, authorization, authentication, and authorization; facility management; procedure management; order management; scope management; scope and patient-procedure pairing; and other administrative functions. A relational / SQL database may provide storage and lookup. V. Embodiment

[0123] Embodiments of the invention may include any one or more of the following features, alone or in any combination.

[0124] The endoscope 100 may include a handle and an insertion shaft, the insertion shaft having a camera at its distal end. The insertion shaft may have solid-state lighting and imaging circuitry at or near its tip designed to provide illumination and imaging of the inside of a body cavity for a surgeon during surgery. The proximal portion of the handle may include electronics for driving the lighting circuitry and for receiving imaging signals from the imaging circuitry. The proximal handle portion may be designed to allow for sterilization between uses. The joint between the proximal handle portion and the insertion shaft may be designed to detachably connect the insertion shaft to the proximal handle portion. When detached, the joint may allow for removal of the insertion shaft for disposal and replacement. When connected, the joint may be designed to transfer mechanical force from the surgeon's hand to the insertion shaft and provide an electrical connection between the proximal handle circuitry and the lighting and imaging circuitry. The handle may have a proximal portion and a distal portion. The distal portion may be located between the insertion shaft and the proximal handle portion. The insertion shaft may be rigidly secured to the distal handle portion. A joint may be arranged to connect and disconnect the distal and proximal portions of the handle. The distal handle portion may be designed to indirectly transmit mechanical force between the surgeon's hand and the insertion shaft and to provide an indirect electrical connection between the proximal handle circuitry and the illumination and imaging circuitry. The handle may have a rotation collar with surface features designed to assist the surgeon in rotating the insertion shaft in a roll dimension about the axis of the insertion shaft relative to the proximal handle portion. Electronics within the proximal handle portion may be designed to sense the roll of the insertion shaft and provide an angular rotation signal that may be designed to allow righting of the displayed image received from the imaging circuitry. The attachment to the image sensor may be designed to allow panning of the image sensor about a pitch or yaw axis perpendicular to the central axis of the insertion shaft. One or more ultraviolet LEDs inside the endoscope may be designed to sterilize the area inside the endoscope. A hose for insufflation fluid or gas may be designed to be placed on or near the central axis of the proximal handle portion.Two or more insertion shafts, each with different dimensions than the others, can be connected to the proximal handle portion at a joint, allowing the proximal handle to be used in procedures with different insertion shaft requirements. A sterilization cabinet can be designed to sterilize endoscope components. The insertion shaft at the endoscope tip has rigid proximal and distal portions. The distal portion is bendable to direct the imaging circuit's field of view in a desired direction. An illumination device and solid-state imaging circuit are located at or near the distal tip of the articulating distal portion. The illumination device is designed to illuminate the interior of the body cavity for the surgeon during surgery, and the imaging circuit is designed to capture images thereof. The interchangeable endoscope tip coupling is designed to detachably connect the insertion shaft to the handle portion at a joint and to disconnect the joint. The coupling has a mechanical connector. When the joint is disconnected, the mechanical connector allows the insertion shaft to be removed from the handle for disposal and replacement. When the joint is connected, the joint is designed to provide mechanical force transmission between the surgeon's hand and the insertion shaft. An electrical connector is designed to connect the insertion shaft to electronics within the handle. The handle electronics are designed to drive the illumination device and receive imaging signals from the imaging circuit, and the handle is designed to allow sterilization between uses. A control force transmission element is designed to allow the surgeon to orient the imaging circuit by transmitting mechanical force directed by the surgeon to the articulating distal section. The distal bendable section includes a series of articulating rigid segments. A sheath or cover covers the articulating rigid segments designed to reduce penetration or pinching. The distal bendable section is formed of a solid component, is bendable in its lateral and elevation dimensions, and is relatively incompressible in compression in its longitudinal dimension. The distal bendable section is extendable from and retractable into a solid sheath. The distal bendable section is bendable in one dimension.The bendable distal section is bendable in two orthogonal dimensions. The imaging circuit is mounted within or near the distal tip of the articulating distal section via a pannable mount. The pannable mount is designed as two sides of a parallelogram. The imaging circuit is mounted on structural segments hinged to the two sides of the parallelogram. Passages and openings are designed to allow irrigation fluid to pass through to improve the field of view through a lens or window on the imaging circuit. Passages and openings are designed to allow inflation fluid to pass through to enlarge a cavity for surgery. The mechanical connector of the coupling includes a twist lock designed to secure the insertion shaft of the endoscope to the handle portion. Multiple endoscope tips are bundled and packaged with a handle. The handle has electronics designed to drive the illumination device and receive imaging signals from the imaging device. The multiple tips and handle are packaged for shipping and sale together. The illumination device is an LED mounted at or near the distal tip. The illumination device is the output end of a fiber optic cable driven by an illumination source within the handle. The camera 410 may be enclosed within a plastic casing. The plastic casing may be formed as an overmolded jacket designed to protect the camera 410 from bodily fluids and to structurally hold the distal components in an operational configuration. The overmolded jacket may be designed to hold a transparent window in an operational configuration with the camera 410. The overmolded component may be formed from clear plastic. The overmolded component may be designed to function as a lens for the image sensor 410. The image sensor 410 may be mounted on a flexible circuit board. An illumination LED 418 may be mounted on the flexible circuit board 416. The LED 418 and image sensor 410 may be mounted on opposite sides of the flexible circuit board 416. The image sensor 410 may be protected behind a transparent window.The window may be molded with two thicknesses: a thinner portion designed to allow mounting and illumination light to pass through, and a thicker portion over the camera 410. The handle may include a circuit board with circuitry for controlling and receiving signals from the camera 410. The handle and its components may be designed without metal fasteners or adhesives, except for those captured by the overmolding. The endoscope's control buttons may be molded with protrusions that function as return springs. The protrusions may be attached to the endoscope handle via melting. The circuit board may be overmolded with plastic that seals the circuit board from contact with water. The circuit board may be attached to the handle via melting. The handle's components may be joined together into a unitary structure via melting. The handle's components may be joined by a resilient clip designed to hold the two components together before being joined together via melting. The handle may be formed from two concentric shells. Rotation of the two shells relative to each other may be controlled via one or more O-rings that frictionally engage with each of the two shells. The handle may have a high-friction elastomeric overmolded layer. The insertion shaft may be connected to the handle via a separable joint. The separable joint's water joint may be molded for an interference seal without an O-ring. The separable joint's water cavity may be designed to impart swirl to water flowing from the handle to the insertion shaft. The insertion shaft may be formed from stainless steel and connected to the handle via a separable joint. The plastic components of the endoscope may be bonded to the insertion shaft via a plastic overmolding into aligned slots at an oblique angle in the wall of the insertion shaft without adhesive. The water joint may be formed as two cones in an interference fit. The cones may be interlocked at a large diameter. The cones may be interlocked via a raised ridge on the edge of the inner male cone. The obturator 104 may be designed to pierce tissue for introduction of the endoscope. Features for twist-locking the obturator 104 into the trocar 102 may be compatible with features for twist-locking the endoscope into the trocar.

[0125] The device may include a computer processor and memory. The processor is programmed to receive video image data from an image sensor at the distal end of the endoscope and display the image data in real time to a surgeon. The processor is programmed to process the image data received from the image sensor with a machine learning model that is trained to simultaneously upsample the image data to a higher resolution than that captured by the image sensor, sharpen edges, and enhance local contrast.

[0126] The device may include a computer processor and memory. The processor is programmed to receive video image data from an image sensor at the distal end of the endoscope and display the image data to the surgeon in real time. The video image data has a frame rate at which the image data is generated by the image sensor. The processor is programmed to control the image sensor and / or an illumination source designed to illuminate the scene viewed by the image sensor, the control being programmed to underexpose or overexpose every other frame of the video image data. The processor is programmed to process the image data received from the image sensor to combine consecutive pairs of frames of image data, adjust the dynamic range, enhance overly bright or overly dark portions of the image, expose details, and generate combined frames at the full frame rate of the video as generated by the image sensor.

[0127] The device may include a computer processor and memory. The device may include a computer processor and memory. The processor is programmed to receive video image data from an image sensor at the distal end of the endoscope and display the image data to a surgeon in real time. The processor is programmed to sum an error in the intensity of the image relative to a setpoint intensity. The processor is programmed to simultaneously control at least two of gain, exposure, and illumination with a PID control algorithm to achieve image display at the setpoint intensity, and the maximum change per step of the PID control is damped to prevent pulsation.

[0128] Embodiments may include one or more of the following features, alone or in any combination: The processor may be further programmed to control the image sensor and / or an illumination source designed to illuminate a scene viewed by the image sensor. The control may be programmed to under- or over-expose every other frame of video image data. The processor may be further programmed to process image data received from the image sensor to combine consecutive pairs of frames of image data, adjust the dynamic range, enhance overly bright or overly dark portions of the image, and expose details. The processor may be further programmed to generate the combined frames at the full frame rate of the video as generated by the image sensor. The processor may be further programmed to sum an error in the intensity of the image relative to a setpoint intensity. The processor may be further programmed to simultaneously control at least two of the gain, exposure, and illumination with a PID control algorithm to achieve an image display at the setpoint intensity. The maximum change per step of the PID control may be attenuated to prevent pulsation. The processor may be further programmed to process image data received from the image sensor with a machine learning model trained to simultaneously upsample the image data to a higher resolution than that captured by the image sensor, sharpen edges, and enhance local contrast. The processor may be further programmed to enhance the video image data with dynamic range compensation. The processor may be further programmed to adjust exposure time, illumination intensity, and / or gain during image capture to adjust for exposure saturation. The processor may be further programmed to enhance the video image data with noise reduction. The processor may be further programmed to enhance the video image data with lens correction. The processor may be further programmed to enhance at least two of dynamic range compensation, noise reduction, and lens correction in addition to resolution. The processor may be further programmed to rotate the image display to compensate for endoscope rotation.The processor may be further programmed to adjust exposure time, illumination intensity, and / or gain in image capture to adjust exposure saturation.

[0129] A computer reads a machine-readable serial number of a medical device when the medical device is used. Based on the reading of the serial number, the computer records a decrease in the level of on-hand inventory of a class of medical devices of which the medical device is a component. The computer calculates when a decrease in on-hand inventory of the class indicates a level that warrants a reorder to replenish the on-hand inventory. When warranted, the computer places an order with a supplier of medical devices of the class to replenish the on-hand inventory of medical devices of the class.

[0130] The one or more processors and machine-readable non-transitory memory of the computer system have instructions stored therein. The instructions are programmed to cause the processor to read a machine-readable serial number of the medical device when the medical device is used and to record a decrease in the level of on-hand inventory of a class of medical devices of which the medical device is a component based on reading the serial number. The instructions are programmed to cause the processor to calculate when a decrease in on-hand inventory of the class indicates a level that warrants a reorder to replenish on-hand inventory. The instructions are programmed to cause the processor to place an order with a supplier of medical devices of the class to replenish on-hand inventory of the class of medical devices.

[0131] The endoscope has an image sensor designed to collect photons and generate video. The image sensor is of a type that has characteristics that vary from sensor to sensor within manufacturing tolerances. The endoscope has a connector for connecting the endoscope to a computer image processor. The image processor is programmed to retrieve data from a database designed to store characteristics of the endoscope's image sensor and / or image sensor behavior, the database storing characteristics specific to a particular sensor or particular class of sensors. The retrieved data describes the characteristics of the image sensor from the database. The image processor is programmed to calculate a normalized video based on the retrieved characteristics and the video from the image sensor.

[0132] The computer processor is programmed to receive image data from an image sensor of the endoscope and retrieve data from a database designed to store characteristics of the image sensor of the endoscope and / or characteristics of the image sensor's behavior. The image sensor may be of a type with different characteristics for each sensor, and the database stores characteristics specific to a particular sensor or class of sensors. The computer calculates a normalized video based on the retrieved characteristics and the video from the image sensor.

[0133] The database stores information about specific individual medical devices, including their physical location and / or ownership. An optical reader reads the 2D identification code on the packaging of the medical device. The database record is updated to indicate the current physical location and / or ownership based on the read 2D identification code.

[0134] The computer system has one or more processors and machine-readable non-transitory memory having instructions stored therein. The instructions are programmed to cause the processor to store information about a particular individual medical device, including its physical location and / or ownership. An optical reader reads a 2D identification code on the packaging of the medical device. A database record is updated to indicate the current physical location and / or ownership based on the read 2D identification code.

[0135] The computer may read the machine-readable serial number of the endoscope when the endoscope is used. Based on the reading of the serial number, the level of on-hand inventory for that class of endoscope may be decremented and recorded. Based on the decrement in the on-hand inventory level, a reorder level may be calculated. The computer may calculate when the level of on-hand inventory warrants a reorder to replenish the on-hand inventory. Based on the calculated basis for reordering, the computer may place an order with a supplier of endoscopes for the class to replenish the on-hand inventory for the class of endoscopes. The medical device may be an endoscope. The endoscope may have an image sensor. The image sensor may be designed to collect photons to generate video. The image sensor may be of a type that has different characteristics from sensor to sensor within manufacturing tolerances. The program may be programmed to connect one of the medical devices or the endoscope to a computer processor. The processor may be programmed to retrieve characteristics of the connected medical device from a database. The database may be designed to store information about specific individual medical devices, endoscopes, or image sensors. The database may be accessed based on the read serial number. The database may store characteristics of the image sensor of the endoscope. The database may store characteristics of the image sensor's behavior. The database may store characteristics specific to a particular sensor. The database may provide characteristics of an endoscope's image sensor from the database for calculating normalized video based on the acquired characteristics and video from the image sensor. The database may store the physical location of individual medical devices, endoscopes, or components. The database may store the physical location and / or ownership of individual medical devices, endoscopes, or components. The optical reader may read an identification code on the packaging of the medical device. The database record of the physical location and / or ownership may be updated to indicate the current physical location and / or ownership based on the read identification code. When a medical device or endoscope can be connected to its control computer, the computer can be programmed to retrieve characteristics of the connected device from the database.The identification code may be a machine-readable 2D optical code. The identification code may be stored in a machine-readable electronic memory. A database may interchangeably store information regarding device models for a class of manufactured medical devices. The database may be designed to index to device model information based on the read identification code of an individual medical device. The database may be designed to store data describing the color spectral performance of the image sensor. The database may be designed to store data describing the image plane resolution of the image sensor. The database may be designed to store calibration characteristics of the endoscope's image sensor specific to a particular image sensor of a particular endoscope. The image processor may be programmed to calculate normalized video based on the acquired calibration characteristics and video from the image sensor. The calibration characteristics may describe the white balance of the image sensor, or the color correction gamma curve of the image sensor, or the distortion correction of the image sensor, or any two or more of these characteristics, or any three or more of these characteristics. The computer may calculate a delivery date forecast based on the read 2D identification code. The computer may update the database record of physical location and / or ownership to indicate the current physical location based on the read 2D identification code or machine readable memory read.

[0136] The various processes described herein may be implemented by appropriately programmed general-purpose computers, special-purpose computers, and computing devices. Typically, a processor (e.g., one or more microprocessors, one or more microcontrollers, one or more digital signal processors) receives instructions (e.g., from a memory or similar device) and executes those instructions to perform one or more processes defined by those instructions. The instructions may be embodied in one or more computer programs, one or more scripts, or other forms. The processes may be executed by one or more microprocessor processes, central processing units (CPUs), computing devices, microcontrollers, digital signal processors, graphics processing units (GPUs), field programmable gate arrays (FPGAs), or similar devices, or any combination thereof. The programs implementing the processes and the data operated on may be stored and transmitted using various media. In some cases, hardwired circuitry or custom hardware may be used in place of, or in combination with, some or all of the software instructions that may implement a process. Algorithms other than those described may be used.

[0137] The programs and data may be stored on a variety of media suitable for the purpose, or on a combination of dissimilar media that can be read and / or written by a computer, processor, or similar device. Media may include non-volatile media, volatile media, optical or magnetic media, dynamic random access memory (DRAM), static RAM, floppy disks, flexible disks, hard disks, magnetic tape, any other magnetic media, CD-ROMs, DVDs, any other optical media, punch cards, paper tape, any other physical media with a pattern of holes, RAM, PROM, EPROM, FLASH-EEPROM, other non-volatile memory, any other memory chip or cartridge, or other memory technology.

[0138] The databases may be implemented using a database management system or an ad-hoc memory organization scheme. Alternative database structures to those described may be readily adopted. Databases may be stored locally or remotely from devices that access data in such databases.

[0139] In some cases, processing may be performed in a network environment including a computer in communication with one or more devices (e.g., via a communications network). The computer may communicate directly or indirectly with the devices via any wired or wireless medium (e.g., the Internet, a LAN, a WAN, or Ethernet, token ring, telephone lines, cable lines, radio channels, optical communications lines, commercial online service providers, bulletin board systems, satellite communications links, or any combination of the above). Transmission media include coaxial cable, copper wire, and optical fiber 430, including the wires with a system bus coupled to the processor. Transmission may occur over the transmission medium or via electromagnetic waves, e.g., infrared, Wi-Fi, Bluetooth, and the like, at various frequencies using various protocols. Each of the devices may itself comprise a computer or other computing device, such as one based on an Intel® Pentium® or Centrino® processor, adapted to communicate with the computer. Any number and type of devices may communicate with the computer.

[0140] A server computer or centralized authority may or may not be necessary or desirable. In various cases, the network may or may not include a central authority device. Various processing functions may be performed on a central authority server, one of several distributed servers, or other distributed devices.

[0141] The following applications are incorporated by reference: U.S. Provisional Application No. 63 / 544,608, filed October 17, 2023, entitled "Artificial Intelligence Agent for Preparing Physician's Note"; U.S. Application No. 18 / 370,375, filed September 19, 2023, entitled "Image Enhancement for Endoscope"; U.S. Provisional Application No. 63 / 538,485, filed September 14, 2023, entitled "Endoscope"; U.S. Provisional Application No. 63 / 534,855, filed August 27, 2023, entitled "Endoscope"; U.S. Provisional Application No. 63 / 531,239, filed August 7, 2023, entitled "Endoscope"; U.S. Provisional Application No. 63 / 437,115, filed January 4, 2023, entitled "Endoscope with Identification and Configuration "Illumination for Endoscope"; U.S. Application No. 17 / 954,893, filed September 28, 2022, entitled "Illumination for Endoscope"; U.S. Provisional Application No. 63 / 376,432, filed September 20, 2022, entitled "Super Resolution for Endoscope" No. 17 / 896,770, filed August 26, 2022, entitled "Endoscope"; U.S. Provisional Application No. 63 / 400,961, filed August 25, 2022, entitled "Endoscope"; U.S. Provisional Application No. 17 / 824,857, filed May 25, 2022, entitled "Endoscope"; U.S. Provisional Application No. 63 / 249,479, filed September 28, 2021, entitled "Endoscope"; U.S. Provisional Application No. 63 / 237,906, filed August 27, 2021, entitled "Endoscope"; U.S. Application No. 17 / 361,711, filed June 29, 2021, entitled "Endoscope with Bendable Camera Shaft"; U.S. Provisional Application No. 63 / 214,296, filed June 24, 2021, entitled "Endoscope with Bendable Camera Shaft"; U.S. Provisional Application No. 63 / 193,387, entitled "Anti-adhesive Window or Lens for Endoscope Tip";U.S. Provisional Application No. 63 / 067,781, filed August 19, 2020, entitled "Endoscope with Articulated Camera Shaft"; U.S. Provisional Application No. 63 / 047,588, filed July 2, 2020, entitled "Endoscope with Articulated Camera Shaft"; U.S. Provisional Application No. 63 / 046,665, filed June 30, 2020, entitled "Endoscope with Articulated Camera Shaft"; U.S. Provisional Application No. 16 / 434,766, filed June 7, 2019, entitled "Endoscope with Disposable Camera Shaft and Reusable Handle"; U.S. Provisional Application No. 62 / 850,326, filed May 20, 2019, entitled "Endoscope with Disposable Camera Shaft"; U.S. Provisional Application No. 16 / 069,220, filed October 24, 2018, entitled "Anti-Fouling "Endoscopes and Uses Thereof"; U.S. Provisional Application No. 62 / 722,150, filed August 23, 2018, entitled "Endoscope with Disposable Camera Shaft"; U.S. Provisional Application No. 62 / 682,585, filed June 8, 2018, entitled "Endoscope with Disposable Camera Shaft";

[0142] For clarity of explanation, the above description focuses on a representative sample of all possible embodiments, i.e., samples that teach the principles of the present invention and convey the best mode contemplated for carrying them out. The present invention is not limited to the described embodiments. Well-known features may not have been described in detail to avoid unnecessarily obscuring the principles related to the claimed invention. Throughout this application and its associated filing history, the term "invention," when used, refers to the entire collection of concepts and principles described. In contrast, a formal definition of exclusively protected property rights is set forth in the exclusively controlled claims. This specification does not attempt to exhaustively list all possible variations. Other undescribed variations or modifications may be possible. Where multiple alternative embodiments are described, in many cases, elements of different embodiments can be combined, or elements of the embodiments described herein can be combined with other modifications or variations not expressly described. The listing of items does not imply that any or all items are mutually exclusive, nor that any or all items are inclusive of any category, unless expressly specified otherwise. In many cases, a feature or group of features may be used separately from the overall apparatus or method described. Many of these undescribed alternatives, variations, modifications, and equivalents are within the literal scope of the following claims, and others are equivalents. The claims may be practiced without some or all of the specific details described herein. In many cases, the steps of the methods described herein may be performed in a different order than presented herein, or in parallel rather than sequentially.

Claims

1. reading, by a computer, a machine readable serial number of the medical device when the medical device is used, and recording a reduction in the level of on-hand inventory of a class of medical device of which the medical device is a component based on said reading of said serial number; calculating, by a computer, when said decrease in on-hand inventory of said class indicates a level that warrants reordering to replenish said on-hand inventory; and placing, by a computer, an order with a supplier of said class of medical devices to replenish on-hand inventory of said class of medical devices based on said calculated basis for reordering. A method for providing

2. 10. The method of claim 1, further comprising connecting one of the medical devices to a computer processor, the processor being programmed to retrieve characteristics of the connected medical device from a database designed to store information about specific individual medical devices, the database being accessed based on the read serial number.

3. the medical device is an endoscope having an image sensor designed to collect photons to generate video, the image sensor being of a type having characteristics that vary from sensor to sensor within manufacturing tolerances; The method further comprises: storing characteristics of the endoscope's image sensor and / or the behavior of the image sensor in a database, the database storing characteristics specific to a particular sensor; 3. The method of claim 2, wherein the image processor is programmed to retrieve the characteristics of the image sensor of the one of the endoscopes from the database and calculate a normalized video based on the retrieved characteristics and the video from the image sensor.

4. The database is further designed to store the physical location and / or ownership of each medical device; The method further comprises: reading an identification code on the packaging of the medical device with an optical reader; and updating said database records of physical location and / or ownership to indicate current physical location and / or ownership based on said read identification code. The method of claim 2 comprising:

5. the medical device is an endoscope having an image sensor designed to collect photons to generate video, the image sensor being of a type having characteristics that vary from sensor to sensor within manufacturing tolerances; The method further comprises: storing characteristics of the endoscope's image sensor and / or the behavior of the image sensor in a database, the database storing characteristics specific to a particular sensor; 5. The method of claim 4, wherein the image processor is programmed to retrieve the characteristics of the image sensor of the one of the endoscopes from the database and calculate a normalized video based on the retrieved characteristics and the video from the image sensor.

6. The method of claim 2 , wherein the identification code is a machine-readable 2D optical code.

7. The method of claim 2 , wherein the identification code is stored in a machine-readable electronic memory.

8. The method of claim 7 , wherein the identification code is stored in a machine-readable non-volatile memory.

9. 3. The method of claim 2, wherein the database stores information about device models of classes of manufactured medical devices in a compatible manner, and the database is designed to index the device model information based on the read identification code of each individual medical device.

10. A computer system one or more processors; and Machine-readable non-transitory memory that stores instructions wherein the instructions cause the one or more processors to reading a machine-readable serial number of said medical device when said medical device is used, and recording a reduction in the level of on-hand inventory of a class of medical devices of which said medical device is a component based on said reading of said serial number; calculating when said decrease in on-hand inventory of said class indicates a level that warrants reordering to replenish said on-hand inventory; placing an order with a supplier of said class of medical devices to replenish on-hand inventory of said class of medical devices based on said calculated basis for reordering; A computer system that is programmed to

11. the medical device is an endoscope having an image sensor designed to collect photons to generate video, the image sensor being of a type having characteristics that vary from sensor to sensor within manufacturing tolerances; The instructions further include causing the one or more processors to: connecting one of the medical devices to a computer processor, the processor being programmed to retrieve characteristics of the connected medical device from a database, the database being designed to store information about specific individual medical devices, the database being accessed based on the read serial number; storing characteristics of the endoscope's image sensor and / or the behavior of the image sensor in a database, the database storing characteristics specific to a particular sensor; Retrieving the characteristics of an image sensor of the one of the endoscopes from the database; and calculating a normalized video based on the retrieved characteristics and the video from the image sensor.

11. The computer system of claim 10, programmed to:

12. The database is further designed to store the physical location and / or ownership of each medical device; The instructions further include causing the one or more processors to: directing an optical reader to read an identification code on packaging of the medical device; and connecting one of the medical devices to a computer processor, the processor being programmed to retrieve characteristics of the connected medical device from a database, the database being designed to store information about a particular individual medical device, the database being accessed based on the read serial number; updating a database record of said physical location and / or ownership to indicate a current physical location and / or ownership based on said read identification code; 11. The computer system of claim 10, programmed to:

13. An endoscope, an image sensor designed to collect photons and generate video, said image sensor being of a type whose characteristics vary from sensor to sensor within manufacturing tolerances; a connector for connecting the endoscope to a computer image processor; wherein the image processor comprises: obtaining data from a database designed to store characteristics of the image sensor of the endoscope and / or the behavior of the image sensor, the database storing characteristics specific to a particular sensor or a particular class of sensors; the acquired data is for describing characteristics of the image sensor from the database; Calculating a normalized video based on the acquired characteristics and the video from the image sensor. It is programmed to Endoscope.

14. reading a machine-readable serial number of the endoscope when the endoscope is used, and recording a reduction in the level of on-hand inventory of the class of endoscope of which the endoscope is a component based on said reading of the serial number; calculating when said decrease in on-hand inventory of said class indicates a level that warrants reordering to replenish said on-hand inventory; placing an order with a supplier of endoscopes of said class to replenish on-hand inventory of endoscopes of said class based on said calculated basis for reordering; 14. The endoscope of claim 13, further configured to be connected to a computer processor programmed to:

15. The database is further designed to store information regarding specific individual medical devices, including physical location and / or ownership; reading the 2D identification code on the packaging of the medical device with an optical reader; 14. The endoscope of claim 13, further comprising updating a database record of said physical location and / or ownership to indicate a current physical location and / or ownership based on said read 2D identification code.

16. The endoscope of claim 13, wherein the database is designed to store data describing the color spectral capabilities of the image sensor.

17. The endoscope of claim 13, wherein the database is designed to store data describing an image plane resolution of the image sensor.

18. obtaining data from a database designed to store calibration characteristics of the endoscope's image sensor specific to the particular image sensor of the particular endoscope; The endoscope of claim 13, wherein the image processor is programmed to calculate normalized video based on the acquired calibration characteristics and the video from the image sensor.

19. The endoscope of claim 18, wherein the calibration characteristic describes the white balance of the image sensor.

20. The endoscope of claim 18, wherein the calibration characteristics describe a color correction gamma curve for the image sensor.

21. The endoscope of claim 18 , wherein the calibration characteristics describe distortion correction of the image sensor.

22. a computer processor programmed to receive image data from an image sensor of an endoscope, retrieving the data from a database designed to store characteristics of the image sensor of the endoscope and / or of the behavior of said image sensor, said image sensor being of a type having different characteristics for each sensor, said database storing characteristics specific to a particular sensor or a particular class of sensors; calculating, in the computer, a normalized video based on the acquired characteristics and the video from the image sensor; A method for providing

23. in the computer, reading a machine-readable serial number of the endoscope when the endoscope is used, and recording a reduction in the level of on-hand inventory of the class of endoscope of which the endoscope is a component based on said reading of said serial number; calculating when said decrease in on-hand inventory of said class indicates a level that warrants reordering to replenish said on-hand inventory; and placing, by a computer, an order with a supplier of said class of endoscopes to replenish on-hand inventory of said class of endoscopes based on said calculated basis for reordering.

23. The method of claim 22, further comprising:

24. The database is further designed to store information regarding specific individual medical devices, including physical location and / or ownership; reading the 2D identification code on the packaging of the medical device with an optical reader; 23. The method of claim 22, further comprising updating a database record of said physical location and / or ownership to indicate a current physical location and / or ownership based on said read 2D identification code.

25. obtaining data from a database designed to store calibration characteristics of the endoscope's image sensor specific to the particular image sensor of the particular endoscope; Furthermore, 23. The method of claim 22, wherein the image processor is programmed to calculate a normalized video based on the acquired calibration characteristics and the video from the image sensor.

26. The method of claim 25 , wherein the calibration characteristics describe the white balance of the image sensor.

27. storing information about specific individual medical devices, including physical location and / or ownership, in a database; reading the 2D identification code on the packaging of the medical device by an optical reader; and updating said database record of physical location and / or ownership to indicate current physical location and / or ownership based on said read 2D identification code. A method for providing

28. reading, by a computer, a machine readable serial number of the medical device when the medical device is used, and recording a reduction in the level of on-hand inventory of a class of medical device of which the medical device is a component based on said reading of said serial number; calculating, by a computer, when said decrease in on-hand inventory of said class indicates a level that warrants reordering to replenish said on-hand inventory; and placing, by a computer, an order with a supplier of said class of medical devices to replenish on-hand inventory of said class of medical devices based on said calculated basis for reordering.

28. The method of claim 27, further comprising:

29. the medical device is an endoscope having respective image sensors designed to collect photons to generate video, the image sensors being of a type having characteristics that vary from sensor to sensor within manufacturing tolerances; The method further comprises: storing characteristics of the image sensor of the endoscope and / or the behavior of the image sensor in a database, the database storing characteristics specific to a particular sensor; and connecting one of the endoscopes to a computer image processor, the image processor being programmed to retrieve the characteristics of an image sensor of the one of the endoscopes from the database and calculate a normalized video based on the retrieved characteristics and the video from the image sensor.

28. The method of claim 27, comprising:

30. calculating a delivery date forecast for medical devices between the time of sale and the time of delivery based on the read 2D identification code; 28. The method of claim 27, further comprising:

31. storing information about specific individual medical devices, including their physical locations, in a database; reading the 2D identification code on the packaging of the medical device by an optical reader; and updating said physical location and / or ownership database record to indicate a current physical location based on said read 2D identification code.

28. The method of claim 27, further comprising:

32. storing information regarding specific individual medical devices, including ownership, in a database; reading the 2D identification code on the packaging of the medical device by an optical reader; and updating said physical location and / or ownership database record to indicate current ownership based on said read 2D identification code.

28. The method of claim 27, further comprising:

33. A computer system one or more processors; and Machine-readable non-transitory memory that stores instructions wherein the instructions cause the one or more processors to causing a database to store information regarding specific individual medical devices, including physical location and / or ownership; causing an optical reader to read a 2D identification code on a package of said medical device; updating a database record of said physical location and / or ownership to indicate a current physical location and / or ownership based on said read 2D identification code; A computer system that is programmed to

34. The instructions further include causing the one or more processors to: reading a machine-readable serial number of said medical device when said medical device is used, and recording a reduction in the level of on-hand inventory of a class of medical devices of which said medical device is a component based on said reading of said serial number; calculating when said decrease in on-hand inventory of said class indicates a level that warrants reordering to replenish said on-hand inventory; placing an order with a supplier of said class of medical devices to replenish on-hand inventory of said class of medical devices based on said calculated basis for reordering; 34. The computer system of claim 33, programmed to:

35. the medical device is an endoscope having respective image sensors designed to collect photons to generate video, the image sensors being of a type having characteristics that vary from sensor to sensor within manufacturing tolerances; The instructions further include causing the one or more processors to: storing characteristics of the endoscope's image sensor and / or the behavior of the image sensor in a database, the database storing characteristics specific to a particular sensor; connecting one of said endoscopes to a computer image processor; 34. The computer system of claim 33, wherein the image processor is programmed to: retrieve the characteristics of an image sensor of the one of the endoscopes from the database; and calculate a normalized video based on the retrieved characteristics and the video from the image sensor.

36. storing in a database information relating to particular individual endoscopes, said endoscopes having image sensors of each type with characteristics that vary from sensor to sensor within manufacturing tolerances, said stored information comprising: Information recording characteristics of the image sensors of individual endoscopes and / or the behavior of said image sensors; and including the physical location and / or ownership of individual endoscopes; reading, with an optical reader, a 2D identification code on the endoscope packaging, and updating a database record of the physical location and / or ownership to indicate a current physical location and / or ownership based on the read 2D identification code; reading, by a computer, a machine-readable serial number of each endoscope when the endoscope is used; and recording a decrease in the level of on-hand inventory of the class of endoscopes of which the endoscope is a component based on said reading of said serial number; recording, based on the read serial number, a relationship between the endoscope having the serial number and the patient for whom the procedure will be performed using the endoscope; calculating, by a computer, when said decrease in on-hand inventory of said class indicates a level that warrants reordering to replenish said on-hand inventory; placing, by the computer, an order with a supplier of said class of endoscopes to replenish on-hand inventory of said class of endoscopes based on said calculated basis for reordering; retrieving the recorded characteristics of the endoscope's image sensor from the database based on the read serial number, and calculating a normalized video based on the retrieved characteristics and the video from the image sensor. A method for providing

Images