Information management and inventory tracking for medical devices

Abstract

During manufacture of the medical device, parameters of the device are measured and the measurement results are stored in a database. For example, parameters of an image sensor of an endoscope may be measured and stored in a database. The medical device has a machine-readable serial number. The computer reads the serial number and records a reduction in the existing inventory of the medical device. And when the existing inventory needs to be reordered to supplement the existing inventory. When needed, the computer orders to a vendor of such other medical devices to replenish inventory. A computer controlling the medical device may read the database to acquire the measured characteristics. The acquired data may be used to calibrate or adjust the functionality of the device. For example, a standardized video may be calculated using the white balance and color correction of the endoscope.

Description

Background Art

[0001] This application claims priority to U.S. Application Serial No. 18 / 370,375, filed September 19, 2023, entitled “Image Enhancement for Endoscope,” U.S. Provisional Application Serial No. 63 / 538,485, filed September 14, 2023, entitled “Endoscope,” U.S. Provisional Application Serial No. 63 / 534,855, filed August 27, 2023, entitled “Endoscope,” U.S. Provisional Application Serial No. 63 / 531,239, filed August 7, 2023, entitled “Endoscope,” and U.S. Provisional Application Serial No. 63 / 437,115, filed January 4, 2023, entitled “Endoscope with Identification and Configuration Information.” These priority applications are incorporated herein by reference.

[0002] This application relates to endoscopes, laparoscopes, arthroscopes, colonoscopes and similar surgical devices or instruments that are particularly suitable for or intended to be used for evaluating, examining, measuring, monitoring, studying or testing the living or dead human and animal body for medical purposes, or for use in conjunction with equipment designed to assist in surgical procedures in performing or preparing for surgical procedures on the human body.

[0003] Endoscopes can be arthroscopy (for joint surgery), laparoscopy (for abdominal surgery), colonoscopy (rectum, colon, and lower small intestine), cystoscope (bladder and urethra), encephaloscope (brain), hysteroscope (vagina, cervix, uterus, and fallopian tubes), sinusoscope (ears, nose, and throat), thoracoscope (chest outside the lungs), bronchoscope (trachea and bronchi), esophagoscope (esophagus and stomach), etc. Endoscopes can have a rigid shaft or a flexible insertion tube. Summary of the Invention

[0004] In general terms, in a first aspect, the present invention provides a method. When a medical device is placed into service, a computer reads a machine-readable serial number of the medical device. Based on the serial number reading, the computer records a decrease in the on-hand inventory level of medical devices of a category of which the medical device is a member. The computer calculates when the decrease in on-hand inventory of the category indicates a need for a reorder to replenish the on-hand inventory. When necessary, the computer places an order with a supplier of medical devices in the category to replenish the on-hand inventory of the category of medical devices.

[0005] In general, in a second aspect, the present invention provides a computer system. The system has one or more processors and a machine-readable, non-transitory memory having instructions stored therein. The instructions are programmed to cause the one or more processors to read a machine-readable serial number of a medical device when the medical device is placed into service and, based on the serial number read, record a decrease in the on-hand inventory level of a class of medical devices of which the medical device is a member. The instructions are programmed to cause the one or more processors to calculate when the decrease in on-hand inventory of the class indicates a need to reorder to replenish the on-hand inventory. The instructions are programmed to cause the one or more processors to place an order with a supplier of medical devices in the class to replenish the on-hand inventory of the class of medical devices.

[0006] In general, in a third aspect, the present invention relates to an endoscope. The endoscope has an image sensor for collecting photons to generate a video. The type of image sensor has characteristics that vary from one sensor to another within manufacturing tolerances. The endoscope has a connector for connecting the endoscope to a computer image processor. The image processor is programmed to retrieve data from a database designed to store characteristics of the endoscope's image sensor and / or the behavior of the image sensor, the database storing characteristics specific to a particular sensor or a particular class of sensors. The data retrieved from the database describes the characteristics of the image sensor. The image processor is programmed to calculate a standardized video based on the retrieved characteristics and the video from the image sensor.

[0007] In general, in a fourth aspect, the present invention provides a method. A computer processor is programmed to receive image data from an image sensor of an endoscope and retrieve data from a database configured to store characteristics and / or behavior of the endoscope's image sensor. The type of image sensor has characteristics that vary from one sensor to another, and the database stores characteristics specific to a particular sensor or class of sensors. The computer calculates a standardized video based on the retrieved characteristics and the video from the image sensor.

[0008] In general, in a fifth aspect, the present invention provides a method. A database stores information related to a specific individual medical device, including physical location and / or ownership. An optical reader reads a two-dimensional identification code on the medical device's packaging. The database record is updated based on the read two-dimensional identification code, indicating the current physical location and / or ownership.

[0009] In general, in a sixth aspect, the present invention provides a computer system. The system comprises one or more processors and a machine-readable, non-transitory memory storing instructions. The instructions are programmed to cause the one or more processors to store information related to a specific individual medical device, including physical location and / or ownership. An optical reader reads a two-dimensional identification code on the packaging of the medical device. A database record is updated based on the read two-dimensional identification code to indicate the current physical location and / or ownership.

[0010] Embodiments may include one or more of the following features. When an endoscope is placed into service, a computer may read the machine-readable serial number of the endoscope. Based on the serial number read, the current inventory level of a category of endoscopes may be reduced and recorded. A reorder level may be calculated based on the reduction in the current inventory level. The computer may calculate when the current inventory level requires reordering to replenish the current inventory. Based on the calculated need for reordering, the computer may place an order with a supplier of the category of endoscopes to replenish the current inventory of the category of endoscopes. The medical device may be an endoscope. The endoscope may have an image sensor. The image sensor may be designed to collect photons to generate video. The type of image sensor has characteristics that vary from one sensor to another within manufacturing tolerances. A program may be programmed to connect one of the medical device or the endoscope to a computer processor. The processor may be programmed to retrieve characteristics of the connected medical device from a database. The database may be designed to store information related to a specific individual medical device, endoscope, or image sensor. The database may be accessed based on the read serial number. The database may store characteristics of the endoscope's image sensor. The database may store characteristics of the image sensor's behavior in the database. The database can store characteristics specific to a particular sensor. The database can provide characteristics of an image sensor from an endoscope in the database and calculate a standardized video based on the acquired characteristics and video from the image sensor. The database can store the physical location of individual medical devices, endoscopes, or components. The database can store the physical location and / or ownership of individual medical devices, endoscopes, or components. An optical reader can read an identification code on the packaging of a medical device. The database record of the physical location and / or ownership can be updated based on the read identification code to indicate the current physical location and / or ownership. When a medical device or endoscope is connected to its control computer, the computer can be programmed to retrieve the characteristics of the connected device from the database. The identification code can be a machine-readable two-dimensional optical code. The identification code can be stored in a machine-readable electronic memory. The database can store information related to device models of medical devices manufactured to be interchangeable. The database can be designed to index to device model information based on the read identification code of an individual medical device. The database can be designed to store data describing the chromatographic capabilities of an image sensor. The database can be designed to store data describing the image plane resolution of an image sensor. The database can be designed to store calibration characteristics of the endoscope's image sensor that are specific to a particular image sensor of a particular endoscope. The image processor can be programmed to calculate a normalized video based on the acquired calibration characteristics and video from the image sensor. The calibration characteristics can describe the image sensor's white balance, the image sensor's color correction gamma curve, the image sensor's distortion correction, or any two or more of these characteristics, or any three or more of these characteristics.The computer can calculate a delivery date prediction based on the read two-dimensional identification code. The computer can update the physical location and / or ownership database record based on the read two-dimensional identification code or the reading of the machine readable memory to indicate the current physical location.

[0011] The advantages and features described above are merely representative embodiments and are presented only to aid understanding of the present invention. However, it should be understood that they should not be considered as limitations of the present invention as defined by the claims. Additional features and advantages of embodiments of the present invention will become apparent from the following description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1A 、 2 , 3, and 4 are perspective views or perspective cross-sectional views of an endoscope and / or endoscope-related devices.

[0013] Figure 1B 、 5A 5D, 5F, 6E, and 6H are block diagrams of computers or processors.

[0014] Figure 5E is a time sequence of video frames.

[0015] Figures 6A-6D , 6F, and 6G are screenshots. DETAILED DESCRIPTION The structure of the specific embodiment is as follows. I. Overview IA endoscopic surgery IB overall structure IC integrated sterile packaging ID disposable handpiece II. Partially reusable, partially disposable / replaceable endoscopes, and the coupling joint therebetween III. Retractable, bendable, or articulated camera heads IV. Additional functions of endoscope V.Endoscope tip Molding and assembly of components for VA endoscope tips Fiber optic irradiation of VB disposable endoscope tip VC has a light guide end design VD diffusion terminal surface VI. Anti-fouling and anti-fog VI.A. Heating VI.B. For endoscope delivery packaging, small bottles of liquid are used to protect the coating on the lens / window of the endoscope VII. Lens cover with optical correction for use during insertion of an off-view endoscope VIII. Avoid using fasteners, springs and other small parts VIII.A. Push button with embedded spring VIII.B. Overmolding of the Case Over the Circuit Board VIII.C. Avoid using internal fasteners VIII.D. Rotational resistance through O-ring VIII.E. Ultrasonic welding of the two halves of the handle housing VIII.F. Thermoplastic elastomer coated handle IX. Liquid Flow IX.A. Liquid-tight seal IX.B. Induced spiral flow X. Molding and joining XA diagonal slots for joining dissimilar materials Joining of XB obturator components XC twist-locks sections together XI. Endoscope tip XI.A. Molding and Assembly of Components of Endoscope Tips XII. Image Processing Unit XII.A. Image Processing XII.B. HDR exposure fusion to maintain frame rate XII.C. Automatic Exposure XII.D. Super-resolution Video Processing XII.E. Diagnosis and Lesion Detection XII.F. Endoscope Control XII.G. Flexible Boards and Electronics in Endoscope Handles XII.H. Cable XII.I. Wireless communication replaces cables XII.J. Isolation XII.K. Other peripheral equipment XII.K.1. Monitor XII.K.2.USB port XII.K.3. Connecting to Cloud Storage XII.K.4. USB connection for keyboard and mouse XII.K.5. Microphone XII.K.6. Insufflation tube XIII. Machine-readable serial number XIII.A. Machine-readable serial number XIII.B. Use machine-readable serial numbers to reduce errors, ensure non-expiration and ensure sterile single use XIII.C. Use machine-readable serial numbers to communicate patient data into the electronic medical record XIII.D. Use machine-readable serial numbers for inventory control, location tracking, reordering, and stock management XIII.E. Configuration / Capability Information XIII.F. Calibration Information XIII.G. Implementation XIV. Examples I. Overview IA endoscopic surgery

[0016] refer to Figure 1A , an endoscope 100 (such as an arthroscope, laparoscope, or other), a trocar 102, and an obturator 104 may be used for joint surgery, joint access, or other minimally invasive surgery.

[0017] Various endoscope tip designs may have the following characteristics. The entire tip may be small enough to fit the dimensions of the endoscope, typically the dimensions in the table in paragraph

[0026] below. In some cases, the diameter of the tip may be slightly larger or smaller than the shaft. The tip can maintain mechanical stability within this diameter for the camera 410, illumination, fluid injection or evacuation ports, procedural tools, etc. The tip can be sealed to resist the increased pressure typically used to transfer tissue outside the field of view of the endoscope to prevent the intrusion of body tissue and fluids and insufflation fluids. The tip can deliver or allow the delivery of illumination light via an LED 418 mounted in the tip or using an optical fiber 430 to transmit light from a handle or controller. The opaque portion of the tip assembly can exclude stray light from the imperfect optical path within the tip of the illumination fiber / LED / light guide and reflected light from the surgical cavity. The tip can be manufactured in the desired quantity and cost. The tip can have a configuration that is non-damaging to surrounding tissue, for example, without sharp points or edges. The endoscope can be formed of biocompatible materials, such as stainless steel and / or certain plastics. In some cases, the tip may have a puncture point. The tip can be designed to resist atomization or dirt. The tip can allow cleaning, preferably in situ, of the surgical site. IB overall structure

[0018] refer to Figure 1BThe endoscope 100 can be part of an overall system designed to deliver high-definition video for endoscopic surgery. The system can provide real-time high-definition video for display on a video monitor and as stored video and still image capture; illumination of the surgical cavity, irrigation and / or inflation (insufflation) of the surgical site, and image refinement, such as zooming, rotation, removal or reduction of hot spots and other artifacts, etc.

[0019] The system may include an endoscope, including an insufflation tube, communication / control / power / illumination cables, a cannula, and an obturator. An image processing unit (IPU) or main controller may be reused in multiple procedures. If illumination is provided via fiber optics, there may also be a light box, typically near the IPU, to align the fiber optics with other necessary cords and hoses. One or more of the endoscope, tubing, cables, cannula, and obturator may be designed for single use and sold together as an integrated kit.

[0020] refer to Figure 5A and 5B , the endoscope can have electronics in the handle that control the camera and illumination (LED or fiber optic). The IPU can have a computer processor for various image processing functions, as well as controllers for electromechanical devices in the endoscope, Wi-Fi or similar radio communications, USB and cloud storage, etc. Because the endoscope is disposable, it is easy to provide sterility. The connecting cable can also be disposable so that it can be delivered in sterile packaging. The IPU is relatively expensive and difficult to sterilize, so it is not in the sterile area.

[0021] refer to Figure 5F , various isolation connectors can provide electrical isolation between the patient and the wall voltage components of the IPU. IC integrated sterile packaging

[0022] The endoscope, tubing, and cable can be designed for single use and packaged and sold together as an integrated kit. In addition, one or more obturators and cannulas can be packaged and sold with the kit. The kit can be sold sterile. The package can be designed to be opened within the sterile field surrounding the patient during surgery. The lid on the package can be made of Tyvek. (Tyvek ) or some similar film that is transparent to ethylene oxide or similar disinfectants, so that the packaging and components can be sterilized together during manufacturing. The film cover will remain in place until shortly before surgery. This eliminates the need to disinfect or sterilize the endoscope immediately before surgery. The tray containing the components can be transparent so that the contents of the tray can be seen before removing the Tyvek lid.

[0023] Because the components are sold together, they can be calibrated against each other. The various characteristics of the illumination, image sensor, lenses, filters, etc. can be calibrated against each other as a group at the factory. White balance can be one of the parameters calibrated at the factory - because the components are single-use and sold as an integrated package, they can be calibrated internally at the factory and follow this joint calibration throughout the life of the product. In contrast, with traditional endoscopes, the light source and endoscope are separate, and the color temperature or balance of the illumination source varies from light source to light source, and the color sensitivity of the image sensor's pixels also varies from endoscope to endoscope, so the user must perform white balancing as part of the preparation for each procedure. In configurations where endoscopes are sold as single-use configurations, their electronic serial number is associated with the calibration factors measured at the factory (see below §IV.A and

[0088] to

[0095] ), an endoscope can be calibrated by imaging a white surface that provides equal portions of red, green, and blue pigments to the test surface, where the illumination results in medium-level, non-saturated pixel values ​​in the image sensor, and a correction coefficient matrix can be calculated to adjust the color balance of the pixels of the image sensor's signal. ID disposable handpiece

[0024] The endoscope itself can be designed for single use. The image sensor, lens, filter, and cover window, as well as the illumination emitter (LED 418 or the distal end of a fiber optic illumination fiber or waveguide) can be located at the distal end of the insertion shaft. The sensor, lens, filter, cover window, and illumination emitter can be designed to interoperate with each other to allow insertion of small diameter insertion shafts. Single use ensures sterility even for components with complex geometries and materials that cannot be autoclaved (such as the electronics of the endoscope). The endoscope can have electronic tracking capabilities to ensure single use (see below §IV.B and

[0096] to

[0107] ).

[0025] Typical dimensions for various surgical specialties might be as follows (measured in millimeters): II. Additional functions of endoscope

[0026] See also Figure 2 The disposable shaft portions 110, 120 can be separated into an outer sleeve 132 for protection and strength and an inner shaft portion 134 that carries various illumination, optical, and fluid delivery components. Illumination can be provided by an LED 418 at or near the distal end, or from an illumination source in the handle via an optical fiber 430, or at an external controller.

[0027] The endoscope can have handles 112, 114, 120 and a shaft 110 for insertion into the body. At or near the distal end 116 of the shaft 110 can be a lens, electronic image sensor, filter, or other optical component 410. The orientation of the camera can be fixed within the endoscope or can be translatable. The camera 410 can be located at the tip 116, viewed from the outside of the shaft, or can be recessed a short distance behind the structural end of the shaft. At or near the tip can also be an illumination source, such as an LED 418. The tip 116 can have a rigid, pointed trocar tip, or can have a spoon-shaped portion that extends beyond the distal surface of a window in the tip 116, or can be flexible (in the manner of a colonoscope tip), in each case extending slightly beyond the distal surface of a window in the tip 116 to provide physical protection for the tip 410 during insertion or to protect the camera 410 from surgical cutting devices.

[0028] The illumination can be visible light, infrared light, and / or ultraviolet light. In some cases, an illumination LED (light emitting diode) or other illumination source can be placed in the reusable handle 112, 114 or in the docking station / controller, and the disposable shaft can have an optical fiber 430 to transmit the light to the tip, and the connector 130 can have an optical coupler. In other cases, the illumination LED 418 can be placed in the tip 116 to directly illuminate the surgical cavity; in this case, the connector 130 can have a power connector. In some cases, the LED 418 can be recessed from the tip, or placed somewhere on the shaft, or can be in an external controller, and the optical fiber 430 can carry the illumination light to the tip. The optical fiber 430 can be configured, for example, to have a split so that the light will be arranged in a desired pattern around the image sensor to better distribute the light into the surgical cavity around the camera.

[0029] The shaft 110 itself can be rigid, made of a non-corrosive metal such as stainless steel or coated aluminum. In some cases, the surgical cavity surrounding the endoscope tip 400 can be insufflated with a gas (typically carbon dioxide) or flushed with saline. In either case, fluid inflow and outflow can be affected by the passage through the shaft.

[0030] The shaft 110 may also carry power lines to the illumination LED 418 and the camera 410, and signal lines that carry the video signal from the camera 410 back to the electronics in the reusable portions 112, 114 of the handle. Power for the camera 410 may be provided by conductors in a flexible cable or on a printed circuit board (flexible or rigid), and may be insulated with a conformal and insulating coating such as parylene. The same flexible circuit board 416 may have signal conductors for the video signal from the image sensor 410. The video signal may be transmitted from the image sensor 410 to the handle using any video signal protocol, such as MIPI-CSI2 (Mobile Industry Processor Interface - Camera Serial Interface 2) or HDMI. In some cases, the parylene coating may improve biocompatibility.

[0031] Shaft 110 may also carry cables or other mechanical elements to control the panning of camera 410 .

[0032] Reference again Figure 2 The handle can include a rotation collar. The rotation collar can have various features that facilitate rotation. For example, the depression 302 can provide a good grip for the fingers to achieve a small rolling torque. The fins can provide greater leverage for a larger rolling torque and can also provide a fixed rotation reference point.

[0033] The proximal handle 114 may include a rotation sensor so that the angular orientation of the camera 410 can be determined. For example, one or more magnets 320 may be mounted on the inner surface of the proximal handle 114, and the printed circuit board 322 (rotating with the rotating collar 112 and the disposable lid 120) may have a Hall effect sensor 324 that detects the magnets. This can be used to calculate the rotational orientation, which in turn can be used to "correct" the image from the camera 410 on the video display screen.

[0034] The distal end of the shaft, the camera 410 mounted therein, and the mounting of components within the shaft 110 can be designed to be robust. Occasionally, during a procedure, the tip of the endoscope may come into contact with a scraper, ablation probe, or cauterization probe, and it may be desirable to make the tip robust to such contact. To reduce the risk that a component may become dislodged and remain in the patient, the disposable shaft and its components can be designed to avoid joints that are at higher risk of mechanical failure. Disposable optical systems can prevent image degradation that occurs when non-disposable optical devices are reused across multiple procedures.

[0035] Endoscopes as a genus include arthroscopy, laparoscopes, colonoscopes, and other specialized scopes for various body cavities. For arthroscopy used for joint surgery, the shaft can be as small as 6 mm, 5 mm, 4.5 mm, 4 mm, 3.6 mm, 3.3 mm, 3 mm, 2.8 mm, 2.6 mm, 2.4 mm, 2.2 mm, 2 mm, or 1.8 mm and have high rigidity. For other endoscopes, such as colonoscopes, the diameter may be larger and the shaft may be flexible.

[0036] Endoscopes are available with a handle and multiple tips, each individually sealed to ensure sterility.

[0037] See also Figure 3 The hoses 160, 162 for flushing / insufflating fluids / gases and flushing / insufflating fluids / gases, as well as the electrical connection cord 164, can be permanently attached 340, 342 to the disposable lid 120. This arrangement allows the hose 162, which carries water out of the surgical cavity and thus becomes contaminated, to be disposable, and prevents any fluid from coming into contact with the reusable portion 114 of the handle. The hoses and cords 160, 162 can be routed through a channel 354 that extends the length of the reusable handles 112, 114. The inner diameter of channel 344 can be large enough to easily pass the hoses and cords 160, 162, 164, and connectors 350, 352, and have continuous, smooth walls that allow for easy sterilization, allowing for ready replacement of replaceable components. Channel 354 can be offset from the central axis to allow the printed circuit board 322 to be located on the central axis. The connectors 350, 352 at the ends of the hoses and cords 160, 162 can be small enough to pass through channel 354. Thus, shaft 110, cap 120, hoses, and cables 160, 162 can be replaced by inserting connectors 350, 352 and the hoses and cables 160, 162 through passage 344. Electrical wire 164 can have a connector 354 at or near junction 130, and hose 160, used for flushing / insufflating fluids / gases flowing into the surgical cavity, can also have a connector at junction 130 to allow for reuse of the hose, or can be permanently attached 340 to reduce the potential for leaks. Having the hoses and cables 160, 162 generally on axis reduces unwanted cable tumbling during endoscope use and reduces unwanted torque on cap 120. Forming shaft 120, cap 120, and hoses 160, 162 as an integral unit for replacement reduces the potential for leaks and improves the sterility of the replacement procedure. II.A. Tip Design with Light Guide

[0038] The optical fiber 430 can be extruded into various shapes to improve light transmission, such as a rectangular or U-shaped light guide 450 that can extend along the length of the endoscope from the illumination source to the U-shaped emitting surface at the distal end of the endoscope. In some cases, at least a portion of the length of a single optical fiber 430 can be replaced with a shaped light guide 450, such as a circular or U-shaped transparent light guide ring around the perimeter of the tip chassis 438, 480. The light guide 450 can be a two-component structure with two different refractive indices for internal reflection similar to that of an optical fiber. In other cases, the light guide 450 can be made of a transparent light-transmitting medium coated with a reflective coating such as aluminum, gold, or silver. In some cases, the shaped light guide 450 can extend only a short distance, for example, the length of the inner tip section, and a conventional round optical fiber can be used to carry light from the illumination source to the proximal end of the light guide 450 at the tip chassis 438, 480. III. Image Processing Unit

[0039] refer to Figure 5A and 5B , the image processing unit (IPU) can use an interface board to drive and receive signals from the endoscope via cables and a custom or off-the-shelf motherboard. In some cases, the motherboard can be an off-the-shelf motherboard with an Intel CPU and an Nvidia GPU. The motherboard provides most of the external interface ports. The patient can be isolated from the line voltage (110 or 120V 60Hz in the United States and 240V 50Hz in Europe) through a medical grade AC / DC power supply and a separate interface board (called a patient interface board). The patient interface board processes the signals, converting them between the signal form used inside the IPU and the signal form to and from the endoscope. III.A. Image Processing

[0040] The image processing computer can perform image processing. The GPU provides a well-documented API that can be used to accelerate graphics processing, while the software running on the motherboard can have an internal API that allows the software processing components to be combined for image enhancement. A series of video chips in the endoscope handle and IPU (image processing unit) box can convert the very small high-speed video signals from the sensor (such as the MIPI-CSI2 interface in Bayer format) into signals suitable for transmission distances greater than a few centimeters, and into protocols (such as YCbCr422 or MPEG) that are more easily processed and stored by the various stages of the imaging pipeline. The IPU processor can receive data (which can be video data, still images, telemetry data, etc.) from the endoscope via the handle board, cable, and patient interface board. The IPU can capture still images from the video and / or process the video through image correction and enhancement software to deliver high-quality images on a monitor, or store on some storage medium or in the patient record.

[0041] Various video signal processing chips, image signal processors (ISPs) and graphics processing units (GPUs) can perform multiple video transformations on the video data received from the endoscope before the data is displayed on a monitor or saved to an output device. The IPU box can have multiple processors, including a dedicated image signal processor (ISP), a general-purpose CPU (such as an Intel Pentium), a graphics accelerator (GPU), a field programmable gate array (FPGA), custom accelerator hardware, and perhaps other processors. The video transformations can be performed in one or another of these processors, or in software, or in some combination of hardware and software. The sum of the processing power can be selected to ensure that the image processing can be performed within the image latency requirements. The following transformations can be performed: Receives raw image data from the endoscope’s image sensor in a Bayer format MIPI-CSI2 stream and re-encodes it into a YCbCr422 or h.264 MPEG stream for improved processability Convert the MIPI-CSI2 video stream to a USB 3.0 video stream compliant with the UVC specification via a video stream processor such as the Cypress CX3. HDR or WDR processing (High Dynamic Range or Wide Dynamic Range) - software (a) to expand the dynamic range of a captured image by avoiding overexposed or underexposed areas of the video. This is achieved by combining consecutive overexposed and underexposed image frames from the image sensor, reducing the display intensity of abnormally bright pixels to reduce hot spots, and increasing the display brightness of abnormally dark pixels in the frame to improve the visibility of the image. See Figure 5E HDR / WDR processing can use the Mertens exposure fusion algorithm. Rotation and image correction based on a handle-based rotation sensor. This includes displaying a rotation position indicator around the perimeter of the circular mask on the user interface, which can be displayed as an arrow. Distortion correction (systematic distortion due to fisheye distortion or similar distortion in lens specifications, or specific distortion measured within a specific range, corrected in the IPU via inverse transformation) to remove artifacts. Cropping the rectangular image received from the endoscope into a rectangle that can be rotated around the center point of the display. Applying a circular mask to this rectangular crop provides the user with a circular image display. This can replicate the view surgeons are used to from dozens of rod lens endoscopes. Furthermore, outside the cone of view provided by the lens, the outer edges of the image may be distorted or obscured by the edges of the lens housing, conveying more noise than information. Automatic exposure by adjusting exposure time and gain in the image capture pipeline to achieve the desired average image brightness. Demosaicing Black Level Correction Gain adjustment Shadow correction Defect correction Noise reduction Tone mapping Color correction and white balance correction Zoom in / out within the target image Lens resolution correction Local contrast enhancement Edge enhancement Image magnification (enlarging the circle displayed on the monitor may result in loss of the upper and lower limbs displayed in the circle) • Reformat and compress video data for storage on a storage device, and decompress stored video data for display. Control transfers over network connections for storage in the cloud or on local storage on the IPU, or other non-cloud storage · Below in §III.D Paragraphs

[0051] to

[0060] discuss super-resolution—this is upsampling from a lower resolution (e.g., 1280x720) to 2160x2160 ("4K") resolution. The frame writer is the final stage that places the video into the video system's frame buffer for display or storage. The fully processed video stream can be displayed on a video monitor or can be sent to a storage device or a network interface.

[0042] Parallelism can be achieved by dividing the pipeline into multiple stages. For example, each stage can be assigned to a core in a multi-core CPU or a different functional unit in a GPU. III.B. HDR Exposure Fusion to Maintain Frame Rate

[0043] refer to Figure 5E HDR exposure fusion can be performed on pairs of frames captured simultaneously by two different cameras, and the images are then merged pairwise. Exposure fusion algorithms include Mertens-Kautz-Van Reeth or Hugin / Enfuse.

[0044] In other cases, a single image sensor can be programmed to overexpose frame n, then underexpose frame n+1, then overexpose frame n+2, and so on. This can be controlled by strobing the LED 418 at the frame rate or by controlling the exposure time of the image sensor. The short exposure time frames can reveal details in overexposed portions of the image ("hot spots"), while the overexposed frames can reveal details in underexposed portions of the image ("dark areas"). By merging the frames, both hot spots and dark areas are captured in the output image, thereby increasing the dynamic range that can be captured.

[0045] The same class of HDR exposure fusion algorithms can then be used to merge frames pairwise, except applied to overlapping pairs of frames, to merge frame n with frame n+1, then frame n+1 with frame n+2, then frame n+2 with frame n+3, etc. This maintains the output frame rate at the input frame rate. III.C. Automatic Exposure

[0046] The auto-exposure algorithm can be used to adjust for fluctuations in the light intensity level of the scene being captured by the image sensor to achieve a target brightness level. If the camera is moved closer to the subject with static gain, exposure, and illumination intensity, the overall scene becomes brighter, so the exposure time, gain, and / or illumination intensity per frame should be reduced to capture less light. Conversely, if the camera is moved further away from the subject, the overall scene becomes darker, and the exposure time, gain, and / or illumination intensity should be increased to capture more light.

[0047] Auto-exposure implementations can control both exposure time and gain to achieve a target intensity set point. Gain control can be analog gain in units of pixels of the image sensor, or digital gain applied in the image sensor or digital image processing pipeline. The brightness set point can be set via a user "brightness" control, or it can be set automatically. The auto-exposure algorithm can perform the following steps: 1. Split the frame into nxn pixel blocks. 2. Calculate the average intensity of each block. 3. Compare the calculated intensity for each block to an intensity set point (which can be set for each block or for the image as a whole) to obtain an error value for each block. Each block can be assigned a weight to scale its calculated error value. The weight allows some blocks to be more important than others (i.e., blocks in the middle of the grid are weighted more than those further out). 4. Sum all weighted block errors to get the total error value. 5. Evaluation changes: If the total error value is below the defined change threshold, no change is made. b. If the total error value is above the defined change threshold, scale the change for one update period and update the change threshold relative to the magnitude of the total error using the following equation. Maximum change threshold = maximum change threshold + (total error x multiplier), where the multiplier is less than 1 to allow for a damped response. c. The maximum threshold is set to minimize user perception of discrete light level changes in similar usage environments, but to allow for rapid updates when the environment changes rapidly from dark to light or light to dark. A multiplier is used to adjust this response to achieve the fastest response time to large changes in ambient conditions while preventing oscillations in the user-perceived light level. 6. Input the total error into the exposure or gain PID control: a. If the scene is too bright: (i) If the gain is at its minimum value, the exposure PID control operates (ii) Otherwise, the gain PID control operates b. If the scene is too dark: (i) If the exposure reaches the maximum value, the gain PID control operates. (ii) Otherwise, the exposure PID control runs c. Depending on the implementation, any two or more parameters may replace gain and exposure, including illumination intensity, exposure time, etc. 7. Write the resulting exposure and gain to the ISP.

[0048] The auto-exposure algorithm can be run downstream of the WDR algorithm, possibly in the immediate aftermath. This reduces the auto-exposure algorithm's sensitivity to frame-by-frame variations in the exposure time used by the WDR algorithm. The auto-exposure algorithm can be run every few frames (rather than every frame) to reduce processing bandwidth. The per-block intensity calculations can be parallelized to run on the GPU.

[0049] The software may provide that many parameters of the algorithm can be adjusted via a configuration file loaded as part of system startup, including the number of frames allowed to run between recalculations of the auto-exposure parameters, the block size for step 1, the average intensity set point for step 3, the block weight map for step 3, and the PID coefficients for the PID calculation for step 5. III.D. Super-resolution Video Processing

[0050] refer to Figure 5C and 5D The input of the super resolution block can be a low resolution video (e.g., a 720p block of input pixels (“pixels”) or a 1280p block of input image), and the output can be an enhanced quality 2160p pixel (“pixel”) image. The “super resolution” block can further have Figure 5DBlock diagram shown. A machine learning model can be used to combine noise reduction, lens resolution correction, edge enhancement, local contrast enhancement, and upscaling into an integrated module. When performed separately, each of these functions makes various trade-offs, and image enhancement in one stage may interfere with and degrade the enhancement in another. For example, many noise reduction algorithms tend to result in blurred images. Traditional edge sharpening tends to amplify noise. By combining all of these functions in a single machine learning model, these trade-offs can be reduced.

[0051] Various types of machine learning models can be used with Figure 5C and Figure 5D The disclosed systems are used together, including fully convolutional neural networks, generative adversarial networks, recurrent generative adversarial networks, or deep convolutional networks. Convolutional neural networks (CNNs) are particularly useful for image processing. Super-resolution CNN blocks can be formed by combining the following: A CNN upscaling module from NexOptic Technologies Inc. in Vancouver, British Columbia. This allows the processor to infer pixel-to-pixel interpolation based on local information and information from previous and next frames to increase apparent resolution. ●Noise reduction module from NexOptic. This reduces noise from image sensors, electronics, and stray light photons Lens Resolution Correction Module from NexOptic: This step can enhance the performance of a lens by understanding the transfer function of a fixed image through the lens. Local contrast enhancement module from NexOptic. This can assist surgeons by increasing the contrast between light and dark, various shades of red, etc. Dynamic Range Compensation - Portions of the image that are washed out due to overexposure can be balanced against portions of the image that are washed out due to darkness. The overall dynamic range can be adjusted to improve contrast and extract details lost in overexposed or underexposed portions (see Figure 5E ). Edge Enhancement Module from NexOptic. This can reduce resolution loss (blurring) that may be introduced by the lens system (e.g., due to limitations in lens size or complexity) or by the camera's motion of objects in the scene, and can improve edge extraction to aid the surgeon by making structures more distinct at the surgical site. ● High entropy random noise interferes with data compression. CNNs can be trained to recognize and remove random pixel noise, which can improve data compression.

[0052] By combining all of these functions into a single CNN, local contrast, edge enhancement, and noise reduction can all be improved simultaneously. Much like human neural networks cleverly optimize multiple parameters simultaneously, computer CNNs can be trained to optimize several properties simultaneously. Hardware contrast and edge enhancement can be disabled. In some cases, degradation and training may involve at least two of the parameters in the list above, for example, resolution and edge enhancement, or resolution and local contrast. In some cases, any three of these types of image degradation can be trained into the model, for example, resolution, local contrast, and edge enhancement, or resolution, image sensor noise, and lens correction. In some cases, the model can be trained on any four of these parameters. In some cases, all five can be trained.

[0053] In one example embodiment, given an input sequence of low-resolution frames High-resolution frame I i The sequence corresponds to the low-resolution frame. The super-resolution frame can be calculated in T is the radius of the time neighborhood F i is the warp operator from frame i to the current frame S i is the decimation of frame i

[0054] Video super-resolution models can be performed in two steps: a motion estimation and compensation process, followed by an upsampling process. Alternatively, instead of explicitly calculating and compensating for motion between input frames, motion information can be implicitly utilized to generate a dynamic upsampling filter, and super-resolution frames can be directly constructed by locally filtering frames constructed at the center of the computation window. A machine learning model can be trained by capturing a reference video at normal resolution and then degrading the reference video via a transformation that simulates resolution loss, introduces noise, lens aberrations and similar lens noise, reduces contrast, and / or reduces edges. The machine learning model can be trained to recover the full-resolution original reference video. The same training may be sufficient to allow videos captured at normal resolution to be upsampled to a higher resolution. A lens model can be created from a combination of design data and images captured of standard test patterns (e.g., a checkerboard or Cartesian line array) to detect and measure lens defects specific to the lens design or to each endoscope, and to create a universal transform or store registration corrections for a specific endoscope. In some cases, high-quality reference data can be displayed on a physical monitor and viewed via an endoscope camera. A machine learning model can be trained to reconstruct the reference data from the camera video. Training can utilize total variation (TV) regularization loss to reduce visual artifacts.

[0055] A lens correction model can account for imperfections in the lens system that remain after all constraints have been balanced, for example, by creating a lens model and passing a large set of ultra-high resolution images captured with a camera with a very high-quality lens (to establish a baseline "perfect" image) through the lens model, and then training a CNN to correct the set of images that have passed through the lens model to transform each image into a "perfect" image.

[0056] Super-resolution CNNs may produce better overall image quality (compared to raw data directly from the camera and compared to using all classical blocks independently). Combining classical enhancement algorithms with enhanced CNNs can provide the opportunity to tune the parameters of the classical algorithms in parallel based on CNN training, where the parameters of the classical algorithms need to be tuned serially. Super-resolution CNNs can allow for tunable runtime performance via architectural choices, thereby trading off overall image quality and speed.

[0057] In some cases, a CNN can retrain itself on the fly. For example, frames can be taken alternately under intentionally underexposed (too dark) and normally illuminated at a moment when the camera and image are stationary relative to each other. The CNN can be retrained to identify hotspots where detail has been lost due to overexposure, as well as hotspots where detail has been lost in dark areas of underexposed frames. In some cases, several machine learning systems can be chained together, for example, one for enhancing dynamic range, one for reducing blur and sharpening edges, one for identifying frame-to-frame motion, one for improving contrast, and one for upsampling for super-resolution.

[0058] In some cases, the bypass feature can disable the super-resolution neural network and instead upsample the image to 2160x2160 resolution via traditional means such as bicubic interpolation.

[0059] NexOptic components are available under the product name Super Resolution, as described in U.S. Patent No. 11076103, Gordon, Photographic Underexposure Correction Using a Neural Network, and U.S. Publication No. 2021 / 0337098 Al, Gordon, Neural Network Supported Camera Image or Video Processing Pipelines, both of which are incorporated by reference. III.E. Diagnosis and Lesion Detection

[0060] In some cases, Figure 5BThe image processing pipeline may include processors for detecting various pathologies. For example, during a colonoscopy, the image processing pipeline may have a processor for detecting polyps. During an esophagoscopy, the image processing pipeline may have a processor for detecting Barrett's esophagus. III.F. Endoscope Control

[0061] The endoscope may have several controls including buttons on the endoscope, a touch screen on the face of the IPU, and a graphical user interface with the touch screen that can be accessed over the Internet from an external computer.

[0062] A single button on the endoscope controls three things: (a) still frame capture, (b) video recording on / off, and (c) LED adjustment, including high / low beam. For example, a single press captures the current view as a still frame. A double press starts or stops video recording. A triple press, or a three-second press, adjusts the LED brightness.

[0063] The IPU may have front panel controls for the endoscope, including image adjustment, color, brightness, zoom, etc. In user-visible or system setup / test mode, controls on the IPU front panel or accessed via a computer over the Internet may control: LED illumination - Because the endoscope on button is just a single momentary connected switch, it does not provide fine control, only coarse on / off control. Another user interface can provide finer illumination control ●Sensor Controls – Adjust hue or color balance, zoom, and more. ●Control image and video storage in the IPU non-volatile memory - which part of which video is stored, etc.

[0064] Adjusting LED brightness requires careful integration with the image sensor. If brightness is controlled using traditional pulse-width modulation (PWM) that is not synchronized with the image sensor's frame sync, banding may appear in the image. Alternatively, a constant current source or a voltage-controlled current source can be used to adjust LED brightness and avoid banding. III.G. Flex Boards and Electronics in Endoscope Handles

[0065] A flexible circuit board 416 transmits signals and power from the handle to the components at the tip. At the tip, molded plastic parts (brackets or chassis 412, 414, 438) hold all components in the proper orientation. Components (image sensor, lens, filter, window, and mounting) can be selected to ensure the desired offset angle (typically 0°, 30°, 45°, or 60° on axis) and the desired field of view (typically 50°, 60°, 70°, 80°, 90°, 100°, 130°, or 180°).

[0066] The distance from the image sensor in the tip to the receiver on the circuit board in the handle is approximately 115mm to 330mm, which is relatively long for a MIPI-CSI2 video connection. The flexible circuit board allows for the circuit layout and shielding to create an impedance-matched signal path for the video data from the video sensor, resulting in low radiated emissions, low loss, and low sensitivity to external interference. The isolated reference potential connection from the internal insertion shaft to the handle circuit board prevents interference from the RF ablation or coagulation device by allowing the video signal from the image sensor to float relative to the applied RF energy, thereby minimizing interference on the signal conductors that transmit the MIPI-CSI2 signal from the image sensor to the handle board.

[0067] The rigid circuit board (HB PCBA circuit board (handle board printed circuit board assembly") in the handle can have a microprocessor, a magnetic sensor, and a transmitter chip. The transmitter chip can receive low-power, high-bandwidth, high-speed signals that can be transmitted using MIPI-CSI2 streams from the image sensor received on the flexible board, and convert the video signal into a serial signal suitable for transmission to the IPU via a 3-meter cable. Because 3 meters is a relatively long distance, the cable can be carefully impedance-matched with low insertion loss to ensure signal integrity. The serial signal is received on the IPU, converted back to a MIPI-CSI2 interface, and passed to the image signal processor (ISP) for processing. III.H. Cable

[0068] The IPU can be connected to the endoscope via a custom cable. The cable can be approximately 3 meters (10 feet) long—enough to allow the surgeon to move freely and keep the unsterile IPU at an acceptable distance from the patient. The connector can be customized to ensure that the endoscope cannot be connected to other devices that cannot provide the necessary patient isolation.

[0069] The cable can use a USB Type A or C connector, because the connector has good shielding and physical insertion characteristics, so even in this application, the cable does not carry USB signals or use USB protocols. The cable can have a protective cover that extends a few millimeters beyond the end of the USB connector (alternatively, the USB connector can be recessed below the end of the protective cover). When the cable is disconnected from the IPU, the cover can provide insulation around the connector, which provides the creepage and clearance distances required for patient electrical isolation, for example, if the end of the cable happens to touch something live or grounded. The cover can be keyed so that it will only connect to the correct port on the IPU, will not (easily) plug into a generic USB connector, and ensures that the cable is only connected to the connector on the IPU in the correct way. The cable end and the plug on the IPU box can be color coded to each other.

[0070] The cable can power the endoscope, send command signals to the endoscope, retrieve configuration information stored in the endoscope's onboard memory, and bring the video signal from the endoscope back to the IPU. The cable can also support a scheme for detecting that the endoscope is connected to the IPU. This is achieved by sensing the voltage change on a pin of the endoscope cable, where the pin is pulled to a logic high voltage when the cable is disconnected and forced to a logic low voltage when the cable is connected. The pin on the cable can be connected to a pull-up resistor on the IPU side and pulled to GND on the handle board side, so when the handpiece is connected to the IPU, the handle board pulls the pin down and the processor can detect that the handpiece is connected. III.I. Wireless communication instead of cables

[0071] The cable connection between the IPU and the handpiece can be replaced with wireless transmission such as Bluetooth, Wi-Fi, or other wireless protocols. In these cases, the handpiece can have a battery with a capacity to power the handpiece for the maximum duration of the procedure. Wireless connectivity can provide an alternative architecture to achieve electrical isolation of the patient, as required by the IEC 60601-1 standard. III.J. Isolation

[0072] refer to Figure 5F The patient interface board can electrically isolate the main board from the patient-facing cable and endoscope by providing an optical connection or transformer to interrupt the copper signal path. Data isolation can be provided between the video stream processor (such as the Cypress CX3) and the main board via a fiber optic cable driven by USB 3.0 transceivers at both ends of the cable. There are no power conductors, which allows the copper conductors to be interrupted while communication occurs via the USB 3.0 communication protocol.

[0073] The physical interface between the endoscope and the IPU can be a USB 3.0 cable consisting of three twisted pairs, a ground conductor, and a pair of power wires, although the physical layer communication is not USB 3.0. The patient interface board can electrically isolate the processing circuitry from the patient-facing cable and endoscope by providing an optical connection or transformer to interrupt the copper signal path. The isolation mechanism can isolate the patient from the possibility of electric shock and prevent excessive leakage current.

[0074] The IPU box may include a transformer 1170 that steps down the 120 / 220V AC voltage to a secondary voltage used internally to operate the processing circuitry 1172 of the IPU box, and a second transformer 1180 may isolate the secondary circuitry 1172 from the patient and patient-facing circuits.

[0075] Two safety capacitors 1182 and 1184 can be placed in series across the primary and secondary of the isolation transformer. The purpose of capacitors 1182 and 1184 is to create a shunt for common-mode currents generated in the isolated switching power supply utilizing transformer 1180. The lower impedance of these capacitors, relative to the parasitic capacitance between the patient isolation island 1174 (including the endoscope) and ground, absorbs most of the common-mode current, thereby reducing the common-mode current propagating between the patient isolation island 1170 (including the endoscope) and ground, thereby reducing radiated emissions. These two capacitors can be surface-mount ceramic capacitors to minimize their impedance at higher frequencies. Capacitor 1186 can be placed differentially across the secondary of transformer 1180, creating a low impedance across the transformer's secondary at high frequencies. This low impedance allows common-mode current traveling on the transformer's positive output to travel to the transformer's negative output, through capacitor 1186, and back into the transformer via capacitors 1182 and 1184. The two capacitors 1182 and 1184 may be placed in series and may be UL listed safety capacitors to comply with IEC 60601 requirements.

[0076] A second pair of two capacitors 1192, 1194 in series can connect the USB connector shell (the metal shielding sleeve on the female side of the USB connector) to two mounting holes that are tied to the grounded IPU chassis to provide a short return path to ground for common-mode currents injected into the endoscope and / or endoscope cable. Capacitor pairs 1192, 1194 can be placed symmetrically on each side of the USB connector shell, both connected to a grounded chassis mounting point (e.g., to the outer shell 1196 of the IPU 1100) to improve shielding against common-mode currents injected into the endoscope cable.

[0077] The values ​​of capacitors 1182, 1184, 1186, 1188, 1192, 1194 are selected to provide sufficient reduction of common mode current and comply with the leakage requirements of IEC 60601-1.

[0078] In accordance with IEC 60601-1 patient isolation requirements, a pure fiber optic cable can be used to transmit high-speed video data from the patient isolation circuit 1174 to the secondary circuit 1172. The fiber optic cable can contain a USB 3.0 transceiver at each end of the cable. High-speed video from the endoscope can be converted from the MIPI-CSI2 protocol used by the image sensor to the USB 3.0 protocol via an integrated circuit. The USB 3.0 SuperSpeed ​​RX and TX data pairs can be converted to optical signals transmitted over the fiber optic cable via the optical transceiver. The optical transceivers at each end of the cable can be locally powered, eliminating the need to run power and copper wires through the fiber optic cable, thereby maintaining compliance with IEC 60601-1 isolation requirements.

[0079] The patient interface board can provide the endoscope interface, including the isolation diagram and Type BF patient isolation required by IEC 60601-1. This includes isolation of the power supply, as well as isolation of any other interfaces with copper wires that can conduct electricity (USB ports, etc.). III.K. Other peripheral devices III.K.1. Monitor

[0080] The IPU can drive a video monitor so that the surgeon can have a real-time display of the surgery. III.K.2.USB port

[0081] A USB port may be provided on the front of the unit for use with a USB flash drive, which may be connected to the motherboard via a cable. Four USB ports may be provided on the rear of the unit for use with a USB mouse and keyboard. Ethernet and Wi-Fi interfaces may be provided from the motherboard for network connectivity to cloud storage (see §§ IV.C and IV.D below).

[0108] to

[0115] ). An analog microphone input may be provided at the rear of the unit, as well as a Bluetooth interface that can be used to annotate during a procedure. A speaker may be provided in the IPU. An AC power plug may power the IPU. The AC power may be controlled by a power switch. III.K.3. Connection to cloud storage

[0082] The IPU and programming may allow for the capture and saving of video, images, metadata, and other data. The programming on the IPU may allow for the updating of the IPU's software. This data may be uploaded or backed up, for example, via Wi-Fi, Bluetooth, or a similar wireless connection to the cloud, or may be stored to an external removable USB flash drive connected to a USB port. This flash drive may then be used to transfer the data to the patient record as needed by the facility, or to upload it from an external PC to cloud storage (see §§ IV.C and IV.D, below).

[0108] to

[0115] ).

[0083] Video can be stored in two-minute increments. If a write error occurs, the length of lost video is kept within this limit. Stored videos and still images can be annotated with date, time, and location metadata, as well as the serial numbers of the endoscope and IPU. In the cloud, the serial number can be used to connect videos and images to the correct patient case.

[0084] At the end of each surgical day, the day's case data can be stored in a cloud server or on a USB drive. If the connection to the cloud fails, the USB drive provides an easily accessible backup. The surgeon can later access the cloud storage or USB drive to transfer the data to the patient's case and annotate it with their own notes. III.K.4. USB connection for keyboard and mouse

[0085] During normal operation, the endoscope button is the only available user input. A USB keyboard and mouse can be connected to the system to perform system configuration. The keyboard and mouse can allow access to service or configuration screens. III.K.5. Microphone

[0086] The IPU may have a connector for a wired microphone and may allow for the connection of a wireless microphone. This may allow for real-time annotation of the video captured by the surgeon. System settings may allow the user to specify whether they wish to enable audio and then connect a microphone with a 3.5mm jack or a Bluetooth interface. IV. Machine-readable serial number IV.A. Machine-readable serial number

[0087] Each endoscope shipped may have one or more endoscope-specific data encoded in a machine-readable, scannable, and / or human-readable form. The data may include one or more of the endoscope's serial number, configuration data, manufacturing calibration data, tracking data, etc. This data may be used for a variety of purposes.

[0088] The information can be encoded on the packaging of the endoscope in the form of a machine-readable code, embedded in the packaging, or embedded in the endoscope as a scannable code. The scannable code can be any form of matrix (2D) or linear bar or machine vision code that can be scanned by a smartphone. Examples include QR codes, Code 39, Code 49, Code 93, Code 128, Aztec codes, Han Xin barcodes, Data Matrix codes, JAB codes, Maxi codes, PDF417 codes, SPARQ codes, and other codes. The scannable code can be an RFID or similar tag that can be scanned by a sensor in a mobile phone. The scan can be optical or can use any IEEE 802 or related communication protocol, including Bluetooth, RFID (ISO 14443), or NFC (ISO 18092). The scannable code can be encoded on the packaging, on the endoscope handle, or on the nose cover of the insert end of the replaceable endoscope. Alternatively, it can be stored in EEPROM memory in the handheld device, connected via SPI (Serial Peripheral Interface), I2C (Inter-Integrated Circuit), USB or a single wire protocol so that it can be read when the endoscope is inserted into the image processing unit (IPU). The endoscope can have a small amount of non-volatile memory that can be read and written by the IPU during initial device manufacturing. The memory can store an electronically readable serial number that is written to the memory during the manufacturing process. The memory can also store configuration information for each endoscope, such as the endoscope model, serial number, white balance coefficients, lens properties that can be corrected in the IPU, focus parameters, etc. The memory can also be used to store usage information, such as a timestamp or usage time determined by the IPU to prevent the endoscope from being reused after 24 hours. To ensure tamper resistance, the information written to the handle memory can be written under a secure or encrypted protocol used between the IPU and the handle microprocessor.

[0089] The information can be stored as a single piece of data (essentially a serial number, or some other data that semantically equivalently uniquely identifies the endoscope) that can be used as an index key to a database at the server, which in turn has complete data about the endoscope. In some cases, the various operating parameters of the endoscope can be stored in a database at the server, and the model number or serial number can be used as a lookup key to retrieve this configuration data and parameter set. In other cases, the operating parameters can be personalized for each individual endoscope. For example, at the beginning of an arthroscopic shoulder surgery, the IPU can confirm that the endoscope to be used is indeed the appropriate diameter, length, and optical power. These two approaches can be combined so that some parameters are stored based on the model number, while other parameters can be stored based on the endoscope alone.

[0090] Data stored in on-board memory or in a remotely accessible database may include: ●Unique serial number or database lookup key ●Model and version number (integer or ASCII) A textual description of the part model / model identifier that can be displayed on the Control Display (typically a 32-character ASCII string) Calibration / standardization data A complete configuration specification – for example: oThe manufacturer's part number of the image sensor, which may allow many additional properties of the image sensor to be looked up in tables within the IPU, including: oImage sensor dimensions (rows x columns) oFrame rates and frame report rates supported by the sensor oMinimum / maximum integration time and integration time configuration resolution (e.g., 0.1 to 100 ms in 1 ms increments) o Identifier of the illumination source on the endoscope - white, infrared, ultraviolet, individual colors, image plane resolution (720 ×480, 1280×720, etc.). o Identifier of the sensor in the image plane – for example, one bit on / off for each of red, green, blue, ICG infrared, and other colors to be expanded in future software updates o Establish information such as white balance, color correction gamma curve, distortion correction coefficient, sensor color sensitivity, illumination color, etc. o (Boolean) Enable / disable illumination source in the handpiece o (Boolean) Enable / disable defogging heater in the handpiece o (Boolean) Enable / disable rotation sensor in the handpiece o (Boolean) Enable / disable focus control in the handpiece ●Calibration / normalization data – e.g. oCorrection data for lens focal length changes oCorrection coefficients to compensate for image sensor color sensitivity, illumination color, white balance, and distortion correction o LED illumination brightness coefficient. Identifiers for enabling / disabling certain image enhancement parameters based on hardware image configuration - This can be used to pre-configure image processing settings based on the intended imaging application for this endoscope. For example, a bit vector can enable or disable optimization of resolution, contrast, smoothing, and other optical properties. ● Various size and length characteristics, which may be important for controlling water pressure, etc. Manufacturing date ●Date and time of first use Date and time of initial connection and disconnection The length of time the endoscope was used during the procedure

[0091] Storing data in onboard memory (rather than in an off-board database) can improve field adaptability. Onboard data storage can reduce the need for IPU software updates and can improve robustness if the endoscope is used in parts of a hospital or facility without reliable internet access. In some cases, particularly where the endoscope uses interchangeable replaceable parts (e.g., an insertion shaft that is interchangeable on a reusable handle), each individual replaceable part may have a machine-readable serial number.

[0092] Data stored in the controller can be encrypted with a decryption key stored in the IPU. Encryption can improve security by preventing malicious actors from corrupting memory contents or otherwise interfering with the normal operation of the system.

[0093] Data can be communicated in a fixed-field binary protocol or a "keyword=key" protocol (similar to the JSON protocol for web pages).

[0094] The connector can be a standard connector (such as USB-A) or a proprietary connector. Proprietary connectors can ensure that incompatible devices cannot be plugged together. Proprietary connectors can allow for additional pins to support all the required signals and video, such as video signals over twisted pair, higher current to power the heater in the handle, and optical connectors for illuminating the fiber. IV.B. Use machine-readable serial numbers to reduce errors, ensure non-expiration and ensure sterile single use

[0095] A machine-readable serial number can be used to improve the efficiency of starting new procedures. When the endoscope is about to be used, the serial number can be scanned, which can be a two-dimensional optical barcode on the box in the packaging or on the endoscope itself, or scanned by remote sensing (e.g., RFID tag), or the serial number can be read from the EEPROM or similar non-volatile memory on the endoscope itself, for example, when the endoscope is inserted into the IPU. Alternatively, the box or packaging can be printed with information such as product model, batch, manufacturing date, expiration date, and serial number to provide redundancy in case the machine-readable information cannot be read.

[0096] refer to Figure 6A and 6BAs an example of pairing an endoscope with a specific patient and procedure, a member of the surgical staff may request a list of patient names displayed by the computer, for example, patients scheduled for appointments during the next few hours. The staff member may click a button associated with a specific patient to pair that patient with the endoscope. The computer may respond by turning on a camera and asking the staff member to present a matrix (2D) barcode on the endoscope packaging to the camera. When the camera finds the matrix (2D) barcode, the computer may pair the endoscope associated with the matrix (2D) barcode with the patient and their medical record. Once the human patient, the patient's electronic medical record, and the endoscope are paired to each other, one or more of the following operations may be performed: Patient and endoscope database records can be linked to each other. • The database record for an endoscope can be marked to indicate that the endoscope is about to be used. A database can be queried to ensure the endoscope has not been previously paired with a different patient. (In some cases, the software may provide an emergency override, as long as the endoscope has not actually been used on another patient.) The computer may do many validation checks and use constraints: oBased on the machine-readable serial number and the database, the computer can check whether this endoscope was assigned to the facility / location where it was used. Based on the machine-readable serial number and database, the computer can check whether the endoscope to be used is appropriate for the patient’s scheduled procedure. For example, if the endoscope is an arthroscopy and the surgeon is a gastroenterologist, or if the scheduled procedure is a colonoscopy, the computer might issue a warning to the human user and seek confirmation. oThe computer can check that all expiration dates are met and that the endoscope is not affected by recalls or other product actions. • The endoscope's database record may be updated to indicate the storage location of the video, still image, or voice dictation stored in association with the patient's medical case.

[0097] In another example, the computer may first read a two-dimensional barcode or other machine-readable optical code and, based on that code, present a list of patients scheduled for surgery using that form of endoscope, or may request a patient identifier such as name, patient record number, etc.

[0098] When the endoscope is inserted into the IPU for use, the above steps and cross-checks may be performed or repeated. If the endoscope has not been previously paired with a patient, these checks may be performed for the first time and the computer may prompt the human user to provide the patient's identity. In addition, the initial connection to the IPU may result in the following updates to the database of endoscope data in the endoscope's database record: • A symbol may be provided to indicate that the endoscope has been removed from its packaging and is no longer sterile. If an endoscope is sold for single use only, to ensure sterility and reliability, the fact that it has been used may be recorded in the manufacturer's database and / or in the memory of the endoscope itself. This single use may be recorded as a single binary flag that, when set, prohibits further use. First use can be timestamped and / or located so that the endoscope cannot be reused within a certain time period (e.g., two or four hours). This allows the endoscope to be inserted multiple times during a single procedure (e.g., to untie the cable or reset after a power failure) and still be sufficiently protected from reuse.

[0099] When a procedure begins, or when an endoscope is inserted into the IPU, the IPU might run a dialog box to confirm that the endoscope and procedure are compatible. For example, the IPU might query the patient's electronic medical record to confirm the procedure to be performed and whether the attached endoscope is appropriate for that procedure. If a mismatch is detected, the IPU might issue a warning and request confirmation and override. The exact serial number of the endoscope used might be stored in the medical record for review.

[0100] During a procedure, an endoscope or IPU may experience an error or malfunction. In this case, the IPU or endoscope can communicate with a computer or database to initiate a complaint or feedback process with the manufacturer, hospital, or relevant reporting agency. Alternatively, the software can provide the user with the ability to initiate a feedback event. Error codes may be stored in the patient record for review.

[0101] The stored data enables a single IPU to be used for multiple endoscope configurations, reducing the complexity of inventorying, supplying, and using different endoscopes for different purposes.

[0102] If the endoscope is refurbished, the marking can be cleared to allow reuse.

[0103] The machine-readable serial number provides an index into the manufacturer's literature library, providing near-instant access to information if needed during surgery. IV.C. Use machine-readable serial numbers to transfer patient data to the electronic medical record

[0104] refer to Figure 6CPairing an endoscope with a patient can improve the efficiency of entering information into the patient's electronic medical record. During the procedure, the surgeon or assistant can mark all or parts of the video for permanent storage in the patient's electronic medical record or in another database maintained by the hospital / customer or endoscope manufacturer. In some cases, the IPU can calculate the doctor's narration during the operation as speech-to-text. The IPU can connect to a cloud application via Wi-Fi or Ethernet. Images and videos can be sent to the cloud application in real time after each operation or can be stored on a USB drive. Images and videos can be sent to the cloud application as a real-time stream or can be collected in the IPU's storage for periodic uploads, such as at the end of the day. Images and videos can be stored with the patient's electronic medical record in their original form or in a form after some selection and editing. They can be annotated by the doctor or assistant with text or oral narration. Based on the pairing established by a machine-readable serial number, the video or images from the operation can be automatically routed to storage associated with a specific patient (such as the patient's electronic medical record, or a patient portal hosted in a database separate from the hospital or surgeon's electronic medical record).

[0105] The video can be anonymized, edited, and / or sent to the patient in the form of a voice dictation, as described in patent application serial number 16 / 278,112, filed February 17, 2019, which is incorporated by reference. The video can improve the patient's postoperative recovery and provide a report specific to the patient. IV.D. Use machine-readable serial numbers for inventory control, location tracking, reordering, and stock management

[0106] refer to Figure 6D and 6E , the purchaser / hospital can interact with the database to set minimum inventory levels. Alternatively, a computer system accessible to the manufacturer can determine average usage, location-based delivery times, and any pending or in-transit inventory to calculate reorder inventory levels. As each endoscope is used, one or more computers can reduce the existing stock level, and if the reduced level suggests a need for a reorder compared to the reorder stock level, the computer can automatically enter a reorder to maintain the stock at the appropriate level. The computer can be integrated with a distribution center or third party through an API or similar EDI (electronic data interchange) connection to place the reorder quantity. A given user ID can have one or more billing locations and one or more shipping addresses.

[0107] Endoscope packaging is scanned as needed, typically upon arrival at the hospital / purchaser site to enable inventory checks and when inventory is moved from one internal location to another (e.g., a storage room on a different floor or wing). Additionally, the system can use tracking information from UPS, FedEx, or other shippers / logistics management to determine the location of in-transit inventory from the manufacturer through the distribution chain to the hospital / purchaser. The system can use tracked proof of delivery as a signal that the product has been received at the customer site.

[0108] If the system detects that an endoscope appears to have gone missing, the system might issue a warning. For example, the system could calculate the typical inventory time at a given location (perhaps two weeks, for example) and issue a notification if an endoscope hasn't been scanned or moved in a multiple of that time. Similarly, the system could issue warnings about unexpected inventory movements. The system could be programmed to eliminate false positives and over-reporting—for example, a move to a shipping center or through the hospital's internal distribution system could cause an endoscope to take an unexpected route, but this should be suppressed to avoid over-reporting.

[0109] refer to Figure 6F and 6G The system can provide dashboards that deliver real-time data to surgeons, facility managers, and staff, as well as the supply chain, providing immediate information about inventory levels and supply chain flows. For example, the dashboard can show orders placed but not yet received, shipment tracking information, delivery estimates, current on-hand inventory levels, and recent usage patterns. Figure 6F An inventory list of endoscopes currently in stock and ready for use is shown, including model name, model description, batch number (often useful in the event of a product recall), serial number, and expiration date. Figure 6G The most recently used endoscopes are shown - serial number along with the location, time and user. This information can be presented on an aggregate or statistical level, or as a list of individual endoscopes and their current status.

[0110] This tracking can improve utilization and inventory management by ensuring “just in time” ordering.

[0111] The database can store information that tracks the history of the endoscope. If the serial number is remotely scannable (for example, in an RFID tag), the location of the endoscope can be tracked through the distribution channel and in storage at the purchasing hospital. This information can be used to ensure that the endoscope has not exceeded any time limits, has not been stored in a location known to exceed temperature limits, etc. For example, the IPU can enforce a 24-hour limit after first use by reading the time of first use from non-volatile memory on the handle board PCBA. At the beginning of the procedure, the IPU can query over the Internet to confirm that the endoscope has not exceeded the manufacturer's date and that the endoscope is still within specifications and is not subject to any safety recalls. IV.E. Configuration / Capability Information

[0112] Each machine-readable serial number may be associated with information describing a specific configuration and / or capability associated with the endoscope. The serial number can be used to trace back to records of how the device was manufactured and the components used in the manufacturing process. In some cases, a reusable endoscope handle may be designed for use with interchangeable components, and each interchangeable or replaceable component may have a machine-readable serial number, which in turn may be associated with a database record with a specific model number, which may have a list of specific configuration and / or capability information. This information can be checked when pairing an endoscope with a patient, or at similar times when the specific capabilities of a particular endoscope or component need to be compared with the specific needs of a patient. The serial number can be used to associate patients and track inventory. IV.F. Calibration Information

[0113] During manufacturing, certain measurement and calibration data may be measured. For example, white balance, general level settings, color correction gamma curves, distortion correction coefficients, sensor color sensitivity, illumination color, any focus anomalies in different parts of the field of view, etc. This data can be stored in a database indexed by the endoscope's serial number so that the image processing pipeline can compensate. IV.G. Implementation Methods

[0114] A database used to implement the features described above might include tables that track information such as:

[0115] A database might have tables describing endoscopes. One table might describe the properties of a model of an endoscope, properties that are common to endoscopes of a product ID class. A second table might describe individual endoscopes.

[0116] A database might have tables describing surgeries. One table might describe the characteristics of a class of surgeries. A second table might describe a single surgery for a specific patient.

[0117] A database might have tables that describe order and inventory status.

[0118] A database might have a table that describes patients.

[0119] The database may have tables that set up relationships between endoscopes, procedures, patients, surgeons, inventory status, etc.

[0120] Other tables might represent surgeons, surgical specialties, surgical offices, surgical staff, system administrators, addresses, passwords, and similar security information; links to videos, voice dictation, and images in the patient's medical record; procedure lists, access permissions so that patients see only the patient information they are authorized to view, etc.

[0121] refer to Figure 6H , the system architecture can be arranged around the client-server principle. Users can interact with user device client software, such as browser HTML or downloaded applications on mobile phones or desktop computers. On the server side, the front end can be provided by off-the-shelf components such as Amazon AWS, ALB (Application Load Balancer) and / or CloudFront content delivery network. The application server can include components for patient management, inventory management, user ID management, permissions, authentication and authorization management, facility management, surgery management, order management, endoscope management, pairing between endoscopes and patients and surgeries, and other management functions. Relational / SQL databases can provide storage and lookup. V. Examples

[0122] Embodiments of the present invention may include any one or more of the following features, alone or in any combination.

[0123] Endoscope 100 may include a handle and an insertion shaft, with a camera positioned at the distal end of the insertion shaft. The insertion shaft may have solid-state illumination and imaging circuitry at or near its tip, designed to illuminate and image the interior of a body cavity for the surgeon during surgery. The proximal end of the handle may include electronics for driving the illumination circuitry and receiving imaging signals from the imaging circuitry. The proximal handle portion may be designed to allow sterilization between uses. The joint between the proximal handle portion and the insertion shaft may be designed to detachably connect the insertion shaft to the proximal handle portion. When detached, the joint may allow the insertion shaft to be removed for disposal and replacement. The joint may be designed so that, when connected, it transmits mechanical force from the surgeon's hand to the insertion shaft and provides an electrical connection between the proximal handle circuitry and the illumination and imaging circuitry. The handle may have a proximal portion and a distal portion. The distal portion may be located between the insertion shaft and the proximal handle portion. The insertion shaft may be rigidly secured to the distal handle portion. The joint may be arranged to connect and disconnect the distal and proximal portions of the handle. The distal handle portion can be designed to indirectly transmit mechanical force between the surgeon's hand and the insertion shaft and provide an indirect electrical connection between the proximal handle circuitry and the illumination and imaging circuitry. The handle can have a rotating collar with surface features designed to assist the surgeon in rotating the insertion shaft in a roll direction relative to the proximal handle portion about its axis. Electronic components within the proximal handle portion can be designed to sense the roll of the insertion shaft and provide an angular rotation signal designed to straighten the displayed image received from the imaging circuitry. The image sensor can be mounted to allow the image sensor to pan around a pitch or yaw axis perpendicular to the central axis of the insertion shaft. One or more ultraviolet LEDs within the endoscope can be designed to disinfect the interior of the endoscope. A hose for insufflating fluids or gases can be designed on or near the central axis of the proximal handle portion. Two or more insertion shafts of different sizes can be connected to the proximal handle portion at a joint to facilitate use of the proximal handle in surgeries requiring different insertion shafts. A sterilization cabinet can be designed to sterilize components of the endoscope. The insertion shaft of the endoscope tip has a rigid proximal portion and a distal portion. The distal portion is bendable to guide the field of view of the imaging circuit to the desired direction. An illuminator and a solid-state imaging circuit are located at or near the distal end of the hinged distal portion. The illuminator is designed to illuminate the inside of the body cavity for the surgeon during surgery, while the imaging circuit is designed to capture images of the inside of the body cavity. The coupling of the replaceable endoscope tip is designed to detachably connect the insertion shaft to the handle portion at a coupling, and the coupling can be disconnected. The coupling has a mechanical connector. When the coupling is separated, the mechanical connector allows the insertion shaft to be removed from the handle for handling and replacement. When the coupling is connected, the coupling is designed to provide mechanical force transmission between the surgeon's hand and the insertion shaft.An electrical connector is designed to connect the insertion shaft to electronics within the handle. The handle electronics are designed to drive the illuminator and receive imaging signals from the imaging circuitry, and the handle is designed to be sterilized between uses. A control force transmission element is designed to allow the surgeon to steer the imaging circuitry by transmitting surgeon-directed mechanical forces to the articulated distal portion. The distal flexible portion comprises a series of hinged rigid segments. A sheath or covering over the hinged rigid segments is designed to reduce invasiveness or compression. The distal flexible portion is formed from a solid component that is flexible in its transverse and longitudinal dimensions and relatively incompressible in its longitudinal dimension when compressed. The distal flexible portion is extendable from and retractable into the solid sheath. The distal flexible portion is bendable in one dimension. The distal flexible portion is bendable in two orthogonal dimensions. The imaging circuitry is mounted within or near the distal tip of the articulated distal portion via a pannable mounting. The pannable mounting can be designed as two sides of a parallelogram. The imaging circuitry is mounted on structural segments hinged on two sides of the parallelogram. The channels and apertures are designed to deliver irrigation fluid to improve the view of the imaging circuitry through the lens or window. The channels and apertures are designed to expand the surgical cavity by expanding the fluid. The mechanical connector of the coupling includes a twist lock designed to secure the endoscope insertion shaft to the handle portion. Multiple endoscope tips are bundled and packaged together with the handle. The handle contains electronics designed to drive the illuminator and receive imaging signals from the imaging circuitry. The multiple tips and handle are packaged together for integrated shipping and sales. The illuminator is an illuminating LED mounted at or near the distal tip. The illuminator is the emitting end of an optical fiber driven by an illumination source within the handle. The camera 410 can be enclosed in a plastic housing. This plastic housing can be formed as an overmolded sheath designed to protect the camera 410 from bodily fluids and structurally retain the tip components in the operative configuration. The overmolded sheath can be designed to retain the transparent window when the camera 410 is in the operative configuration. Overmolded components can be made of transparent plastic. The overmolded component can be designed to serve as a lens for image sensor 410. Image sensor 410 can be mounted on a flexible circuit board. Flexible circuit board 416 can mount illumination LED 418. LED 418 and image sensor 410 can be mounted on opposite sides of flexible circuit board 416. Image sensor 410 can be protected behind a transparent window. The window can be molded in two thicknesses, with a thinner portion designed to mount and allow illumination light to pass through, and a thicker portion positioned above camera 410. The handle can contain a circuit board with circuitry for controlling and receiving signals from camera 410. The handle and its components can be designed to eliminate the use of metal fasteners and adhesives, unless achieved through overmolding. The endoscope's control buttons can be molded with protrusions that serve as return springs. These protrusions can be adhered to the endoscope handle via a melt process.The circuit board can be overmolded with plastic to encapsulate it and protect it from water. The circuit board can be installed into the handle using a melt process. The components of the handle can be joined together to form a single structure using a melt process. The components of the handle can be joined using a resilient clip designed to secure the two components to each other and then joined together using a melt process. The handle can be constructed from two concentric housings. Rotation of the two housings relative to each other can be controlled by one or more O-rings frictionally engaged with the respective housings. The handle can be overmolded with a high-friction elastomer. The insertion shaft can be connected to the handle via a separable joint. The water joint of the separable joint can be molded to provide an interference seal without the need for O-rings. The water chamber of the separable joint can be designed to create a vortex in water flowing from the handle to the insertion shaft. The insertion shaft can be made of stainless steel and connected to the handle using a separable joint. The plastic components of the endoscope can be joined to the insertion shaft by overmolding the plastic into angled notches in the wall of the insertion shaft, without the use of adhesives. The water joint can be formed as two cones with an interference fit. The cone can be interference fit at the major diameter. The cone can be interference fit by a raised ridge on the lip of the convex cone. The obturator 104 can be designed to penetrate tissue to introduce an endoscope. The feature that twists and locks the obturator 104 into the trocar 102 can be compatible with the feature that twists and locks the endoscope into the trocar.

[0124] A device may include a computer processor and a memory. The processor is programmed to receive video image data from an image sensor at a distal end of an endoscope and display the image data to a surgeon in real time. The processor is programmed to process the image data received from the image sensor using a machine learning model trained to simultaneously upsample the image data to a higher resolution than captured by the image sensor to sharpen edges and enhance local contrast.

[0125] A device may include a computer processor and a memory. The processor is programmed to receive video image data from an image sensor at a distal end of an endoscope and display the image data to a surgeon in real time. The video image data has a frame rate of the image data generated by the image sensor. The processor is programmed to control the image sensor and / or an illumination source designed to illuminate a scene viewed by the image sensor, the control being programmed to underexpose or overexpose every other frame of the video image data. The processor is programmed to process the image data received from the image sensor to combine consecutive pairs of frames of the image data to adjust the dynamic range, thereby enhancing overly bright or dark portions of the image to expose details, and to generate a combined frame at the full frame rate of the video generated by the image sensor.

[0126] A device may include a computer processor and a memory. The device may include a computer processor and a memory. The processor is programmed to receive video image data from an image sensor at a distal end of an endoscope and display the image data to a surgeon in real time. The processor is programmed to sum errors in image intensity relative to a set point intensity. The processor is programmed to simultaneously control at least two of gain, exposure, and illumination using a PID control algorithm to achieve image display at the set point intensity, and to dampen the maximum change in each PID control step to prevent fluctuations.

[0127] Embodiments may include one or more of the following features, alone or in any combination. The processor is further programmed to control the image sensor and / or an illumination source designed to illuminate the scene viewed by the image sensor. The control may be programmed to underexpose or overexpose the video image data every other frame. The processor may also be programmed to process image data received from the image sensor to combine pairs of consecutive frames of image data to adjust the dynamic range, thereby enhancing overly bright or dark portions of the image and revealing details. The processor may also be programmed to generate the combined frames at the full frame rate of the video generated by the image sensor. The processor may also be programmed to sum the error in image intensity relative to a setpoint intensity. The processor may also be programmed to simultaneously control at least two of gain, exposure, and illumination using a PID control algorithm to achieve image display at the setpoint intensity. The maximum change in each step of the PID control may be damped to prevent fluctuations. The processor may also be programmed to process the image data received from the image sensor using a machine learning model trained to simultaneously upsample the image data to a resolution higher than the image sensor capture resolution, sharpen edges, and enhance local contrast. The processor may also be programmed to enhance the video image data using dynamic range compensation. The processor may also be programmed to adjust exposure time, illumination intensity, and / or image capture gain to adjust exposure saturation. The processor may also be programmed to enhance video image data through noise reduction. The processor may also be programmed to enhance video image data through lens correction. In addition to resolution, the processor may also be programmed to enhance at least two of dynamic range compensation, noise reduction, and lens correction. The processor may also be programmed to rotate the image display to compensate for rotation of the endoscope. The processor may also be programmed to adjust exposure time, illumination intensity, and / or gain in image capture to adjust exposure saturation.

[0128] When a medical device is placed into service, a computer reads the machine-readable serial number of the medical device. Based on the serial number read, the computer records a decrease in the on-hand inventory level of the medical device of the category of which the medical device is a member. The computer calculates when the decrease in on-hand inventory of the category indicates a need for a reorder to replenish the on-hand inventory. If warranted, the computer places an order with the supplier of the medical device in the category to replenish the on-hand inventory of the category.

[0129] A computer system comprising one or more processors and a machine-readable non-transitory memory storing instructions. The instructions are programmed to cause the one or more processors to read a machine-readable serial number of a medical device when the medical device is placed into service and, based on the serial number read, record a decrease in the on-hand inventory level of a class of medical devices of which the medical device is a member. The instructions are programmed to cause the one or more processors to calculate when the decrease in on-hand inventory of the class indicates a need for a reorder to replenish the on-hand inventory. The instructions are programmed to cause the one or more processors to place an order with a supplier of the class of medical devices to replenish the on-hand inventory of the class of medical devices.

[0130] The endoscope has an image sensor designed to collect photons to generate a video. The type of image sensor has characteristics that vary from one sensor to another within manufacturing tolerances. The endoscope has a connector for connecting the endoscope to a computer image processor. The image processor is programmed to retrieve data from a database designed to store characteristics and / or behavior of the endoscope's image sensor, the database storing characteristics of a specific sensor or class of sensors. The retrieved data describes the characteristics of the image sensor from the database. The image processor is programmed to calculate a standardized video based on the retrieved characteristics and the video from the image sensor.

[0131] A computer processor is programmed to receive image data from an endoscope's image sensor and retrieve data from a database designed to store characteristics of the endoscope's image sensor and / or its behavior. Image sensors have characteristics that vary from one sensor to another, and the database stores characteristics specific to a particular sensor or class of sensors. The computer calculates a standardized video based on the acquired characteristics and the video from the image sensor.

[0132] The database stores information related to specific individual medical devices, including their physical location and / or ownership. An optical reader reads a 2D identification code on the medical device's packaging. The database record is updated based on the read 2D identification code, indicating the current physical location and / or ownership.

[0133] The computer system has one or more processors and a machine-readable, non-transitory memory storing instructions. The instructions are programmed to cause the one or more processors to store information related to a specific individual medical device, including the physical location and / or ownership. An optical reader reads a two-dimensional identification code on the medical device's packaging. A database record is updated based on the read two-dimensional identification code to indicate the current physical location and / or ownership.

[0134] When an endoscope is placed into service, a computer may read the machine-readable serial number of the endoscope. Based on the serial number read, the existing inventory level of a category of endoscopes may be reduced and recorded. A reorder level may be calculated based on the reduction in the existing inventory level. The computer may calculate when the existing inventory level requires reordering to replenish the existing inventory. Based on the calculated reorder level, the computer may place an order with a supplier of endoscopes in the category to replenish the existing inventory of the category of endoscopes. The medical device may be an endoscope. The endoscope may have an image sensor. The image sensor may be designed to collect photons to generate video. The type of image sensor may have characteristics that vary from one sensor to another within manufacturing tolerances. A program may be programmed to connect one of the medical device or the endoscope to a computer processor. The processor may be programmed to retrieve characteristics of the connected medical device from a database. The database may be designed to store information related to a specific individual medical device, endoscope, or image sensor. The database may be accessed based on the read serial number. The database may store characteristics of the endoscope's image sensor. The database may store characteristics of the image sensor's behavior in the database. The database can store characteristics specific to a particular sensor. The database can provide characteristics of the image sensor of an endoscope from the database and calculate a standardized video based on the acquired characteristics and video from the image sensor. The database can store the physical location of individual medical devices, endoscopes, or components. The database can store the physical location and / or ownership of individual medical devices, endoscopes, or components. An optical reader can read an identification code on the packaging of the medical device. The database record of the physical location and / or ownership can be updated based on the read identification code to indicate the current physical location and / or ownership. When a medical device or endoscope can be connected to its control computer, the computer can be programmed to retrieve the characteristics of the connected device from the database. The identification code can be a machine-readable two-dimensional optical code. The identification code can be stored in a machine-readable electronic memory. The database stores information related to device models of a class of interchangeably manufactured medical devices. The database can be designed to index device model information based on the read identification code of the individual medical device. The database can be designed to store data describing the chromatographic capabilities of the image sensor. The database can be designed to store data describing the image plane resolution of the image sensor. The database can be designed to store calibration characteristics specific to a particular image sensor of a particular endoscope. The image processor can be programmed to calculate a normalized video based on the acquired calibration characteristics and video from the image sensor. The calibration characteristics can describe the image sensor's white balance, the image sensor's color correction gamma curve, the image sensor's distortion correction, or any two or more of these characteristics, or any three or more of these characteristics. The computer can calculate a delivery date prediction based on the read two-dimensional identification code.The computer may update the physical location and / or ownership database record based on the read two-dimensional identification code or the reading of the machine-readable memory to indicate the current physical location.

[0135] Various processes described herein can be realized by general-purpose computers, special-purpose computers and computing devices of appropriate programming. Typically, a processor (e.g., one or more microprocessors, one or more microcontrollers, one or more digital signal processors) will receive instructions (e.g., from a memory or similar device) and execute these instructions to perform one or more processes defined by these instructions. Instructions can be embodied in one or more computer programs, one or more scripts or in other forms. Processing can be performed on one or more microprocessors, central processing units (CPUs), computing devices, microcontrollers, digital signal processors, graphics processing units (GPUs), field programmable gate arrays (FPGAs) or similar devices or any combination thereof. The program for realizing processing and the data operated on it can be stored and sent using various media. In some cases, hard-wired circuits or custom hardware can be used to replace some or all of the software instructions that can realize these processes, or hard-wired circuits or custom hardware can be used in combination with some or all of these software instructions. Algorithms other than the described algorithms can be used.

[0136] Programs and data may be stored in a variety of media suitable for the purpose, or in a combination of different kinds of media that can be read and / or written by a computer, processor, or similar device. The media may include nonvolatile media, volatile media, optical or magnetic media, dynamic random access memory (DRAM), static RAM, floppy disks, flexible disks, hard disks, magnetic tape, any other magnetic media, CD-ROMs, DVDs, any other optical media, punch cards, paper tape, any other physical medium with a pattern of holes, RAM, PROM, EPROM, FLASH-EEPROM, other nonvolatile memory, any other memory chip or cassette, or other storage technology.

[0137] The database may be implemented using a database management system or a temporary storage organization scheme. Alternative database structures to those described may be readily adopted. The database may be stored locally or remotely from the device accessing the data in such a database.

[0138] In some cases, processing may be performed in a network environment that includes computers that communicate with one or more devices (e.g., via a communications network). The computer may communicate directly or indirectly with the devices via any wired or wireless medium (e.g., the Internet, a LAN, WAN or Ethernet, a token ring, a telephone line, a cable line, a radio channel, an optical communication line, a commercial online service provider, a bulletin board system, a satellite communication link, or any combination of the above). Transmission media include coaxial cables, copper wire, and optical fibers 430, including circuits comprising a system bus coupled to the processor. Transmission may occur over the transmission medium or over electromagnetic waves using various protocols at various frequencies, such as via infrared, Wi-Fi, Bluetooth, etc. Each device itself may include a computer or other computing device, such as a computer based on or Centrino TM Those devices of the processor that are suitable for communicating with the computer.Any number and type of devices can communicate with the computer.

[0139] A server computer or centralized authority may or may not be necessary or desirable. In various cases, the network may include or not include a central authority. Various processing functions may be performed on a central authority server, one of several distributed servers, or other distributed devices.

[0140] The following applications are incorporated by reference: U.S. Provisional Application Serial No. 63 / 544,608, filed October 17, 2023, entitled “Artificial Intelligence Agent for Preparing Physician's Note”; U.S. Provisional Application Serial No. 63 / 544,608, filed September 19, 2023, entitled “Image Enhancement for Endoscopy”; U.S. Provisional Application No. 18 / 370,375, entitled "Endoscope"; U.S. Provisional Application No. 63 / 538,485, filed on September 14, 2023, entitled "Endoscope"; U.S. Provisional Application No. 63 / 534,855, filed on August 27, 2023, entitled "Endoscope"; U.S. Provisional Application No. 63 / 531,239, filed on August 7, 2023, entitled "Endoscope"; U.S. Provisional Application No. 63 / 437,115, filed on January 4, 2023, entitled "Endoscope with Identification and Configuration Information"; U.S. Provisional Application No. 63 / 437,115, filed on January 4, 2023, entitled "Illumination for Endoscopes"; U.S. Provisional Application No. 63 / 538,485, filed on September 14, 2023, entitled "Endoscope"; U.S. Provisional Application No. 63 / 534,855, filed on August 27, 2023, entitled "Endoscope"; U.S. Application No. 17 / 954,893, filed September 20, 2022, entitled Super Resolution for Endoscope Visualization; U.S. Provisional Application No. 63 / 376,432, entitled "Visualization" filed on August 26, 2022; U.S. Provisional Application No. 17 / 896,770, entitled "Endoscope," filed on August 25, 2022; U.S. Provisional Application No. 63 / 400,961, entitled "Endoscope," filed on August 25, 2022; U.S. Provisional Application No. 17 / 824,857, entitled "Endoscope," filed on May 25, 2022; U.S. Provisional Application No. 63 / 249,479, entitled "Endoscope," filed on September 28, 2021; U.S. Provisional Application No. 63 / 237,906, entitled "Endoscope," filed on August 27, 2021; and U.S. Provisional Application No. 63 / 290,961, entitled "Endoscope with Flexible Camera Shaft," filed on June 29, 2021. U.S. Application No. 17 / 361,711 for a device with a Bendable Camera Shaft;U.S. Provisional Application No. 63 / 214,296, filed on June 24, 2021, entitled Endoscope with Bendable Camera Shaft; U.S. Provisional Application No. 63 / 193,387, filed on June 24, 2021, entitled Anti-adhesive Window or Lens for Endoscope Tip; U.S. Provisional Application No. 63 / 067,781, filed on August 19, 2020, entitled Endoscope with Articulated Camera Shaft; U.S. Provisional Application No. 63 / 047,588, filed on July 2, 2020, entitled Endoscope with Articulated Camera Shaft; U.S. Provisional Application No. 63 / 067,781, filed on August 19, 2020, entitled Endoscope with Articulated Camera Shaft; U.S. Provisional Application No. 63 / 047,588, filed on July 2, 2020, entitled Endoscope with Articulated Camera Shaft; U.S. Provisional Application No. 63 / 067,781, filed on August 19, 2020, entitled Endoscope with Articulated Camera Shaft; U.S. Provisional Application No. 63 / 046,665, filed on June 7, 2019, entitled Endoscope with Disposable Camera Shaft and Reusable Handle; U.S. Provisional Application No. 62 / 850,326, filed on May 20, 2019, entitled Endoscope with Disposable Camera Shaft; U.S. Provisional Application No. 16 / 069,220, filed on October 24, 2018, entitled Anti-Fouling Endoscopes and Uses Thereof; U.S. Provisional Application No. 16 / 434,766, filed on June 7, 2019, entitled Endoscope with Disposable Camera Shaft and Reusable Handle; U.S. Provisional Application No. 62 / 850,326, filed on May 20, 2019, entitled Endoscope with Disposable Camera Shaft; U.S. Provisional Application No. 16 / 069,220, filed on October 24, 2018, entitled Anti-Fouling Endoscopes and Uses Thereof; U.S. Provisional Application No. 16 / 069,220, filed on August 23, 2018, entitled Endoscope with Disposable Camera Shaft U.S. Provisional Application No. 62 / 722,150, filed on June 8, 2018, entitled Endoscope with Disposable Camera Shaft; U.S. Provisional Application No. 62 / 682,585, filed on June 8, 2018, entitled Endoscope with Disposable Camera Shaft.

[0141] For clarity of explanation, the above description focuses on a representative sample of all possible embodiments, teaches the principles of the invention, and conveys a sample of the best modes contemplated for carrying out the invention. The invention is not limited to the described embodiments. To avoid unnecessarily obscuring the principles related to the claimed invention, well-known features may not be described in detail. Throughout this application and its related document history, when the term "invention" is used, it refers to the entire collection of ideas and principles described; in contrast, the formal definition of exclusively protected property rights is set forth in the exclusively controlling claims. This description does not attempt to be exhaustive of all possible variations. Other variations or modifications not described are also possible. Where multiple alternative embodiments are described, in many cases it will be possible to combine elements of different embodiments, or to combine elements of the embodiments described herein with other modifications or variations not expressly described. Unless expressly stated otherwise, the listing of items does not imply that any or all items are mutually exclusive, nor does it imply that any or all items are comprehensive in any way. In many cases, a feature or group of features can be used separately from the entire described device or method. Many undescribed alternatives, variations, modifications, and equivalents are within the literal scope of the following claims, and others are equivalent. The claims may be practiced without some or all of the specific details described in the specification.In many cases, the method steps described in this specification may be performed in an order different from that presented in this specification, or performed in parallel rather than sequentially.

Claims

1. A method comprising the following steps: reading, by a computer, a machine-readable serial number of the medical device when the medical device is placed into service, and recording a reduction in an on-hand inventory level of medical devices of a class of which the medical device is a member based on the reading of the serial number; calculating, by a computer, when said reduction in on-hand inventory of said category indicates a need for reordering to replenish said on-hand inventory level; as well as An order is placed by the computer with a supplier of the category of medical devices based on the calculated reordering need to replenish an existing inventory of the category of medical devices.

2. The method according to claim 1: One of the medical devices is connected to a computer processor, which is programmed to retrieve characteristics of the connected medical device from a database designed to store information related to a specific individual medical device, the database being accessed based on the read serial number.

3. The method according to claim 2: The medical device is an endoscope having an image sensor designed to collect photons to produce video, the image sensor being of a type having characteristics that vary from one sensor to another within manufacturing tolerances; And the method further comprises the following steps: storing characteristics of the endoscope's image sensor and / or behavior of the image sensor in a database, the database storing characteristics specific to a particular sensor; as well as The image processor is programmed to retrieve the characteristics of an endoscope's image sensor from the database and calculate normalized video based on the retrieved characteristics and the video from the image sensor.

4. The method according to claim 2: The database is further designed to store the physical location and / or ownership of individual medical devices; And the method further comprises the following steps: reading an identification code on the packaging of the medical device by an optical reader; as well as Based on the read identification code, the physical location and / or ownership database record is updated to indicate the current physical location and / or ownership.

5. The method according to claim 4: The medical device is an endoscope having an image sensor designed to collect photons to produce video, the image sensor being of a type having characteristics that vary from one sensor to another within manufacturing tolerances; And the method further comprises the following steps: storing characteristics of the endoscope's image sensor and / or behavior of the image sensor in a database, the database storing characteristics specific to a particular sensor; as well as The image processor is programmed to retrieve the characteristics of an image sensor of an endoscope from the database and calculate a normalized video based on the retrieved characteristics and the video from the image sensor.

6. The method according to claim 2, wherein: The identification code is a machine-readable two-dimensional optical code.

7. The method according to claim 2, wherein: The identification code is stored in a machine-readable electronic memory.

8. The method according to claim 7, wherein: The identification code is stored in a machine-readable non-volatile memory.

9. The method according to claim 2, wherein: The database stores information related to device models of medical devices manufactured in an interchangeable category, and the database is designed to index the device model information based on a read identification code of an individual medical device.

10. A computer system comprising: one or more processors; as well as a machine-readable non-transitory memory having stored therein instructions programmed to cause the one or more processors to: reading a machine-readable serial number of a medical device when the medical device is placed into service, and recording a reduction in an on-hand inventory level of medical devices of a class of which the medical device is a member based on the reading of the serial number; calculating when said reduction in on-hand inventory of said category indicates a need for reordering to replenish said on-hand inventory level; as well as Based on the calculated need for reordering, an order is placed with a supplier of the category of medical devices to replenish an existing inventory of the category of medical devices.

11. The computer system according to claim 10: The medical device is an endoscope having an image sensor designed to collect photons to produce video, the image sensor being of a type having characteristics that vary from one sensor to another within manufacturing tolerances; The instructions are further programmed to cause the one or more processors to: connecting one of the medical devices to a computer processor, the processor being programmed to retrieve characteristics of the connected medical device from a database designed to store information related to a specific individual medical device, the database being accessed based on the read serial number; storing characteristics of the endoscope's image sensor and / or behavior of the image sensor in a database, the database storing characteristics specific to a particular sensor; as well as The characteristics of an image sensor of an endoscope are obtained from the database, and a normalized video is calculated based on the obtained characteristics and the video from the image sensor.

12. The computer system according to claim 10: The database is further designed to store the physical location and / or ownership of individual medical devices; The instructions are further programmed to cause the one or more processors to: directing an optical reader to read an identification code on the packaging of said medical device and connecting one of said medical devices to a computer processor, said processor being programmed to retrieve characteristics of said connected medical device from a database designed to store information associated with a particular individual medical device, said database being accessed based on the read serial number; Based on the read identification code, the physical location and / or ownership database record is updated to indicate the current physical location and / or ownership.

13. An endoscope comprising: an image sensor designed to collect photons to produce video, the image sensor being of a type having characteristics that vary from one sensor to another within manufacturing tolerances; A connector for connecting the endoscope to a computer image processor, the image processor being programmed to: acquiring data from a database designed to store characteristics of an image sensor of the endoscope and / or behavior of the image sensor, the database storing characteristics specific to a particular sensor or a particular class of sensors; obtaining data describing characteristics of the image sensor from the database, and Based on the acquired characteristics and the video from the image sensor, a normalized video is calculated.

14. The endoscope of claim 13, further designed to be connected to a computer processor, the computer processor being programmed to: reading a machine-readable serial number of the endoscope when the endoscope is placed into service, and recording a reduction in an on-hand inventory level of endoscopes of a class of which the endoscope is a member based on the reading of the serial number; calculating when said reduction in on-hand inventory of said category indicates a need for reordering to replenish said on-hand inventory level; as well as Based on the calculated need for reordering, an order is placed with a supplier of the category of endoscopes to replenish an existing inventory of the category of endoscopes.

15. The endoscope according to claim 13: The database is further designed to store information related to specific individual medical devices, including physical location and / or ownership; reading a two-dimensional identification code on the packaging of the medical device by an optical reader; and Based on the read two-dimensional identification code, the physical location and / or ownership database record is updated to indicate the current physical location and / or ownership.

16. The endoscope according to claim 13, wherein the database is designed to store data describing the chromatographic capabilities of the image sensor.

17. The endoscope according to claim 13, wherein the database is configured to store data describing an image plane resolution of the image sensor.

18. The endoscope according to claim 13: obtaining data specific to the particular image sensor of the particular endoscope from a database designed to store calibration characteristics of the image sensor of the endoscope; The image processor is programmed to calculate a normalized video based on the acquired calibration characteristics and the video from the image sensor.

19. The endoscope according to claim 18, wherein The calibration characteristics describe the white balance of the image sensor.

20. The endoscope according to claim 18, wherein The calibration characteristics describe a color correction gamma curve of the image sensor.

21. The endoscope according to claim 18, wherein The calibration characteristics describe a distortion correction of the image sensor.

22. A method comprising the steps of: at a computer processor, the computer processor being programmed to receive image data from an image sensor of the endoscope, obtaining data from a database designed to store characteristics of the image sensor of the endoscope and / or behavior of the image sensor, the image sensor being of a type having characteristics that vary from one sensor to another, the database storing characteristics specific to a particular sensor or class of sensors; At the computer, a normalized video is calculated based on the acquired characteristics and the video from the image sensor.

23. The method according to claim 22, further comprising the steps of: reading, at a computer, a machine-readable serial number of the endoscope when the endoscope is placed into service, and recording a reduction in an on-hand inventory level of endoscopes of a class of which the endoscope is a member based on the reading of the serial number; calculating when said reduction in on-hand inventory of said category indicates a need for reordering to replenish said on-hand inventory level; as well as An order is placed by the computer with a supplier of the category of endoscopes based on the calculated need for reordering to replenish an existing inventory of the category of endoscopes.

24. The method according to claim 22: The database is further designed to store information related to specific individual medical devices, including physical location and / or ownership; reading a two-dimensional identification code on the packaging of the medical device by an optical reader; and Based on the read two-dimensional identification code, the physical location and / or ownership database record is updated to indicate the current physical location and / or ownership.

25. The method according to claim 22, further comprising the steps of: obtaining data specific to the particular image sensor of the particular endoscope from a database designed to store calibration characteristics of the image sensor of the endoscope; The image processor is programmed to calculate a normalized video based on the acquired calibration characteristics and the video from the image sensor.

26. The method according to claim 25, wherein The calibration characteristics describe the white balance of the image sensor.

27. A method comprising the steps of: In a database, storing information related to specific individual medical devices, including physical location and / or ownership; reading a two-dimensional identification code on the packaging of the medical device by an optical reader; Based on the read two-dimensional identification code, the physical location and / or ownership database record is updated to indicate the current physical location and / or ownership.

28. The method according to claim 27, further comprising the steps of: reading, by a computer, a machine-readable serial number of the medical device when the medical device is placed into service, and recording a reduction in an on-hand inventory level of medical devices of a class of which the medical device is a member based on the reading of the serial number; calculating, by a computer, when said reduction in on-hand inventory of said category indicates a need for reordering to replenish said on-hand inventory level; as well as An order is placed by the computer with a supplier of the category of medical devices based on the calculated reordering need to replenish an existing inventory of the category of medical devices.

29. The method according to claim 27: The medical device is an endoscope having a corresponding image sensor designed to collect photons to produce video, the image sensor being of a type having characteristics that vary from one sensor to another within manufacturing tolerances; And the method further comprises the following steps: storing characteristics of the endoscope's image sensor and / or behavior of the image sensor in a database, the database storing characteristics specific to a particular sensor; as well as One of the endoscopes is connected to the computer image processor, the image processor being programmed to retrieve the characteristics of the image sensor of the one endoscope from the database and to calculate a normalized video based on the retrieved characteristics and the video from the image sensor.

30. The method according to claim 27, further comprising the steps of: For medical devices between the sales time and the delivery time, a prediction of the delivery date is calculated based on the read two-dimensional identification code.

31. The method according to claim 27, further comprising the steps of: In a database, information related to a specific individual medical device is stored, including the physical location; reading a two-dimensional identification code on the packaging of the medical device by an optical reader; Based on the read two-dimensional identification code, the physical location and / or ownership database record is updated to display the current physical location.

32. The method according to claim 27, further comprising the steps of: In the database, information related to specific individual medical devices is stored, including ownership; reading a two-dimensional identification code on the packaging of the medical device by an optical reader; Based on the read two-dimensional identification code, the physical location and / or ownership database record is updated to indicate current ownership.

33. A computer system comprising: one or more processors; as well as a machine-readable non-transitory memory having stored therein instructions programmed to cause the one or more processors to: In a database, storing information related to specific individual medical devices, including physical location and / or ownership; reading a two-dimensional identification code on the packaging of the medical device by an optical reader; and Based on the read two-dimensional identification code, the physical location and / or ownership database record is updated to indicate the current physical location and / or ownership.

34. The computer system of claim 33, wherein the instructions are further programmed to cause the one or more processors to: reading a machine-readable serial number of a medical device when the medical device is placed into service, and recording a reduction in an on-hand inventory level of medical devices of a class of which the medical device is a member based on the reading of the serial number; calculating when said reduction in on-hand inventory of said category indicates a need for reordering to replenish said on-hand inventory level; as well as Based on the calculated need for reordering, an order is placed with a supplier of the category of medical devices to replenish an existing inventory of the category of medical devices.

35. The computer system of claim 33: The medical device is an endoscope having a corresponding image sensor designed to collect photons to produce video, the image sensor being of a type having characteristics that vary from one sensor to another within manufacturing tolerances; The instructions are further programmed to cause the one or more processors to: storing characteristics of the endoscope's image sensor and / or behavior of the image sensor in a database, the database storing characteristics specific to a particular sensor; and One of the endoscopes is connected to a computer image processor, the image processor being programmed to retrieve the characteristics of the image sensor of the one endoscope from the database and to calculate a normalized video based on the retrieved characteristics and the video from the image sensor.

36. A method comprising the steps of: In a database, information is stored relating to a particular individual endoscope having a corresponding image sensor of a type having characteristics that vary from one sensor to another within manufacturing tolerances, the stored information including: Recording information on characteristics of an individual endoscope's image sensor and / or behavior of said image sensor; and the physical location and / or ownership of individual endoscopes; reading the two-dimensional identification code on the packaging of the endoscope by an optical reader, and updating the physical location and / or ownership database record based on the read two-dimensional identification code to indicate the current physical location and / or ownership; When the corresponding endoscope is put into use, the machine-readable serial number of the endoscope is read by a computer; and recording a reduction in an on-hand inventory level of endoscopes of a category of which the endoscope is a member based on the reading of the serial number; Based on the read serial number, recording the relationship between the endoscope of the serial number and the patent of the program to be used for the endoscope; calculating, by a computer, when said reduction in on-hand inventory of said category indicates a need for reordering to replenish said on-hand inventory level; and placing an order, by the computer, based on the calculated need for reordering, with a supplier of the category of endoscopes to replenish the existing stock of the category of endoscopes, Based on the read serial number, recorded characteristics of the image sensor of the endoscope are retrieved from the database, and a normalized video is calculated based on the retrieved characteristics and the video from the image sensor.

Images