Endoscope camera shooting system capable of being used for vocal cord mucosa wave camera shooting

Abstract

The invention discloses an endoscope camera shooting system capable of being used for vocal cord mucous membrane wave camera shooting, and relates to the technical field of endoscope camera shooting. A main control unit of the system automatically detects a light pulse time point and calculates the light pulse frequency, and automatically adjusts and controls the exposure of an image sensor according to the stroboscopic light pulse frequency, so that the exposure intensity of each frame of image is consistent; the problem of video flicker caused by inconsistent exposure of a common global shutter camera is solved, exposure starting and stopping are synchronously controlled according to light pulse time points, and all light pulses are included in the exposure time range as much as possible. The problem of image brightness caused by insufficient exposure of an image sensor due to insufficient light incidence under a stroboscopic pulse light source is solved to the maximum extent, whether the endoscope light source is a stable light source or a stroboscopic light source is automatically judged, and the working mode is automatically switched. And vocal cord mucous membrane waves can be shot under the stroboscopic light source condition.

Description

Technical Field

[0001] The present invention relates to the technical field of endoscopic imaging, and more specifically, it relates to an endoscopic imaging system that can be used for vocal cord mucosal wave imaging. Background Art

[0002] Dynamic laryngoscopy, also known as stroboscopic laryngoscopy, is mainly used to observe the vibration characteristics of the vocal cords during phonation. It is the only examination that can see the movement pattern of the vocal cord mucosal wave. The changes in the mucosal wave directly reflect the histological changes of the double vibrators of the vocal cords and are a sensitive indicator of whether there are pathological changes in the vocal cords. It can observe the vibration law of the vocal cords and provide an objective basis for the diagnosis of vocal cord diseases (such as vocal cord cysts, early vocal cord cancer) and the comparison before and after vocal cord surgery. Therefore, dynamic laryngoscopy is of great significance in the fields of laryngology, voice medicine, and art voice medicine. Its principle is that when a person vocalizes, the frequency is relatively high and the vocal cords vibrate relatively fast, making it difficult for the naked eye to observe the true situation of the vocal cord vibration. Therefore, to observe the true situation of the vocal cord vibration in detail, a certain method is needed to slow down the rapid vibration of the vocal cords relatively. This method is the examination method of dynamic laryngoscopy. The examination method of dynamic laryngoscopy is basically the same as that of a straight tube magnifying laryngoscope, and the difference is that the light source is a pulsed flash light source, that is, the straight tube magnifying laryngoscope and the cold light source of the flash laryngoscope are connected through an optical fiber. The imaging system can record the true vibration state of the vocal cords, and then through video playback, the vibration of the vocal cords and the conditions of laryngeal lesions can be observed in detail. When vocalizing, due to the extremely fast vibration of the vocal cords, the true closed state of the glottis cannot be determined with an ordinary laryngoscope and can only be determined under dynamic laryngoscopy. The shooting speed of ultra-high-speed imaging equipment can reach more than 1000 frames per second. Ultra-high-speed cine photography can observe the vibration of the vocal cords more carefully, but due to factors such as high cost and inoperability on some patients, it cannot be widely used clinically. With the wide application of LED light sources in the field of endoscopes, the technology of using an LED cold light source to generate a pulsed stroboscopic light source and using an endoscopic camera to shoot the vibration of the vocal cords has been more studied and applied.

[0003] Ordinary endoscopic cameras mostly adopt the rolling shutter exposure scheme (row scanning exposure) for cost considerations. Under static light source conditions, when shooting static objects, the images obtained by the rolling shutter exposure scheme are clear and stable, meeting the requirements of most medical endoscopic imaging. However, when shooting moving objects, the "jelly effect" will occur, and the images will be distorted. When the light source has a flickering phenomenon, stripes will appear due to different exposure brightness in each row.

[0004] In a global exposure image sensor, all pixel points complete exposure in the same time period. However, when an ordinary global shutter exposure mode camera shoots a vibrating vocal cord, since the light source is pulsed, the effective illumination time is only about 10%. Whether using automatic exposure or timed exposure, because it is impossible to maintain a uniform exposure amount, the exposure amount of each frame of the captured image is different, and the output image is bright and dark, with obvious flicker visible to the naked eye, and the image quality is difficult to meet the requirements of vocal cord vibration inspection. Summary of the Invention

[0005] Aiming at the deficiencies of the prior art, the present invention discloses "an endoscopic imaging device that can be used for mucosal wave imaging". By detecting the change in incident light intensity to determine whether it is operating in a stroboscopic light source mode, calculating the emission frequency of the stroboscopic pulsed light source, and controlling the exposure start and end time points of the image sensor to control the exposure amount of each frame, the exposure amount of each frame of the image is made consistent, solving the problem of stripe or flicker in the captured image when the existing medical endoscopic imaging device is used for vocal cord vibration inspection. It can be used both as an ordinary endoscopic imaging device and for vocal cord mucosal wave imaging.

[0006] To achieve the above object, the present invention provides the following technical solutions: An endoscopic imaging system that can be used for vocal cord mucosal wave imaging, comprising an AC-DC switching power supply module, a DC-DC power supply module, a main control unit, an exposure light intensity detection circuit, a key input circuit, an image sensor, an image signal processor, a character overlay unit, and a video output expansion interface; The exposure light intensity detection circuit is a light intensity detection circuit with a visible light photosensitive diode as the core. The photosensitive diode is structurally placed close to the image sensor, and the light-receiving surface is parallel to the image sensor, and can receive the reflected light and stray light incident on the image sensor area; it is connected in series with a fixed-value resistor and connected to a fixed voltage, and the voltage at the series connection point will change synchronously with the light intensity of the light-receiving surface of the photosensitive diode. The main control unit automatically determines the exposure working mode according to the input result of the exposure light intensity detection circuit, and transmits the determined exposure working mode to the image signal processor through a serial data communication port. The image signal processor sets the internal register parameters of the image sensor through a serial data communication port according to the received exposure mode instruction and starts the corresponding exposure mode; The image sensor outputs an image at a predetermined frame frequency and exposure integration time, and the external trigger exposure mode is controlled by the input pin level outside the image sensor to control the exposure start and end times, and the exposure frequency and frame exposure time are controlled by the input signal of its specific pin; The main control unit calculates the number of light pulses for each frame exposure according to the time period of the light pulse and controls the exposure frequency.

[0007] Further, connect the series connection point of the photosensitive diode and the series resistor to the AD input interface of the main control unit, and the main control unit can detect the change of the exposure light intensity through the AD conversion function.

[0008] Further, when no light irradiates the light-receiving surface of the photosensitive diode, the dark current is extremely small, showing a high-resistance characteristic. When light irradiates its light-receiving surface, the passing current increases sharply, showing a low-resistance characteristic.

[0009] Compared with the prior art, the present invention has the following beneficial effects: Since the present invention adopts a global shutter image sensor, it has a better dynamic image effect than a rolling shutter image sensor with the same clarity level, and solves the problem of image water ripples of a rolling shutter camera under the illumination condition of a stroboscopic pulsed light source; The main control unit automatically detects the light pulse time point and calculates the light pulse frequency, and automatically adjusts and controls the exposure amount of the image sensor according to the frequency of the stroboscopic light pulse, so that the exposure intensity of each frame of image is consistent, and solves the problem of video flickering caused by inconsistent exposure amounts of ordinary global shutter cameras; Synchronously control the start and stop of exposure according to the light pulse time point, and include all light pulses within the exposure time range as much as possible, and make up for the problem of insufficient picture brightness caused by insufficient exposure of the image sensor due to insufficient light injection under the stroboscopic pulsed light source to the greatest extent; Automatically judge whether the endoscope light source is a stable light source or a stroboscopic light source and automatically switch the working mode, which can be used as an ordinary medical endoscope camera and can also capture vocal cord mucosal waves under the condition of a stroboscopic light source. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 It is a system principle block diagram of an endoscope imaging system that can be used for vocal cord mucosal wave imaging; Figure 2 It is a working timing diagram of the external trigger mode of the image sensor. DETAILED DESCRIPTION OF THE INVENTION

[0011] As Figure 1 shown, an endoscope imaging system that can be used for vocal cord mucosal wave imaging, the present invention includes an AC-DC switching power supply module, a DC-DC power supply module, a main control unit, an exposure light intensity detection circuit, a key input circuit, an image sensor, an image signal processor, a character overlay unit, a video output expansion interface, etc.

[0012] The AC-DC switching power supply module converts the low-voltage alternating current input from the AC power grid into a low-voltage direct current power supply for the system to work by means of transformer isolation.

[0013] The DC-DC power module converts the low-voltage DC power output by the AC-DC switching power module into an even lower DC operating voltage to supply the MCU, image sensor, etc. as the operating power source.

[0014] The main control unit is a 32-bit programmable high-speed single-chip microcomputer with built-in software, equipped with multiple groups of data serial communication ports, digital I / O, AD inputs, built-in timers, etc., and can provide precise system clock control through an external crystal oscillator.

[0015] The exposure light intensity detection circuit is a light intensity detection circuit centered around a visible light photosensitive diode. Structurally, the photosensitive diode is placed close to the image sensor, with its light-receiving surface parallel to the image sensor, and can receive the reflected light and stray light incident on the image sensor area; when no light irradiates the light-receiving surface of the photosensitive diode, the dark current is extremely small, presenting a high-resistance characteristic, and when light irradiates its light-receiving surface, the passing current increases sharply, presenting a low-resistance characteristic. Connect it in series with a fixed-value resistor and connect it to a fixed voltage, and the voltage at the series connection point will change synchronously with the light intensity of the light-receiving surface of the photosensitive diode; due to the special structural characteristics of the endoscope camera system, when there is no specific light source providing illumination, the incident light amount on the camera image sensor is extremely small, and the light intensity sensed by the photosensitive diode is also very small. When a stable endoscope cold light source is used for illumination, the light intensity sensed by the photosensitive diode changes little. When a laryngoscope stroboscopic light source provides pulsed light illumination, the light intensity detection circuit centered around the photosensitive diode can detect the rapid bright-dark changes. The response time of the photosensitive diode is generally in the order of dozens of nanoseconds, while the light pulse width is at least 100 μs. Connect the series connection point of the photosensitive diode and the series resistor to the AD input interface of the main control unit, and the main control unit can detect the change in exposure light intensity through the AD conversion function.

[0016] The key input circuit consists of multiple tactile keys and resistors, connected to the digital I / O ports of the main control unit. The main control unit determines whether a key is pressed through periodic scanning, and the main control unit realizes functions such as white balance and function selection according to the predefined functions of each key, and modifies the relevant parameters of video image processing, such as brightness, gain, sharpness, RGB color compensation, etc. The main control unit sends the relevant parameters of the currently set video image processing to the image signal processor and the character overlay unit through the serial data communication port.

[0017] The image sensor is a color high-definition global shutter image sensor, which not only has the function of setting the timing exposure mode through internal registers but also has the function of starting and stopping exposure triggered by pin levels. The preferred model is the Sony IMX265 high-definition image sensor.

[0018] The image signal processor is a high-speed DSP or FPGA, a programmable data processor. As the host, the image signal processor sets the working mode and parameters of the image sensor through a serial communication port, receives the image data uploaded by the image sensor, etc., and can control the start and stop of the exposure of the image sensor through the IO port. The image signal processor also acts as a slave, receiving the video control parameters downloaded by the main control unit through the serial data communication port and receiving the start and stop exposure instructions from the central control unit through the IO port.

[0019] The video output expansion interface is electrically connected to the image signal processor, receives the video image output by the image signal processor, and converts it into multiple video formats such as DVI and VGA for output.

[0020] The main control unit is connected to the image signal processor through a serial communication port and a digital IO port. The image signal processor is connected to the image sensor through a serial communication port and a digital IO port. The image signal processor is connected to the video output expansion interface through a video data communication interface.

[0021] Based on the input result of the exposure light intensity detection circuit, the main control unit's built-in software automatically determines whether the imaging device operates in the timed exposure mode suitable for stable light sources or the external trigger exposure mode suitable for strobe light sources, and transmits the determined exposure working mode to the image signal processor through the serial data communication port. The image signal processor sets the internal register parameters of the image sensor through the serial data communication port according to the received exposure mode instruction and starts the corresponding exposure mode.

[0022] The timed exposure mode is a commonly used exposure mode for endoscopic devices. The image sensor outputs images at a pre-determined frame rate and exposure integration time. The external trigger exposure mode is controlled by the input pin level outside the image sensor to control the start and end time of exposure, and the exposure frequency and frame exposure time are controlled by its specific pin input signals.

[0023] Determination of the number of exposure pulses: The exposure frequency of the imaging device is generally between 30 and 60 frames. When the doctor examines the patient's vocal cords, the patient will be required to emit a stable low-frequency long sound with a frequency range between 100 and 400 Hz, that is, the frequency of the strobe light is generally between 100 and 400 Hz, and the light pulse duty ratio is about 10%. In order to make the amount of light incident on each frame of exposure consistent, the main control unit calculates the number of light pulses for each frame of exposure according to the time period of the light pulse and controls the exposure frequency between 30 and 60 frames. Each exposure time covers at least two light pulses. For example, when the light pulse frequency is 100 Hz - 150 Hz, the exposure time for each frame accounts for three light pulses, and when the light pulse frequency is 150 - 200 Hz, the exposure time for each frame accounts for 4 light pulses. When the light pulse frequency is 400 - 500 Hz, each frame is exposed to 10 light pulses; in this way, the brightness of the image for each frame of exposure is basically the same, solving the problem of inconsistent brightness of each frame of image in the timed exposure mode under the strobe light source mode.

[0024] As Figure 2 shown, the explanations are as follows: A operation: The incident light intensity of the image sensor detected by the photosensitive diode. The high level is the effective time of the light pulse, and the low level is the off-light pulse time; B operation: External trigger for image sensor exposure (low effective) C operation: Exposure integration time (ripple section) D operation: Preparation and upload of the previous frame of image data (shadow section) Exposure mode control and working timing description: 1. After power-on, the built-in software of the main control unit defaults the current light source to a common non-stroboscopic light source. The main control unit notifies the image signal processor to capture images in the timed exposure mode through the digital communication port and digital IO port. At the same time, the built-in software of the main control unit samples the output of the exposure light intensity detection circuit at a frequency of 200K and caches the AD results into the internal RAM space; 2. When the device works in the timed exposure mode, the built-in software of the main control unit continuously judges the AD results input by the exposure light intensity detection circuit. Whether the change in the incident light intensity of the image sensor conforms to the incident light characteristics of the stroboscopic light source. If it conforms, the main control unit immediately sets the image signal processor to enter the external trigger exposure working mode through the serial data communication port. The image signal processor sets the image sensor to enter the external trigger exposure working mode. After the setting is completed, the main control unit issues an instruction to start the exposure during the off-light pulse period. The main control unit program starts timing from the rising edge of the first light pulse until the time after the rising edge of the next light pulse as the current light pulse period. Taking the number of ADs as the timing unit, each AD number is 5us, and it is converted into the light pulse frequency. According to the light pulse frequency, the number of light pulses for each frame of exposure is determined to control the exposure frame rate at 30 - 60Hz. For example, when the light pulse frequency is 100Hz, two complete light pulses are used for each exposure; The built-in software of the main control unit continuously performs exposure intensity detection and exposure trigger start / stop control output according to the timing of the "Image Sensor External Trigger Mode Working Timing Diagram", and automatically adjusts the number of exposure light pulses for each frame; 3. When the device works in the external trigger exposure mode, the built-in software of the main control unit continuously judges the AD results input by the exposure light intensity detection circuit. When the change in the light pulse disappears and remains long bright or long dark for more than 50ms, it indicates that the current light source has changed to a steady light source. The main control unit immediately sets the image signal processor to enter the timed exposure working mode through the serial data communication port. The image signal processor sets the image sensor to enter the timed exposure working mode; 4. In the timed exposure working mode, the main control unit works according to the above "2"; 5. When the camera uses the laryngoscope stroboscopic light source as the illumination source, the main control unit sends an exposure signal to the image signal processor through the digital IO port with a specific time advance before the light pulse is emitted. After obtaining this signal in an interrupt manner, the image signal processor immediately sends an exposure start signal to the image sensor through the digital IO port. After receiving the exposure start signal, the image sensor starts the exposure after a system working delay; 6. External trigger for exposure stop: The main control unit calculates the number of light pulses for each exposure according to the current light pulse frequency and determines the exposure stop time based on this number of pulses. When the exposure duration reaches the calculated number of light pulses, the main control unit pulls up the corresponding digital IO port to notify the image signal processor to end the current frame exposure; after receiving the exposure stop signal, the image signal processor stops the current frame exposure after a system working delay, and the image sensor caches the current frame data and then starts the frame data upload operation; 7. When the main control unit starts the exposure, a certain time advance is reserved compared to the emission of the light pulse to ensure that the image sensor starts the exposure before the light pulse arrives; 8. When the frame data upload ends, the previous frame data ends before the current frame exposure ends.

[0025] The above formulas are all calculated by taking the numerical values after removing the dimensions. The formula is obtained by software simulation of a large amount of collected data to get a formula closest to the actual situation. The preset parameters in the formula are set by those skilled in the art according to the actual situation.

[0026] The above embodiments can be implemented in whole or in part by software, hardware, firmware, or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on the computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center in a wired or wireless (such as infrared, wireless, microwave, etc.) manner. The computer-readable storage medium can be any available medium that the computer can access or a data storage device such as a server or data center that contains one or more collections of available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state drive.

[0027] It should be understood that in various embodiments of the present application, the magnitudes of the serial numbers of the above processes do not imply the order of execution, and the order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0028] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professionals can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of the present application.

[0029] Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0030] In several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces, and the indirect couplings or communication connections of the devices or units can be in electrical, mechanical, or other forms.

[0031] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0032] As described above, it is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or substitutions, which should all be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims described above.

Claims

1. An endoscope imaging system that can be used for vocal cord mucosal wave imaging, characterized in that, It includes an AC-DC switching power supply module, a DC-DC power supply module, a main control unit, an exposure light intensity detection circuit, a key input circuit, an image sensor, an image signal processor, a character overlay unit, and a video output expansion interface; The exposure light intensity detection circuit is a light intensity detection circuit with a visible light photosensitive diode as the core. The photosensitive diode is structurally placed close to the image sensor, and its light-receiving surface is parallel to the image sensor, so that it can receive the reflected light and stray light incident on the image sensor area. It is connected in series with a fixed-value resistor and connected to a fixed voltage, and the voltage at the series connection point will change synchronously with the light intensity of the light-receiving surface of the photosensitive diode. The main control unit automatically determines the exposure working mode according to the input result of the exposure light intensity detection circuit, and transmits the determined exposure working mode to the image signal processor through the serial data communication port. The image signal processor sets the internal register parameters of the image sensor through the serial data communication port according to the received exposure mode instruction and starts the corresponding exposure mode. The image sensor outputs images at a pre-determined frame frequency and exposure integration time. The external trigger exposure mode is controlled by the input pin level outside the image sensor to control the start and end times of exposure, and the exposure frequency and frame exposure time are controlled by the specific pin input signals. The main control unit calculates the number of light pulses for each frame of exposure according to the time period of the light pulse and controls the exposure frequency.

2. The endoscopic imaging system for vocal cord mucosal wave imaging according to claim 1, characterized in that, Connect the series connection point of the photosensitive diode and the series resistor to the AD input interface of the main control unit, and the main control unit detects the change in exposure light intensity through the AD conversion function.

3. An endoscopic imaging system for vocal cord mucosal wave imaging according to claim 1, characterized in that, When there is no light irradiating the light-receiving surface of the photosensitive diode, the dark current is extremely small, showing a high-resistance characteristic. When there is light irradiating its light-receiving surface, the passing current increases sharply, showing a low-resistance characteristic.

Images