Electronic endoscope system

Abstract

To switch, according to use, a data processing method for imaging data to be transmitted from an image pickup device to a processor, in an electronic endoscope system.SOLUTION: A processor 200 comprises a data control unit 212 which performs setting whether to make an illumination light on a biological tissue into a consecutive light emitting mode or a pulse light emitting mode and designates a data processing method for imaging data, to an electronic scope 100. A CMOS sensor 14 comprises: a pixel unit 141 which has a plurality of pixels disposed in matrix and generates the imaging data; and a signal processing unit 142 which subjects the imaging data generated by the pixel unit 141 to signal processing by a data processing method designated from the processor 200.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to an electronic endoscope system including an electronic endoscope equipped with an image sensor configured to image a biological tissue.

Background Art

[0002] In the field of medical devices, it is possible to generate an image suitable for diagnosing a lesion hidden in a body cavity by illuminating a biological tissue in the body cavity and imaging the illuminated biological tissue in the body cavity as a subject. An electronic endoscope system is known. A CMOS sensor may be adopted as a solid-state image sensor attached to the tip of an electronic endoscope (electronic scope) used in an electronic endoscope system. A general CMOS sensor has a feature that the start time and end time of charge accumulation are different, for example, in line units (Patent Document 1). The imaging data obtained by the electronic endoscope is transmitted to a processor for an electronic endoscope connected to the electronic endoscope, and an image of the biological tissue as the subject is displayed on a monitor.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, in an electronic endoscope system, there is a desire to perform data processing on the imaging data transmitted from the image sensor to the processor in advance, or to change the type or data transfer rate according to the application. For example, when using an endoscope with a relatively long insertion section, such as when diagnosing the colon or stomach, it is preferable to encode the image data to improve robustness, as the image data will be transmitted over a long distance from the image sensor. However, encoding increases the number of transferred bits due to the addition of padding bits, so data compression may be necessary. On the other hand, when using an endoscope with a relatively short insertion section, such as when diagnosing the vocal cords, it is not always necessary to encode the image data because the transmission of image data from the image sensor is only over a short distance. However, it may be desirable to increase the data transfer rate to make the video smoother for observation.

[0005] Thus, when performing data processing on the imaging data transmitted from the image sensor to the processor, or when changing the data type or data transfer rate, it is necessary to install a separate IC chip for data processing (signal processing) at the tip of the endoscope, in addition to the CMOS sensor which has multiple pixels and generates imaging data. However, it is difficult to install a data processing IC chip at the tip of the endoscope separately from the CMOS sensor due to space limitations. This is especially true when trying to increase the number of pixels to ensure the resolution of the CMOS sensor.

[0006] Therefore, the present invention aims to enable switching of the data processing method for imaging data transmitted from the image sensor to the processor in an electronic endoscope system, depending on the application. [Means for solving the problem]

[0007] One aspect of this disclosure is, An electronic endoscope equipped with an imaging device configured to image biological tissue at its tip, The electronic endoscope system includes a processor that performs processing for display on imaging data transmitted from the imaging device. The aforementioned processor, A light source unit for emitting illumination light onto biological tissue, The system includes a control unit that sets the illumination light to either a continuous emission mode or a pulsed emission mode, and instructs the electronic endoscope on a data processing method for the imaging data. The imaging device is A pixel unit having multiple pixels arranged in a matrix and generating imaging data, The system includes a signal processing unit that performs signal processing on imaging data generated by the pixel unit using a data processing method instructed by the processor.

[0008] The electronic endoscope may include a storage unit for storing length information relating to the length of the insertion portion. In this case, the control unit obtains the length information from the electronic endoscope when the electronic endoscope is connected to the processor and determines a data processing method to instruct the electronic endoscope based on the length information.

[0009] The processor may include an operation input unit that receives user input. In that case, the control unit determines a data processing method to be instructed to the electronic endoscope in response to the operation input to the operation input unit.

[0010] When the control unit sets the illumination light to a continuous emission mode, it may instruct the electronic endoscope to either perform no data processing on the imaging data or to perform encoding processing on the imaging data.

[0011] When the control unit sets the illumination light to pulse emission mode, it may instruct the electronic endoscope to perform a data processing method that compresses the imaging data and then speeds up the data transfer rate. [Effects of the Invention]

[0012] According to the electronic endoscope system described above, the data processing method for imaging data transmitted from the image sensor to the processor can be switched according to the application. [Brief explanation of the drawing]

[0013] [Figure 1] This block diagram shows an example of the configuration of an electronic endoscope system in one embodiment. [Figure 2] This diagram illustrates the normal rolling shutter operation and pseudo-global shutter operation of a CMOS sensor. [Figure 3] This block diagram shows the main components of the system in Figure 1. [Figure 4] This is a flowchart executed by an electronic endoscope processor according to one embodiment. [Modes for carrying out the invention]

[0014] The electronic endoscope system of this embodiment will be described in detail below with reference to the drawings. Figure 1 is a block diagram showing an example of the configuration of the electronic endoscope system 1 of this embodiment. As shown in Figure 1, the electronic endoscope system 1 is a system specifically designed for medical use and comprises an electronic scope (endoscope) 100, an electronic endoscope processor 200 (hereinafter simply referred to as "processor 200"), and a monitor 300.

[0015] The processor 200 includes a system controller 21. The system controller 21 executes various programs stored in the memory 23 and provides integrated control of the entire electronic endoscope system 1. The system controller 21 is also connected to the operation panel 24. The system controller 21 changes each operation of the electronic endoscope system 1 and the parameters for each operation in response to instructions from the operator input into the operation panel 24. The system controller 21 outputs clock pulses to each circuit in the electronic endoscope system 1 to adjust the timing of the operation of each part.

[0016] The processor 200 includes a light source device 30. The light source device 30 emits illumination light L for illuminating a subject such as biological tissue in a body cavity. The illumination light L includes white light, pseudo-white light, or special light. According to one embodiment, the light source device 30 selects one of a mode in which white light or pseudo-white light is constantly emitted as the illumination light L and a mode in which white light or pseudo-white light and special light are alternately emitted as the illumination light L, and based on the selected mode, emits white light, pseudo-white light, or special light. White light is light having a flat spectral intensity distribution in the visible light band, and pseudo-white light is light in which the spectral intensity distribution is not flat and light in a plurality of wavelength bands is mixed. Special light is light in a narrow wavelength band such as blue or green in the visible light band. Light in the blue or green wavelength band is used when observing a specific part in biological tissue while emphasizing it. As will be described later, the light source device 30 operates in either a continuous emission mode in which continuous light is emitted or a pulse emission mode in which pulse light is emitted. The illumination light L emitted from the light source device 30 is condensed by a condenser lens 25 onto the incident end face of the LCB (Light Carrying Bundle) 11 and enters the LCB 11.

[0017] The illumination light L that has entered the LCB 11 propagates through the LCB 11. The illumination light L that has propagated through the LCB 11 is emitted from the emission end face of the LCB 11 disposed at the tip of the endoscope 100 and is irradiated onto the subject through the light distribution lens 12. The return light from the subject illuminated by the illumination light L from the light distribution lens 12 forms an optical image on the light receiving surface of the CMOS sensor 14 through the objective lens 13.

[0018] The CMOS (Complementary Metal Oxide Semiconductor) sensor 14 is an image sensor having a Bayer-type pixel arrangement. The CMOS sensor 14 accumulates the optical image formed by each pixel on the light-receiving surface as charges corresponding to the amount of light, reads them out, generates imaging data, and outputs it. Note that instead of the CMOS sensor 14, a CCD image sensor or other types of imaging devices may be used. The CMOS sensor 14 may also be equipped with a complementary color filter. Details of the CMOS sensor 14 will be described later.

[0019] Inside the connection part of the electronic scope 100, a scope control unit 15 is provided. The scope control unit 15 supplies a synchronization signal (for example, a vertical synchronization signal) for one frame to the CMOS sensor 14, acquires the imaging data of the subject from the CMOS sensor 14 in frame units, and transmits it to the image processing unit 22 of the processor 200. The frame period is, for example, 1 / 30 seconds or 1 / 60 seconds. The image processing unit 22 generates a video format signal by performing predetermined image processing on the imaging data and outputs it to the monitor 300. Note that the scope control unit 15 is not limited to supplying a synchronization signal for one frame. Depending on the CMOS sensor, there may be cases where imaging data is transmitted based on a synchronization signal created by itself. In that case, it is not necessary for the scope control unit 15 to supply a synchronization signal for one frame to the CMOS sensor.

[0020] The scope control unit 15 also accesses the memory 16 to read scope data from the electronic scope 100. The scope data of the electronic scope 100 recorded in the memory 16 (an example of a storage unit) includes length information regarding the length of the insertion portion of the electronic scope 100 into the body cavity, or information related to the length information, and data rate information when transmitting imaging data from the CMOS sensor 14 to the processor 200. The length information allows for the determination, for example, whether the electronic scope 100 has a relatively long insertion portion, such as when diagnosing the large intestine or stomach, or a relatively short insertion portion, such as when diagnosing the vocal cords. The model number and serial number of the electronic scope 100 are examples of information related to the length information. The scope control unit 15 outputs the scope data read from the memory 16 to the system controller 21. The scope data may include, for example, element-specific information such as the number of pixels and resolution of the CMOS sensor 14, as well as information related to the optical system such as the angle of view, focal length, and depth of field.

[0021] The system controller 21 performs various calculations based on the scope data from the electronic scope 100 and generates control signals. Using the generated control signals, the system controller 21 controls the operation and timing of various circuits within the processor 200 so that processing appropriate for the electronic scope 100 connected to the processor 200 is performed.

[0022] The system controller 21 supplies a clock pulse to the scope control unit 15. The scope control unit 15 generates a synchronization signal to operate the CMOS sensor 14 as needed, according to the clock pulse supplied by the system controller 21, and supplies it to the CMOS sensor 14.

[0023] Next, referring to Figure 2, we will explain the cases in which the CMOS sensor 14 operates using a normal rolling shutter method and the cases in which it operates using a pseudo-global shutter method. The standard rolling shutter method is applied when the illumination light from the light source device 30 is continuous light. In Figure 2, the lines that are sloped according to the passage of time from time T1 to time Tn indicate the readout timing of each line. In a typical rolling shutter system, within a frame period set based on a vertical synchronization signal (not shown), the first line begins exposure at time T1, when the previous frame's readout is complete. The last line begins exposure at time Tn, when the previous frame's readout is complete. The charge accumulated in the pixels of each line during exposure is converted into a voltage and read out. This operation is repeated for each frame. In the pseudo-global shutter method, the illumination light from the light source device 30 is pulsed light (strobe light). The duration of this pulsed light emission is shown in Figure 2. In this method, a common exposure period is set for all lines within one frame period so that one pulse of light does not expose the line for two frames. Therefore, the readout time for all lines from time T1 to Tn is shorter compared to a rolling shutter, and a high frame rate (readout speed) may be required.

[0024] The standard rolling shutter method and the pseudo-global shutter method are used appropriately depending on the application in which the electronic scope 100 is used. For example, when diagnosing the colon or stomach, images are acquired using the standard rolling shutter method, while when diagnosing the vocal cords, images are acquired using the pseudo-global shutter method. In laryngeal stroboscopy, which diagnoses the vocal cords, the pseudo-global shutter method shown in Figure 2 is employed to pulse light (strobe light) at a frequency corresponding to the vibration frequency of the vocal cords in accordance with the subject's phonation. Note that even when diagnosing the colon or stomach, the CMOS sensor 14 may be operated using the pseudo-global shutter method.

[0025] When the CMOS sensor 14 is operated using a pseudo-global shutter method by pulsed light emission, the frame rate (readout speed) may be high depending on the electronic scope, which can result in a high data rate (data transfer rate). When diagnosing the large intestine or stomach, an electronic endoscope 100 with a relatively long insertion section is used, resulting in a long cable length for transferring imaging data, which tends to distort the waveform of the imaging data. Therefore, to improve the reliability of data transfer, it is preferable to encode the imaging data, for example, using 8B10B encoding, when transferring data. However, in this case, the number of transferred bits increases due to the addition of padding bits, which tends to make it difficult to transfer imaging data at high data rates, so data compression may be necessary to reduce the number of transferred bits. On the other hand, when diagnosing the vocal cords, an electronic endoscope 100 with a relatively short insertion section is used, so the cable length for transferring imaging data is short, and waveform distortion of the imaging data is less likely to occur. Therefore, it is easier to transfer imaging data at a high data rate.

[0026] From another perspective, if the frame rate (readout speed) is low, and therefore a low data rate is acceptable, data compression to reduce the number of transferred bits is not necessary. In such cases, it may be possible to perform only encoding to improve the reliability of data transfer for the image data.

[0027] Thus, the processing of imaging data transferred from the CMOS sensor 14 to the processor 200 (such as increasing the data rate, encoding, and compression) can vary depending on the intended use of the electronic scope 100. However, conventionally, it has been difficult to place an integrated circuit (IC) for processing imaging data outside the CMOS sensor due to spatial constraints at the tip of the electronic scope. In other words, while it is desirable to ensure that the size of the pixel area of ​​the CMOS sensor is as large as possible because it contributes to the resolution, in that case, it is not possible to place a separate IC at the tip of the electronic scope, or there is a trade-off. Therefore, in this embodiment, a multilayer sensor is used as the CMOS sensor 14, and the CMOS sensor 14 equipped with a signal processing function is placed at the tip of the electronic scope 100. This multilayer sensor has been increasingly put into practical use in recent years, and has a structure in which the pixel part and the signal processing part are stacked on different layers and are integrated into a single package. By selectively or in combination performing image data processing (high data rate conversion, encoding, compression, etc.) in the signal processing part of the CMOS sensor 14, it is possible to realize image data processing according to the intended use of the electronic scope 100.

[0028] Next, referring to the block diagram shown in Figure 3, the main components of the electronic endoscope system 1 in Figure 1 will be described. The following explanation will focus on data processing between the system controller 21 of the processor 200 and the CMOS sensor 14 of the electronic scope 100, with reference to Figure 3. As shown in Figure 3, the system controller 21 has a light emission mode setting unit 211 and a data control unit 212. Each part of the system controller 21 is realized by executing a program. The light emission mode setting unit 211 sets the light emission mode of the light source device 30 (continuous light emission mode or pulsed light emission mode) and transmits a control signal indicating the set light emission mode to the light source device 30. The light source device 30 operates a drive circuit (not shown) according to the light emission mode set by the light emission mode setting unit 211 and emits illumination light (continuous light or pulsed light).

[0029] The data control unit 212 determines a data processing method for the imaging data transferred from the CMOS sensor 14 based on at least one of the information contained in the scope data acquired from the scope control unit 15 of the electronic scope 100: the length information of the insertion part of the electronic scope 100 and the data rate information. The data processing method for the imaging data may include encoding the imaging data and sending it to the processor 200, compressing the imaging data and sending it to the processor 200, or sending the imaging data to the processor 200 at a high data rate, or a combination thereof. Not processing the imaging data is also an option. The data control unit 212 notifies the scope control unit 15 of the determined data processing method for the imaging data. In one embodiment, the data control unit 212 determines a data processing method for the imaging data transferred from the CMOS sensor 14 based on an operation signal from the operation panel 24. The operation panel 24 is operated by the operator.

[0030] The scope control unit 15 reads scope data from the memory 16 and transmits it to the system controller 21, and controls the CMOS sensor 14 so that the data processing method for the imaging data notified by the system controller 21 is performed. The scope control unit 15 also outputs a synchronization signal to the CMOS sensor 14 as needed. In one embodiment, the system controller 21 is configured to acquire scope data from the scope control unit 15, to which the electronic scope 100 is connected to the processor 200. In this case, when various electronic scopes 100 are connected to the processor 200, the data processing method for the imaging data is automatically determined based on the scope data. That is, the optimal data processing method is automatically determined on the system at the time of diagnosis, and it is not necessary for the operator or others to set it.

[0031] As shown in Figure 3, the CMOS sensor 14 has a pixel section 141 and a signal processing section 142. The pixel unit 141 has a pixel array composed of numerous pixels arranged in a two-dimensional matrix. The pixel unit 141 generates imaging data by sequentially scanning each pixel in the pixel array, reading out pixel signals, and operating the electronic shutter. The generation of imaging data is performed based on synchronization signals supplied as needed from the scope control unit 15. The signal processing unit 142 processes the image data generated by the pixel unit 141 according to a data processing method instructed by the system controller 21, and then transmits the image data to the system controller 21. The transmission timing of the image data is performed frame by frame.

[0032] For example, if the data processing method is "no processing," the signal processing unit 142 transmits the image data generated by the pixel unit 141 to the system controller 21 without performing any signal processing on the image data. If the data processing method is "data encoding," the signal processing unit 142 performs encoding processing such as 8B10B on the image data generated by the pixel unit 141 before transmitting the image data to the system controller 21. If the data processing method is "high data rate, encoding and compression," the signal processing unit 142 performs encoding processing such as 8B10B on the image data generated by the pixel unit 141 and also performs data compression before transmitting the image data to the system controller 21 at a high data rate. The data control unit 212 receives imaging data from the CMOS sensor 14 at a data rate specified by a predetermined data processing method, and performs decompression and decoding processing of the received imaging data as needed.

[0033] Next, referring to the flowchart in Figure 4, the processes performed by the electronic endoscope system 1 of this embodiment will be described. When the electronic scope 100 is connected to the processor 200, the system controller 21 of the processor 200 performs an initialization process, including the process of acquiring scope data from the scope control unit 15 (step S2). The processes in steps S4 to S12 after initialization are performed until the observation of the subject is completed (step S14). First, the system controller 21 sets the illumination light emission mode (continuous emission mode or pulsed emission mode) in response to, for example, the operator's input to the control panel 24 (step S4). For example, the continuous emission mode is selected for normal observation of the large intestine, stomach, etc., while the pulsed emission mode is selected mainly for observation of the vocal cords, or for observation of the large intestine, stomach, etc.

[0034] Regardless of whether continuous emission mode or pulse emission mode is set, the system controller 21 determines the data processing method by the CMOS sensor 14 based on at least one of the length information and data rate information included in the scope data acquired in step S2. In one embodiment, when the continuous flash mode is set, the system controller 21 decides whether to perform "data encoding" or "no processing" (effectively no data processing) on ​​the image data (step S8). When the continuous flash mode is set, the electronic scope 100 operates the CMOS sensor 14 using a normal rolling shutter method.

[0035] If the length information included in the scope data indicates that the insertion portion is relatively long, such as when used for observing the large intestine or stomach, it is preferable to encode the image data to improve the reliability of data transfer, but it is not necessary. Not encoding the image data has the advantage of suppressing power consumption associated with the operation of the CMOS sensor 14. Even when encoding the image data, the data rate can be low, so data compression is not necessary. Furthermore, if the length information included in the scope data indicates that the insertion portion is relatively short, there is no need to consider data distortion, and therefore data encoding of the imaging data is not required.

[0036] When the pulse emission mode is set, the electronic scope 100 operates the CMOS sensor 14 using a pseudo-global shutter method, but the data rate for the imaging data may be slow or fast depending on the frame rate (readout speed). If, after referring to the scope data, the data rate is slow (step S6: "slow"), the system controller 21 decides, as in step S8, whether to "encode" or "not process" the data processing method for the imaging data (step S10).

[0037] On the other hand, if the data rate is high after referring to the scope data (step S6: "high speed"), the system controller 21 decides to set the data processing method for the imaging data to "data encoding and compression" (step S12). For example, when the electronic scope 100 is used to observe the vocal cords, or when observing the stomach or large intestine and wanting smoother images, the data rate indicated by the scope data is set to a high speed. In that case, from the standpoint of data reliability, the imaging data is encoded, and since the number of transferred bits increases due to the addition of padding bits associated with data encoding, data compression is performed to support high data rates.

[0038] As described above, according to the electronic endoscope system 1 of this embodiment, the CMOS sensor 14 provided in the electronic scope 100 incorporates a signal processing unit 142 that performs signal processing on imaging data generated by the pixel unit 141, so the overall system is compact and can be mounted on the tip of the electronic scope 100. The signal processing unit 142 of the CMOS sensor 14 performs signal processing according to a data processing method instructed by the processor 200. Therefore, various processes such as encoding, compression, and high-data-rate transmission of imaging data can be realized depending on the intended use of the system.

[0039] As mentioned above, the data processing method for imaging data is not limited to scope data, but may also be determined based on the operator's input from the control panel 24. For example, when using an electronic scope with a long insertion section, it is preferable to use a high frame rate (readout speed) (i.e., a high data rate) to make the image easier to see, but the operator may want to change the frame rate (readout speed) (to a high or low speed) when performing the procedure. In such cases, the data processing method by the CMOS sensor 14 can be flexibly determined.

[0040] Although the electronic endoscope system of the present invention has been described in detail above, the electronic endoscope system of the present invention is not limited to the above embodiments, and various improvements and modifications may be made without departing from the spirit of the present invention. The configuration of the electronic endoscope system is not limited to that shown in Figure 1, and various configurations are possible. For example, the present invention can be applied to a system in which the processor does not have a light source device, and the light source is located at the tip of the electronic scope instead of having an LCB in the electronic scope. Furthermore, the present invention can be applied to a system in which the light source device is configured separately from the processor and the electronic scope. [Explanation of Symbols]

[0041] 1… Electronic endoscope system 11…LCB 12…Light distribution lens 13…Objective lens 14…CMOS sensor 141...Pixel area 142... Signal Processing Unit 15…Scope Control Unit 16…Memory 21... System Controller 211...Lighting mode setting section 212...Data Control Unit 22…Image Processing Unit 23…Memory 24... Control Panel 30…Light source device 100...Electronic scope 200…Processor for electronic endoscopes 300... Monitor

Claims

1. An electronic endoscope equipped with an imaging device configured to image biological tissue at its tip, An electronic endoscope system including a processor that performs processing for display on imaging data transmitted from the imaging device, The aforementioned processor, A light source unit for emitting illumination light onto biological tissue, The system includes a control unit that sets the illumination light to either a continuous emission mode or a pulsed emission mode, and instructs the electronic endoscope on a data processing method for the imaging data, The imaging device is A pixel unit having multiple pixels arranged in a matrix and generating imaging data, The system includes a signal processing unit that performs signal processing on imaging data generated by the pixel unit using a data processing method instructed by the processor, Electronic endoscope system.

2. The aforementioned electronic endoscope includes a storage unit that stores length information relating to the length of the insertion portion, The control unit acquires the length information from the electronic endoscope when the electronic endoscope is connected to the processor, and determines a data processing method to be instructed to the electronic endoscope based on the length information. The electronic endoscope system according to claim 1.

3. The aforementioned processor includes an operation input unit that receives user operation input, The control unit determines a data processing method to be instructed to the electronic endoscope in response to an operation input to the operation input unit. The electronic endoscope system according to claim 1.

4. When the control unit sets the illumination light to continuous emission mode, it instructs the electronic endoscope to perform a data processing method that either substantially does not perform data processing on the imaging data or performs encoding processing on the imaging data. An electronic endoscope system according to any one of claims 1 to 3.

5. When the control unit sets the illumination light to pulse emission mode, it instructs the electronic endoscope to either perform virtually no data processing on the imaging data, or to perform data compression on the imaging data and then increase the data transfer rate. An electronic endoscope system according to any one of claims 1 to 3.

Images