Time division and space division combined imaging method and system

Abstract

The invention provides a time division and space division combined imaging method and system. The system is composed of an image acquisition unit, an image processing unit, an imaging light source module, a transmission line, a central control unit and the like. The image acquisition unit is composed of two image sensors and a beam splitter prism with a band-pass filter, and the light source module uses a four-color LED light source to output light sources of four wave bands. The central control unit generates a control signal to control the light source to emit combined light of different wave bands and controls the image processing unit to obtain four images of different wave bands by obtaining image data transmitted by the image acquisition unit and carrying out odd-even frame separation on the images, and the image processing unit outputs the images after carrying out ISP processing on the four images of different wave bands. The imaging function is achieved, the image color restoration effect is good, different spectral images can be accurately separated, and due to the fact that the two image sensors are used, noise is relatively low after long-time operation, and the process implementation difficulty is low.

Description

Technical Field

[0001] The present invention relates to the field of imaging technology, and in particular to an imaging method and system combining time division and space division. Background Art

[0002] At present, single-chip image sensor imaging is a common camera system implementation solution on the market. Traditional camera systems can obtain single-spectrum or multi-spectral images through a single image sensor. When obtaining multi-spectral images, single-chip image sensors use RGGB color filter arrangement devices to allow the sensor to distinguish the three basic colors of red, green and blue that make up visible light. In comparison, the advantages of RGGB are that it has been around for a long time, relevant research is mature, and it is easier to guess colors than other arrangements. From the perspective of the human eye, the three types of cone cells in the human eye have corresponding response curves for RGB respectively, and G provides visual details. The double G arrangement of RGGB just meets this point. It is the most reliable in practical applications. Of course, its shortcomings are also obvious. The design of a single pixel using only one color will waste a lot of color information and the utilization rate is relatively low. For this reason, CIS engineers have proposed many modification plans for RGGB. In response to the problem of insufficient light utilization of RGGB, upstream CIS manufacturers launched RGBW arrangement sensors for mobile phones around 2012. The disadvantages of this method are dull colors and more difficult color guessing. It is also easier to overexpose due to excessive information in the W channel, so the overall exposure time has to be shortened. As a result, the actual amount of light entering is lower than the theoretical value, and in extreme cases it is even similar to RGGB. Therefore, although most camera systems can use a single-chip image sensor to form an image, the color reproduction effect is not ideal due to the use of single-chip imaging, and the spectrum cannot be accurately separated in multi-spectral imaging scenarios, which limits the application scenarios. Summary of the invention

[0003] The technical problem to be solved by the present invention is that the color restoration effect based on a single image sensor solution is not ideal and the multi-spectral image cannot be accurately separated.

[0004] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0005] An imaging method combining time division and space division, comprising:

[0006] S1, the imaging light source module is configured to emit light of four wavelength bands: a first wavelength band λ1, a second wavelength band λ2, a third wavelength band λ3, and a fourth wavelength band λ4;

[0007] S2, the image acquisition unit uses space division imaging to simultaneously receive images from two independent image sensors 1 and 2, wherein image sensor 1 receives images of light in two wavelength bands, λ1 and λ2, and image sensor 2 receives images of light in two wavelength bands, λ3 and λ4;

[0008] S3, the central control unit generates a control signal and transmits it to the image acquisition unit, and at the same time controls the imaging light source module to emit two of the four bands of λ1, λ2, λ3 and λ4 at the same time. Specifically, the image acquisition unit configures each image sensor to work in the time-division imaging mode. At the current moment, the control signal controls the light source module to first turn off the λ2 and λ4 light, and then emit the λ1 and λ3 combined light. Image sensor 1 receives the λ1 image, and image sensor 2 receives the λ3 image. The image acquisition unit marks the current frame as an odd frame. At the next moment, the control signal controls the light source module to first turn off the λ1 and λ3 light and then switch the combined band to emit the λ2 and λ4 combined light. At the same time, image sensor 1 receives the band λ2 image, image sensor 2 receives the band λ4 image, and the image acquisition unit marks the current frame as an even frame. The two band combined lights are emitted alternately according to the control signal, and the corresponding image sensor receives the relevant band image in the corresponding timing.

[0009] S4, the image acquisition unit transmits the odd and even frame images to the image processing unit, and the image processing unit obtains four monochrome images of different bands including a λ1 image, a λ2 image, a λ3 image, and a λ4 image;

[0010] S5. The image processing unit performs relevant ISP processing on the four monochrome images and then outputs the images.

[0011] Furthermore, the λ1, λ2, λ3 and λ4 include four wavebands: red (R), blue (B), green (G) and infrared (IR).

[0012] Furthermore, the spatial separation imaging is to use a beam splitter prism in combination with a bandpass filter to spatially separate the spectral images received by the two image sensors, and each sensor receives images of two wavelength bands.

[0013] Furthermore, in the time-division imaging mode, the image acquisition unit separates the front and back frames of the image output by the image sensor, and acquires images of different wavelength bands in each frame.

[0014] Furthermore, the control signal frequency is related to f, specifically, the control signal frequency is an integer multiple of f.

[0015] Furthermore, the ISP processing algorithm includes white balance, detail enhancement, color enhancement, and image denoising.

[0016] In order to solve the above technical problems, the present invention also provides another technical solution:

[0017] An imaging system combining time division and space division, the imaging system comprising: a central control unit, an image acquisition unit, an image processing unit, an imaging light source module, a transmission line, etc. The image acquisition unit is composed of two image sensors combined with a spectroscopic prism with a bandpass filter. The imaging light source module uses a four-color LED light source to output four types of light sources. The central control unit generates a control signal to control the light source to emit light of different band combinations and controls the image processing unit to obtain the image data transmitted by the image acquisition unit, and separate the image into odd and even frames to obtain four images of different bands. The image processing unit performs ISP processing on the four images of different bands and then outputs the image.

[0018] The beneficial effects of the present invention are as follows: by inventing an imaging method and system combining time division and space division, the system is composed of an image acquisition unit, an image processing unit, an imaging light source module, a transmission line, a central control unit, etc. The image acquisition unit is composed of two image sensors combined with a spectroscopic prism with a bandpass filter, and the light source module uses a multi-color LED light source to output four types of light sources. The central control unit generates a control signal to control the light source to emit light of different band combinations and controls the image processing unit to obtain the image data transmitted by the image acquisition unit, and separate the image into odd and even frames to obtain four images of different bands. The image processing unit performs ISP processing on the four images of different bands and then outputs the image, thereby realizing the imaging function, achieving a good image color restoration effect, and being able to accurately separate images of different spectra. Moreover, due to the use of two image sensors, the noise is relatively small during long-term operation, and the difficulty of process implementation is low. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A schematic diagram of a timing control signal according to an embodiment of the present invention;

[0020] Figure 2 A schematic diagram of a process flow of an embodiment of the present invention;

[0021] Figure 3 It is a schematic structural diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the described embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] Unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0024] In order to keep the following description of the embodiments of the present invention clear and concise, the present invention omits detailed descriptions of some known functions and known components.

[0025] Embodiment 1 of the present invention provides an imaging method combining time division and space division, which realizes imaging based on a dual image sensor with a 1080P resolution.

[0026] S1, the imaging light source module is configured to emit red band, blue band, green band and near infrared band light;

[0027] S2, image sensor 1 is configured to receive R-band and IR-band images, image sensor 2 is configured to receive G-band and B-band images, the image sensors are set to 120 Hz frame rate working mode, and the central control unit uses the frame synchronization signal output by the image sensor as the timing control signal;

[0028] S2, such as Figure 1 As shown, λ1 is set to R band light, λ2 is set to IR band light, λ3 is set to G band light, and λ4 is set to B band light. The image processing unit controls the endoscope light source module to emit 60Hz R red light G green light combination light and 60Hz IR near-infrared B blue light combination light in sequence according to the generated timing control signal, that is, R+G—>IR+B—>R+G—>IR+B..., the image processing unit performs odd and even frame separation on the monochrome image transmitted by the image acquisition unit according to the timing control signal transmitted by the central control unit, and can obtain 60Hz R red light image, B blue light image, IR infrared light image and G green light image, wherein the odd frame is 60Hz R red light image G green light image, and the even frame is 60Hz IR infrared light image and B blue light image.

[0029] S4. The image processing unit caches four 60 Hz monochrome images, namely, the R red light image, the B blue light image, the IR infrared light image and the G green light image.

[0030] S5. The image processing unit processes the image obtained in step S4 by performing an ISP processing algorithm on the image. Specifically, a pixel-level image fusion algorithm is used, i.e., the R component in the R image, the B component in the B image, and the G component in the G image are combined to sequentially obtain the RGB components of the current image pixel to obtain a 60Hz RGB image. The 60Hz near-infrared image can use the IR image. The RGB image is processed by white balance algorithm, Gamma stretching, color enhancement, detail enhancement and other algorithms to obtain the output image.

[0031] From the above description, it can be seen that the beneficial effects of the first embodiment of the present invention are:

[0032] The image acquisition unit uses two 1080P resolution image sensors set to a 120Hz frame rate to collect four 60Hz monochrome images and transmit them to the image processing unit. The image processing unit obtains the four monochrome images of the endoscope image acquisition unit and separates the odd and even frames, and uses the ISP algorithm to process the images to obtain the output image. The present invention has good color restoration effect, can accurately separate RGB images and near-infrared images, and uses two image sensors, so the noise is relatively low during long-term operation, and the dual image sensor process is simple and the process implementation difficulty is low.

[0033] like Figure 2 As shown, the present invention also provides a second embodiment, an imaging system combining time division and space division, the system matches the above method,

[0034] The working principle of the present invention is: the central control unit controls the light source module to emit two different colors in the airspace, and generates two different combinations of four colors of light in the time domain, which are collected by the image acquisition unit and transmitted to the endoscope image processing unit. The image processing unit caches the four monochrome images, obtains the RGB image and the near-infrared IR image after fusion processing, and obtains the output image after ISP algorithm processing.

[0035] In summary, the present invention provides an imaging method and system combining time division and space division, and the system is composed of an image acquisition unit, an image processing unit, an imaging light source module, a transmission line, a central control unit, etc. The image acquisition unit is composed of two image sensors combined with a spectroscopic prism with a bandpass filter, and the light source module uses a multi-color LED light source to output four types of light sources. The central control unit generates a control signal to control the light source to emit light of different band combinations and controls the image processing unit to obtain the image data transmitted by the image acquisition unit, and separate the image into odd and even frames to obtain four images of different bands. The image processing unit performs ISP processing on the four images of different bands and then outputs the image, realizing the imaging function, achieving a good image color restoration effect, and being able to accurately separate images of different spectra. Moreover, due to the use of two image sensors, the noise is relatively small during long-term operation, and the difficulty of process implementation is low.

[0036] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent transformations made using the contents of the present invention's specification and drawings, or directly or indirectly applied in related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for imaging by combining time division and space division, characterized in that: The following steps are involved: S1, the imaging light source module is configured to emit light of four wavelength bands: a first wavelength band λ1, a second wavelength band λ2, a third wavelength band λ3, and a fourth wavelength band λ4; S2, the image acquisition unit uses space division imaging to simultaneously receive images from two independent image sensors 1 and 2, wherein image sensor 1 receives images of light in two wavelength bands, λ1 and λ2, and image sensor 2 receives images of light in two wavelength bands, λ3 and λ4; S3, the central control unit generates a control signal and transmits it to the image acquisition unit, and at the same time controls the imaging light source module to emit two of the four bands of λ1, λ2, λ3 and λ4 at the same time. Specifically, the image acquisition unit configures each image sensor to work in the time-division imaging mode. At the current moment, the control signal controls the light source module to first turn off the λ2 and λ4 lights and then emit the λ1 and λ3 combined lights. Image sensor 1 receives the λ1 image, and image sensor 2 receives the λ3 image. The image acquisition unit marks the current frame as an odd frame. At the next moment, the control signal controls the light source module to first turn off the λ1 and λ3 lights and then switch the combined band to emit the λ2 and λ4 combined lights. Image sensor 1 receives the band λ2 image, and image sensor 2 receives the band λ4 image. The image acquisition unit marks the current frame as an even frame. The two band combined lights are emitted alternately according to the control signal, and the corresponding image sensor receives the relevant band image in the corresponding timing. S4, the image acquisition unit transmits the odd and even frame images to the image processing unit, and the image processing unit obtains four monochrome images of different bands including a λ1 image, a λ2 image, a λ3 image, and a λ4 image; S5. The image processing unit performs relevant ISP processing on the four monochrome images and then outputs the images.

2. The imaging method combining time division and space division according to claim 1, characterized in that: The λ1, λ2, λ3 and λ4 include four wavebands: red (R), blue (B), green (G) and infrared (IR).

3. The imaging method combining time division and space division according to claim 1, characterized in that: The spatial separation imaging is to use a beam splitter prism in combination with a bandpass filter to spatially separate the spectral images received by two image sensors, and each sensor receives images of two wavelength bands.

4. The imaging method combining time division and space division according to claim 1, characterized in that: The time-division imaging mode is that the image acquisition unit separates the front and back frames of the image output by the image sensor, and each frame obtains an image of a different band.

5. The imaging method combining time division and space division according to claim 1, characterized in that: The control signal frequency is an integer multiple of f.

6. The imaging method combining time division and space division according to claim 1, characterized in that: The ISP processing algorithm includes white balance, detail enhancement, color enhancement, and image denoising.

Images