Time division multiplexing fill light imaging device and method

Abstract

The present application provides a time-division multiplexing fill-light imaging method and device. The method includes: using an image sensor to alternately generate visible light frames and fill-light frames, the visible light frames are image frames generated when the image sensor receives visible light and does not receive fill-light light, and the fill-light frames are image frames generated when the image sensor receives fill-light light; synthesizing adjacent or continuous visible light frames and fill-light frames to obtain a synthesized frame.

Description

Technical Field

[0001] The present invention relates to the technical field of fill-light imaging, and in particular to a device and method for time-division multiplexing fill-light imaging. Background Art

[0002] With the rapid development of image applications, the requirements for the imaging quality of image acquisition equipment under low illumination are getting higher and higher. Compared with scenes with normal illumination, the imaging quality in low illumination environments will decline, mainly manifested in reduced image brightness and increased noise. The biggest factor leading to the decline in imaging quality is the reduction in the amount of light entering the imaging sensor, and the reduction in the signal-to-noise ratio of its output signal. In this regard, there are two common solutions. One solution is to increase the amount of visible light entering the image sensor under low illumination, that is, to use visible light fill light equipment for fill light. Although visible light fill light increases the brightness of the image, high-brightness visible light fill light will interfere with the photographed object, causing overexposure and distortion of the image. Another solution is to use infrared imaging. Although this method can obtain sufficient image brightness without obvious interference to the photographed object, it cannot obtain the true color information of the object well.

[0003] Therefore, in low-light environments, how to obtain images with true color information and appropriate brightness without interfering with the photographed objects has become an urgent problem to be solved. Summary of the invention

[0004] The present application provides a method for time-division multiplexing fill-light imaging and a device using the method. Through a time-division multiplexing visible light frame and fill-light frame generation method, the visible light frame and fill-light frame required to generate a synthetic frame are generated by the same image sensor, which effectively simplifies the imaging optical path and imaging equipment, and generates synthetic frames more efficiently.

[0005] In a first aspect, the present application provides a fill light imaging device, which may include an image sensor, an optical unit, a fill light unit, and a processor. The image sensor is used to alternately generate visible light frames and fill light frames. The type of the first frame generated when the fill light frame and the visible light frame are alternately generated is not limited, and it can be either a visible light frame or a fill light frame. Alternating means generating the two image frames in turn, such as odd frames all generate visible light frames and even frames all generate fill light frames, or one fill light frame is generated for every two visible light frames, or two fill light frames are generated for every visible light frame, etc., which are not limited here. The optical unit is used to image light on the image sensor. The fill light unit is used to fill light for the subject when generating the fill light frame. The processor is used to synthesize adjacent or continuous visible light frames with the fill light frames to generate a synthesized frame. Compared with the prior art, the fill light imaging device uses the same image sensor to alternately generate visible light frames and fill light frames in a time-division multiplexing manner, so that the visible light frames and fill light frames required to generate a synthetic frame no longer require multiple image sensors to complete, effectively reducing the production cost of the imaging equipment and the difficulty of imaging optical path design.

[0006] According to the first aspect, in a first possible implementation of the first aspect, the processor can be used to control the switch of the fill light unit. Specifically, when the image sensor starts to generate fill light frames, the processor controls the fill light unit to start fill light, and when the image sensor ends to generate fill light frames, the processor controls the fill light unit to stop fill light. Optionally, when the image sensor ends to generate visible light frames, the processor controls the fill light unit to start fill light, and when the image sensor starts to generate visible light frames, the processor controls the fill light unit to stop fill light, which is not limited here. The processor controls the fill light unit to alternately perform fill light, so that fill light frames and visible light frames can be alternately generated in continuous image frames generated by the same image sensor, thereby improving the utilization efficiency of the image sensor.

[0007] According to the first aspect, in a second possible implementation of the first aspect, the optical unit can be used to control the type of light that can be irradiated onto the image sensor, and when the image sensor starts to generate fill light frames, the processor can control the optical unit so that the fill light can be irradiated onto the image sensor, and when the image sensor starts to generate visible light frames, the processor can control the optical unit so that the fill light cannot be irradiated onto the image sensor. Optionally, when the image sensor ends generating fill light frames, the processor can control the optical unit so that the fill light cannot be irradiated onto the image sensor, and when the image sensor ends generating visible light frames, the processor can control the optical unit so that the fill light can be irradiated onto the image sensor, which is not limited here. The processor controls the optical unit to alternately change the permeability of the fill light in the optical unit, so that fill light frames and visible light frames can be alternately generated in consecutive image frames generated by the same image sensor.

[0008] According to the second possible implementation of the first aspect, in a third possible implementation of the first aspect, the optical unit may include a filter switching device and a single-pass filter, wherein the single-pass filter is used to filter out other types of light except visible light in the imaging light path of the optical unit, and the filter switching device is used to place the single-pass filter in or out of the imaging light path of the optical unit. Specifically, the processor is used to control the filter switching device to place the single-pass filter outside the imaging light path of the optical unit when the image sensor starts to generate a fill light frame, and to control the filter switching device to place the single-pass filter in the imaging light path of the optical unit when the image sensor starts to generate a visible light frame. Optionally, when the image sensor finishes generating a fill light frame, the processor may control the filter switching device to place the single-pass filter in the imaging light path of the optical unit, and when the image sensor finishes generating a visible light frame, the processor may control the filter switching device to place the single-pass filter outside the imaging light path of the optical unit, which is not limited here. By switching the filters through the filter switching device, the permeability of the fill light in the optical unit is alternately changed, so that the fill light frame and the visible light frame can be alternately generated in the continuous image frames generated by the same image sensor.

[0009] Optionally, the optical unit may further include a double-pass filter to filter out other light in the imaging light path except visible light and fill light. Filtering out other light except visible light and fill light can eliminate interference from other light during imaging and improve imaging quality.

[0010] Optionally, the processor can be used to synthesize an adjacent visible light frame with a fill light frame to generate a synthesized frame. The generation interval between adjacent visible light frames and fill light frames is short, the generated synthesized frame has less motion blur and a clearer image.

[0011] Optionally, the processor may be used to synthesize one or more continuous visible light frames with one or more fill light frames to obtain a synthesized frame. Using multiple visible light frames and multiple fill light frames for synthesis can enrich image information and reduce errors during synthesis.

[0012] Optionally, the number of image sensors in the fill light imaging device can be one. Since the generation of visible light frames and fill light frames can be completed by the same image sensor, only one image sensor can be set in the fill light imaging device, which reduces the size of the device, avoids the complex optical path when multiple sensors are used, and reduces the cost of the device and the difficulty of design and manufacturing.

[0013] Optionally, the color information of the synthesized frame can adopt the color information of the visible light frame, and the brightness information of the synthesized frame can adopt the brightness information of the fill light frame. The synthesized frame adopts the color information of the visible light frame and the brightness information of the fill light frame, taking into account the brightness and color of the image, so that the synthesized frame can enrich the color of the image while ensuring sufficient brightness.

[0014] Optionally, the fill light of the fill light unit may be infrared light, which has strong penetrating power, is not easily blocked, and has low application cost.

[0015] Optionally, the fill light imaging device may include an ambient light detection unit for detecting the ambient light illumination; for example, a photoresistor or other detection elements. When the ambient light detection unit detects that the ambient light illumination is equal to or less than a preset value, the image sensor begins to alternately generate visible light frames and fill light frames. In another implementation, the ambient light illumination can also be detected by an image sensor. Specifically, the gain value of the image sensor can be used as a criterion for judging the intensity of the ambient light illumination. The gain value of the image sensor under the preset value ambient light illumination is used as a preset gain value. When the gain value is equal to or greater than the preset gain value, the image sensor begins to alternately generate visible light frames and fill light frames. When the ambient light illumination is high, high-quality images can be obtained without fill light. By detecting the ambient light and enabling fill light imaging when the illumination is as low as the preset value, it helps to save energy and extend the service life of the fill light equipment.

[0016] Optionally, the processor can also be used to increase the image frame acquisition frame rate of the image sensor when the ambient light illumination is equal to or less than the aforementioned preset value. In one possible implementation, when the ambient light illumination is equal to or less than the aforementioned preset value, the processor increases the image frame acquisition frame rate of the image sensor to n times that when the ambient light illumination is greater than the aforementioned preset value, where n is the sum of the number of visible light frames and fill light frames required to generate a synthetic frame. Increasing the acquisition frame rate at the beginning of fill light imaging helps to reduce the frame rate drop when the synthetic frame is output, thereby reducing the decline in output image quality; when the acquisition frame rate is increased to n times the acquisition frame rate when fill light imaging is not used, and n is the sum of the number of visible light frames and fill light frames required to generate a synthetic frame, the output frame rate of the synthetic frame will be equal to the output frame rate when fill light imaging is not used, and the image quality can maintain a relatively stable output when switching imaging modes, thereby improving the user experience.

[0017] In a second aspect, the present application provides a terminal device, which may include the fill light imaging module described in the first aspect and any one of all possible implementations of the first aspect, and may also include a main processing module. The main processing module is used to perform image processing on the synthesized frame. The terminal device includes the aforementioned fill light imaging module, which can achieve high-quality image acquisition under low illumination and further process the acquired image, and the terminal device only needs to contain one image sensor, which saves device space, reduces device cost, and improves user experience.

[0018] Optionally, the terminal device may further include a communication module for transmitting the image or video corresponding to the composite frame to other devices through an interface transmission or wireless transmission. Transmitting the image or video corresponding to the composite frame to other devices can meet the user's processing requirements for the image or video corresponding to the composite frame, such as transferring storage or interacting with other users, thereby improving the user experience.

[0019] In a third aspect, the present application provides a fill light imaging method, comprising: using an image sensor to alternately generate visible light frames and fill light frames, the visible light frames being image frames generated when the image sensor receives visible light and does not receive fill light, and the fill light frames being image frames generated when the image sensor receives fill light; synthesizing adjacent or continuous visible light frames with the fill light frames to obtain a synthesized frame. Here, the type of the first frame generated when the fill light frame and the visible light frame are alternately generated is not limited, and can be either a visible light frame or a fill light frame. Alternating means generating the two types of image frames in turn, such as generating visible light frames for odd frames and fill light frames for even frames, or generating one fill light frame for every two visible light frames generated, or generating two fill light frames for every visible light frame generated, etc., which is not limited here.

[0020] According to the third aspect, in a first possible implementation of the third aspect, fill light can be applied to the photographed object when fill light frames are generated, and fill light can be stopped when fill light frames are generated. Optionally, fill light can be started when visible light frames are generated, and fill light can be stopped when visible light frames are generated. This is not limited here. By using alternate fill light, fill light frames and visible light frames can be alternately generated in continuous image frames generated by the same image sensor, thereby improving the utilization efficiency of the image sensor.

[0021] According to the third aspect, in a second possible implementation of the third aspect, fill light is continuously applied to the photographed object. When the generation of visible light frames starts, a single-pass filter may be used to filter out the fill light irradiated on the image sensor. When the generation of fill light frames starts, the single-pass filter may be stopped to allow the fill light to irradiate the image sensor. Optionally, when the generation of fill light frames ends, a single-pass filter may be used to filter out the fill light irradiated on the image sensor. When the generation of visible light frames ends, the single-pass filter may be stopped to allow the fill light to irradiate the image sensor. This is not limited here. By using alternate fill light, fill light frames and visible light frames can be alternately generated in continuous image frames generated by the same image sensor, thereby improving the utilization efficiency of the image sensor.

[0022] Optionally, a double-pass filter can be used to filter out other light in the imaging light path except visible light and fill light. Filtering out other light except visible light and fill light can eliminate interference from other light during imaging and improve imaging quality.

[0023] Optionally, an adjacent visible light frame and a fill light frame may be synthesized to generate a synthesized frame. The generation interval between adjacent visible light frames and fill light frames is short, and the generated synthesized frame has less motion blur and a clearer image.

[0024] Optionally, one or more continuous visible light frames may be synthesized with one or more fill-in light frames to obtain a synthesized frame. Using multiple visible light frames and multiple fill-in light frames for synthesis can enrich image information and reduce errors during synthesis.

[0025] Optionally, the color information of the visible light frame can be extracted, the brightness information of the fill light frame can be extracted, and the color information of the visible light frame and the brightness information of the fill light frame can be used as the color information and brightness information of the synthesized frame. The synthesized frame uses the color information of the visible light frame and the brightness information of the fill light frame, taking into account the brightness and color of the image, so that the synthesized frame can enrich the color of the image while ensuring sufficient brightness.

[0026] Optionally, the fill light frame is an image frame generated when the image sensor receives fill light and visible light, and the visible light frame is an image frame generated when the image sensor receives only visible light without the fill light. Not receiving other light during imaging can reduce interference from other light and improve imaging quality.

[0027] Optionally, the fill light may be infrared light, which has strong penetrating power, is not easily blocked, and has low application cost.

[0028] Optionally, when the ambient light illumination is equal to or less than a preset value, an image sensor may be used to alternately generate visible light frames and fill light frames. When the ambient light illumination is high, high-quality images can be obtained without fill light. By detecting the ambient light and enabling fill light imaging when the illumination is as low as a preset value, it helps to save power and extend the service life of the fill light device.

[0029] Optionally, the image frame acquisition frame rate of the image sensor can be increased when the ambient light illumination is equal to or less than the aforementioned preset value. In one possible implementation, the image frame acquisition frame rate of the image sensor can be increased to n times when the ambient light illumination is greater than the aforementioned preset value, where n is the sum of the number of visible light frames and fill light frames required to generate a synthetic frame. Increasing the acquisition frame rate at the beginning of fill light imaging helps to reduce the frame rate drop when the synthetic frame is output, thereby reducing the decline in output image quality; when the acquisition frame rate is increased to n times the acquisition frame rate when fill light imaging is not used, and n is the sum of the number of visible light frames and fill light frames required to generate a synthetic frame, the output frame rate of the synthetic frame will be equal to the output frame rate when fill light imaging is not used, and the image quality can maintain a relatively stable output when switching imaging modes, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A schematic diagram of a light-filling imaging device;

[0031] Figure 2a A schematic diagram of a synthetic frame generation effect;

[0032] Figure 2b is a schematic diagram of a synthetic frame generation method;

[0033] Figure 3 A schematic diagram of a flow chart of a fill light imaging method;

[0034] Figure 4 is a flow chart of another fill-light imaging method;

[0035] Figure 5 It is a logical structure diagram of a fill-light imaging device;

[0036] Figure 6 is a schematic diagram of the logical structure of another fill-light imaging device;

[0037] Figure 7 The figure is a schematic diagram of the logical structure of a terminal device. DETAILED DESCRIPTION

[0038] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0039] The drawings and descriptions discussed below and the various embodiments in the patent document describe the principles of the present invention only by way of example and should not be understood in any way as limiting the scope of the present invention. The visible light referred to in this application is an electromagnetic wave with a wavelength greater than 400nm and less than 700nm; the non-visible light referred to in this application is an electromagnetic wave with a wavelength outside the visible light wavelength range. It will be easy for a person skilled in the art to understand that the principles of the present invention can be implemented in any type of suitable device or system. Specifically, the time-division multiplexing fill light imaging method involved in this application can be applied to cameras and surveillance cameras, etc., especially image acquisition in low-light environments, such as night photography, night video shooting, low-light environment video monitoring, etc. The application scenarios, device configurations, fill light source types, fill light frame exposure frequencies, etc. described in the embodiments of the present invention are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. It is known to those skilled in the art that with the changes in application scenarios, device configurations, and fill light source types, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0040] For the convenience of explanation, the fill light source in the embodiment of the present application is an infrared light source as an example. The fill light frame in the embodiment of the present application is obtained by filling light once every other frame, that is, only exposing in odd frames or only exposing in even frames, and the generation of the synthetic frame is synthesized by an adjacent fill light frame and a visible light frame. It should be noted that the fill light source in the present application can be one or more of non-visible light sources that can be used for imaging, such as infrared light sources, ultraviolet light sources, X-ray light sources, etc., which are not limited here. The visible light frame and the fill light frame in the present application can adopt any alternating generation method that can achieve the purpose of the present invention, that is, any exposure interval method, such as exposing a fill light frame every two visible light frames, or exposing two fill light frames every other visible light frame, etc., which are in line with the technical solution of the present application and are not limited here. Accordingly, the generation of the synthetic frame in the present application can be synthesized by any number of adjacent or continuous visible light frames and exposure frames, that is, the scheme of synthesizing a synthetic frame by a continuous frame sequence including at least one visible light frame and at least one fill light frame at the same time is in line with the technical solution of the present application and is not limited here.

[0041] See also Figure 1 , is a schematic diagram of the physical structure of a possible embodiment of the time-division multiplexing supplementary light imaging device of the present invention. Figure 1 As shown, the imaging device 100 includes a fill light 110, a lens 120, an image sensor 140, a filter switching device 130, and a processor 150. In some possible implementations, the imaging device 100 may not include the filter switching device 130, or the filter switching device 130 may be replaced by a fixed filter. In some possible implementations, the imaging device 100 also includes a photoresistor 160.

[0042] The fill light 110 is used to fill light for the photographed object, and can be implemented as an infrared light-emitting diode (LED) fill light, and the emission center wavelength that can be used includes but is not limited to 760nm, 850nm or 950nm. The fill light 110 can be turned on and off under the control of the processor 150. The switch of the fill light can be controlled by a switch value or a level signal.

[0043] The lens 120 is used to capture light to form an image on the image sensor 140 , and the implementation methods include but are not limited to a wide-angle lens, a standard lens, a telephoto lens, a zoom lens, a retrofocus lens or a pinhole lens.

[0044] The image sensor 140 is used to convert the received optical signal into an electrical signal, and generate a digital image signal with frames as the separation unit based on the electrical signal. The image sensor 140 can realize the conversion between optical signals and electrical signals through a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). The process in which the image sensor 140 finally converts the received optical signal into a digital image signal with frames as the separation unit can be called image exposure. The frame generated by exposure when not receiving the infrared light of the fill light is called a visible light frame, and the frame generated by exposure when receiving the infrared light of the fill light is called a fill light frame. The image sensor 140 generates a frame start and frame end signal at the beginning and end of the image exposure and sends it to the processor 150 to notify the processor 150 of the start and end of the exposure. When generating a fill light frame, the image sensor 140 may not receive visible light when receiving the infrared light of the fill light, or may receive the infrared light of the fill light and the visible light at the same time, which is not limited here.

[0045] The filter switching device 130 is used to switch filters. A filter is an optical element that has selective absorption characteristics for different wavelengths of light, and possible implementation methods include but are not limited to colored glass, dyed film, etc. By switching different filters, the filter switching device 130 can control which wavelengths of light collected by the lens 120 can be irradiated to the image sensor 140. In one possible implementation, when the fill light imaging mode is not turned on, the filter switching device 130 will place a single-pass filter that can only pass visible light on the optical path of the lens, so that only visible light is irradiated to the sensor 140; when the fill light imaging mode is turned on, the filter will be switched to a double-pass filter, so that both infrared light and visible light can be irradiated to the image sensor 140. In another possible implementation, when the fill light imaging mode is turned on, the filter switching device 130 will periodically switch the filter according to a specific frequency corresponding to the fill light frame generation frequency, so that the type of light wavelength band that can be irradiated to the image sensor 140 changes periodically. Optionally, the filter switching device 130 may include both a single-pass filter and a double-pass filter, and periodically switch the single-pass filter and the double-pass filter according to the frame exposure frequency of the image sensor 140, or only include a single-pass filter, and periodically place and remove the single-pass filter from the imaging optical path of the device according to the frame exposure frequency of the image sensor 140. In some possible implementations, the fill light imaging device 100 may not include the filter switching device 130, but directly use a fixed specific filter so that the light irradiated on the image sensor is a fixed number of bands of light; it may also not use a filter, so that all bands of light collected by the lens are irradiated on the sensor. This embodiment is not limited to this.

[0046] The processor 150 is used to control the on and off of the fill light 110, control the acquisition frame rate of the image sensor 140, perform basic processing of digital image signals, and synthesize visible light frames and fill light frames. When the imaging device 100 includes a filter switching device 130, the processor 150 is also used to control the filter switching device 130. The basic processing of the image by the processor 150 includes but is not limited to de-mosaicing, automatic white balance, etc. The synthesis of the visible light frame and the fill light frame refers to generating a synthetic frame through the visible light frame and the fill light frame, the brightness information of the synthetic frame comes from the fill light frame, and the chromaticity information comes from the visible light frame. In some possible implementations, the processor 150 controls the filter switching device 130 to switch the filter to a dual-pass filter, and alternately turns on and off the fill light 110 according to the frame start signal and the frame end signal received from the image sensor 140, so that the image sensor 140 periodically receives infrared light during exposure, and then generates a fill light frame according to the corresponding periodic frequency. In some other possible implementations, the processor 150 controls the fill light 110 to be continuously turned on, and controls the filter switching unit 140 to switch the visible light filter and the double-pass filter according to the frame start signal and the frame end signal received from the image sensor 140, so that the image sensor 140 periodically receives infrared light during exposure, thereby alternately generating fill light frames and visible light frames. The functions of the processor 150 may be implemented by one processor, or by multiple processors, or may be partially or completely integrated in the image sensor 140, which is not limited here.

[0047] The photoresistor 160 is used to detect changes in ambient light illumination. When the ambient light illumination is equal to or less than a preset value, the fill light imaging device 100 turns on the fill light imaging mode.

[0048] Figure 2a It is a schematic diagram of image frame synthesis. For the same scene, the fill light imaging device will generate a visible light frame and a fill light frame, and synthesize the visible light frame and the fill light frame to generate a synthesized frame. The visible light frame is an image frame generated by exposing the image sensor when exposed to visible light without fill light. In low illumination conditions, the brightness information of the visible light frame is insufficient, but the color information is better. The fill light frame is an image frame generated by exposing under the simultaneous irradiation of visible light and fill light infrared light. Its brightness is higher than that of the visible light frame, but it lacks color information. In some possible embodiments, the fill light frame may also be generated by the sensor only by infrared exposure. The synthesized frame is obtained by synthesizing the visible light frame and the fill light frame. Specifically, the synthesized frame retains the color information in the visible light frame and the brightness information in the fill light frame.

[0049] Figure 2bA possible method embodiment for synthesizing visible light frames and fill light frames in YUV color space is shown. Among them, j frame, j+1 frame, j+2 frame and j+3 frame represent image frames collected by the image sensor in time sequence, j frame and j+2 frame are visible light frames, j+1 frame and j+3 frame are fill light frames; k frame represents a synthesized frame synthesized by j frame and j+1 frame, and k+1 frame represents a synthesized frame synthesized by j+2 frame and j+3 frame. YUV is a color encoding method used to compile true-color color space, in which "Y" represents brightness (Luma) and "UV" represents chrominance and concentration (Chrominance, Chroma). In the figure, Y0 represents the brightness information of visible light frame j, and UV0 represents the color information of visible light frame j; Y1 represents the brightness information of fill light frame j+1, and UV1 represents the color information of fill light frame j+1; Y2 represents the brightness information of visible light frame j+2, and UV2 represents the color information of visible light frame j+2; Y3 represents the brightness information of fill light frame j+3, and UV3 represents the color information of fill light frame j+3. When generating synthetic frame k, the processor uses the brightness information Y0 of frame j and the color information UV1 of frame j+1. Similarly, the synthetic frame k+1 uses the brightness information Y2 of frame j+2 and the color information UV3 of frame j+3.

[0050] If the synthesized frame image is recorded as I' and the image frame collected by the image sensor is recorded as I, then the relationship between I and I' of the nth frame (n is a natural number) can be expressed by the following formula:

[0051] I′ Y (n) = I Y (2n-1)

[0052] I′ UV (n) = I UV (2n)

[0053] or

[0054] I′ Y (n) = I Y (2n)

[0055] I′ UV (n) = I UV (2n-1)

[0056] In the present embodiment, the frequency of acquiring fill light frames is to acquire a fill light frame every other visible light frame, that is, odd-numbered frames are visible light frames and even-numbered frames are fill light frames, or odd-numbered frames are fill light frames and even-numbered frames are visible light frames. Frame synthesis adopts a method of synthesizing a visible light frame and a fill light frame into a synthesized frame. It should be noted that if other fill light frame acquisition frequencies are adopted, such as acquiring a fill light frame every two visible light frames, or acquiring two fill light frames every other visible light frame, etc., the technical solutions recorded in the present application are also applicable. Accordingly, if the synthesized frames adopt other synthesis forms, such as generating a synthesized frame from a visible light frame and two fill light frames, etc., the technical solutions recorded in the present application are also applicable. In addition, if frame synthesis is performed in other color spaces, such as in RGB, HSV, Lab and other spaces, the principle is the same. Figure 2b The process is similar to that shown in , and will not be repeated here.

[0057] See also Figure 3 , which is an embodiment of the time-division multiplexing fill-light imaging method of the present invention.

[0058] S210 enables the fill light imaging mode, increases the acquisition frame rate, and switches the filter. The imaging device monitors the ambient illumination in real time. When the ambient illumination is equal to or lower than the maximum illumination required for fill light, the fill light imaging method of the present invention is enabled. The ambient illumination can be estimated by using the gain information of the image sensor. As the ambient illumination decreases, the gain of the image sensor will gradually increase. Therefore, a certain gain value of the sensor can be set as a judgment standard (such as 36dB) as the maximum illumination judgment value for the ambient illumination that requires fill light. When the gain is equal to or higher than this preset value, it is determined that the ambient illumination is equal to or lower than the maximum illumination required for fill light, and the fill light imaging method of the present invention is enabled. In some possible implementations, changes in ambient light illumination can also be detected by a special brightness detection device (such as a photoresistor), which is not limited here. Increase the acquisition frame rate and switch the filter. When the fill light imaging mode is enabled, the imaging device increases the frame rate of the captured image, for example, from 25 frames / second to 50 frames / second. Since each pair of consecutive odd and even frames is synthesized to generate a synthetic frame during the generation process of the synthetic frame, increasing the acquisition frame rate can reduce or eliminate the decrease in the final output frame rate. In this embodiment, when the acquisition frame rate is doubled, the final output frame rate will be the same as the output frame rate before the fill light mode is adopted, thereby ensuring the quality of the image output. When the fill light imaging mode is enabled, the imaging device will also switch the filter to a dual-pass filter to filter out other light in the collected light except visible light and infrared light, thereby improving the imaging quality. In other possible implementations, S220 may only increase the acquisition frame rate without switching the filter.

[0059] S220 periodically turns on and off the infrared fill light source to alternately generate fill light frames and visible light frames. After the fill light imaging mode is enabled, the imaging device periodically turns on and off the infrared fill light source according to the generation frequency of the image frames. The periodic turning on and off of the infrared fill light source here can be turned on for odd frames and turned off for even frames, or turned off for odd frames and turned on for even frames. Correspondingly, in the former case, all odd frames in the image frames generated by the image sensor are fill light frames and even frames are visible light frames, and in the latter case, all odd frames are visible light frames and even frames are fill light frames.

[0060] The synchronization of the switching frequency of the infrared fill light source and the image frame generation frequency can be achieved through the frame interrupt signal generated by the image sensor. When the image sensor generates an image frame, a frame interrupt signal is generated to represent the start and end of the frame exposure. Specifically, the frame start and frame end information of the image frame are recorded in the frame interrupt signal. Possible implementation forms of the frame interrupt signal include but are not limited to fstart (ISP_FE), fstart (ISP_BE), cfg_loss (ISP_FE), cfg_loss (ISP_BE), etc. When the fill light imaging mode is turned on, the image sensor sends the generated frame interrupt signal to the processor, and the processor counts the frame interrupt signal from 0, and the count increases by 1 each time a frame interrupt signal is received. In a possible implementation, when the frame interrupt count is an odd number, the processor turns on the infrared fill light source when it detects the frame start signal, and the infrared fill light source starts to fill light the scene; when the frame end signal is detected, the infrared fill light source is turned off, and the fill light ends. When the frame interrupt count is an even number, the fill light source is not operated. In this way, all odd frames in the image frames generated by the image sensor are fill light frames, and all even frames are visible light frames. In another possible implementation, when the frame interruption count is an even number, the processor turns on the infrared fill light source when detecting a frame start signal, and the infrared fill light source starts to fill light the scene; when detecting a frame end signal, the infrared fill light source is turned off, and the fill light ends. When the frame interruption count is an odd number, the fill light source is not operated. In this way, all even-numbered frames in the image frames generated by the image sensor are fill light frames, and all odd-numbered frames are visible light frames.

[0061] S230 performs basic processing on the image and generates a composite frame. The imaging device performs basic processing on the image frame, including but not limited to de-mosaicing and automatic white balance. Generating a composite frame refers to synthesizing the visible light frame and the fill light frame to generate a composite frame. The brightness information of the composite frame comes from the fill light frame, and the chromaticity information comes from the visible light frame. For specific synthesis methods, please refer to Figure 2a and Figure 2bThere is no order limit between basic image processing and generating synthetic frames. The basic processing can be performed on the visible light frame and the fill light frame before generating the synthetic frame, or the synthetic frame can be generated first and then the basic processing is performed on the synthetic frame. There is no limit here.

[0062] S240 outputs the synthesized frame image.

[0063] See also Figure 4 , which is another embodiment of the time-division multiplexing fill light imaging method of the present invention.

[0064] S310 enables the fill light imaging mode, increases the acquisition frame rate, and turns on the fill light source. The specific activation method of the fill light imaging mode is the same as Figure 2, and will not be repeated here. When the fill light imaging mode is enabled, the imaging device increases the frame rate of the acquired image, for example, from 25 frames / second to 50 frames / second. In this embodiment, when the acquisition frame rate is doubled, the final output frame rate will be the same as the output frame rate before the fill light mode is adopted, thereby ensuring the quality of the image output. When the fill light imaging mode is enabled, the imaging device will also turn on the fill light source to perform infrared fill light on the subject.

[0065] S320 periodically switches the filter to alternately generate visible light frames and fill light frames. After the fill light imaging mode is enabled, the imaging device periodically switches the single-pass filter and the double-pass filter according to the generation frequency of the image frame. The single-pass filter here refers to a filter that can only pass visible light, and the double-pass filter refers to a filter that can only pass visible light and infrared light. The periodic switching of the filter here can be switching the single-pass filter for odd frames and switching the double-pass filter for even frames, or switching the double-pass filter for odd frames and switching the single-pass filter for even frames. Correspondingly, in the former case, all odd frames in the image frames generated by the image sensor are visible light frames and the even frames are fill light frames, and in the latter case, all odd frames are fill light frames and even frames are visible light frames.

[0066] The synchronization of the filter switching frequency and the image frame generation frequency can be achieved through the frame interrupt signal generated by the image sensor. The specific characteristics of the frame interrupt signal are as follows: Figure 3, which will not be repeated here. When the fill light imaging mode is turned on, the image sensor sends the generated frame interrupt signal to the processor, and the processor counts the frame interrupt signal from 0, and the count is increased by 1 each time a frame interrupt signal is received. In one possible implementation, when the frame interrupt count is an odd number, the processor switches the filter to a single-pass filter when detecting the frame start signal, and only visible light can be irradiated on the image sensor; when the frame interrupt count is an even number, the processor switches the filter to a dual-pass filter when detecting the frame start signal, and both visible light and infrared light can be irradiated on the image sensor. In this way, all even-numbered frames in the image frame generated by the image sensor are fill light frames, and all odd-numbered frames are visible light frames. In another possible implementation, when the frame interrupt count is an odd number, the processor switches the filter to a dual-pass filter when detecting the frame start signal, and both visible light and infrared light can be irradiated on the image sensor; when the frame interrupt count is an even number, the processor switches the filter to a single-pass filter when detecting the frame start signal, and only visible light can be irradiated on the image sensor. In this way, all odd-numbered frames in the image frames generated by the image sensor are fill light frames, and all even-numbered frames are visible light frames.

[0067] S330 performs basic processing on the image and generates a composite frame. Figure 3 The description in will not be repeated here.

[0068] S340 outputs the synthesized frame image.

[0069] See also Figure 5 , is a schematic diagram of the logical structure of a possible embodiment of the time-division multiplexing fill-light imaging device of the present invention.

[0070] like Figure 5 As shown, the imaging device 400 includes a fill light unit 410, an optical unit 420, an image sensing unit 430, and a processing unit 440. In some possible implementations, an ambient light detection unit 450 may also be included.

[0071] The fill light unit 410 is used to fill light for the photographed object, and can be implemented as an infrared light emitting diode (LED) fill light, and the emission center wavelength that can be used includes but is not limited to 760nm, 850nm or 950nm. The fill light unit 410 can be turned on and off under the control of the processing unit 440.

[0072] The optical unit 420 is used to capture light to form an image on the image sensing unit 430, and can be implemented as a lens, including but not limited to a wide-angle lens, a standard lens, a telephoto lens, a zoom lens, a retrofocus lens, or a pinhole lens, etc. In some possible implementations, the optical unit 420 may also include a filter so that only light of a specific wavelength can be irradiated onto the image sensing unit 430.

[0073] The image sensing unit 430 is used to convert the received optical signal into an electrical signal, and generate a digital image signal with frames as the separation unit based on the electrical signal. The image sensing unit 430 can realize the conversion between optical signals and electrical signals through a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). The process in which the image sensing unit 430 finally converts the received optical signal into a digital image signal with frames as the separation unit can be called image exposure. The frame generated by exposure when receiving visible light and not receiving fill light infrared light is called a visible light frame, and the frame generated by exposure when receiving fill light infrared light is called a fill light frame. When generating a fill light frame, the image sensing unit 430 can receive fill light infrared light but not visible light, or can receive fill light infrared light and visible light at the same time, which is not limited here.

[0074] The processing unit 440 is used to control the opening and closing of the fill light unit 410. In some possible implementations, the processing unit 440 periodically opens and closes the fill light unit according to the image frame exposure information sent by the image sensor unit 430, so that the image sensor 430 periodically obtains infrared fill light, and then alternately generates fill light frames and visible light frames. The image frame exposure information can be implemented as frame interrupt information including a frame start signal and a frame end signal. The synchronization method of the opening and closing frequency of the fill light unit 410 and the frame exposure frequency can be seen in Figure 3 The corresponding description in , will not be repeated here. The processing unit 440 is also used to control the acquisition frame rate of the image sensing unit 430, basic processing of the digital image signal, and synthesis of the visible light frame and the fill light frame. The basic processing of the image includes but is not limited to de-mosaicing, automatic white balance, etc. The synthesis of the visible light frame and the fill light frame refers to generating a synthetic frame through the visible light frame and the fill light frame, the brightness information of the synthetic frame comes from the fill light frame, and the chromaticity information comes from the visible light frame. The specific synthesis method is as follows Figure 2a and Figure 2b The functions of the processing unit 440 may be implemented by one processor or by multiple processors. In some possible implementations, the functions of the processing unit 440 may be partially or completely integrated into the image sensing unit 430, which is not limited here.

[0075] The ambient light detection unit 450 is used to detect the ambient light illumination and provide the illumination information to the processing unit 440. In some possible implementations, the function of the ambient light detection unit 450 may also be implemented by the image sensing unit 430, which is not limited here.

[0076] See also Figure 6 , which is a schematic diagram of the logical structure of another possible embodiment of the time-division multiplexing fill-light imaging device of the present invention.

[0077] like Figure 6 As shown, the imaging device 500 includes a fill light unit 510 , an optical unit 520 , a light filtering control unit 540 , an image sensing unit 530 and a processing unit 550 .

[0078] The fill light unit 510 is used to fill light for the photographed object, and can be implemented as an infrared light emitting diode (LED) fill light, and the emission center wavelength that can be used includes but is not limited to 760nm, 850nm or 950nm. When the fill light imaging device 500 performs fill light imaging, the fill light unit 510 remains turned on.

[0079] The optical unit 520 is used to capture light to form an image on the image sensing unit 530, and can be implemented as a lens, including but not limited to a wide-angle lens, a standard lens, a telephoto lens, a zoom lens, a retrofocus lens or a pinhole lens.

[0080] The image sensing unit 530 is used to convert the received optical signal into an electrical signal, and generate a digital image signal with frames as the separation unit based on the electrical signal. The image sensing unit 530 can realize the conversion between optical signals and electrical signals through a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). The process in which the image sensing unit 530 finally converts the received optical signal into a digital image signal with frames as the separation unit can be called image exposure. The frame generated by exposure when receiving visible light and not receiving fill light infrared light is called a visible light frame, and the frame generated by exposure when receiving fill light infrared light is called a fill light frame. When generating a fill light frame, the image sensing unit 530 can only receive infrared light, or can also receive infrared light and visible light at the same time, which is not limited here.

[0081] The filter control unit 540 is used to allow the light of a specific wavelength collected by the optical unit 520 to irradiate the image sensing unit 530, while filtering out light of other wavelengths. The filter control unit 540 can change the filter characteristics as needed, so that light of different wavelengths can pass through and irradiate the image sensing unit 530. The filter control unit 540 can be implemented as a filter switching device, which changes the filter characteristics by switching filters. In some possible implementations, the filter control unit 540 periodically changes the filter characteristics according to the instructions of the processing unit 550, so that the image sensing unit 530 periodically obtains infrared fill light, and then alternately generates fill light frames and visible light frames. The image frame exposure information can be implemented as frame interrupt information including frame exposure start and exposure end signals. For the synchronization method of the change frequency of the filter characteristics of the filter control unit 540 and the frame exposure frequency, please refer to Figure 4 The corresponding description in will not be repeated here.

[0082] The processing unit 550 is used for basic processing of digital image signals, controlling the filter control unit 540 and synthesizing visible light frames and fill light frames. Basic image processing includes but is not limited to de-mosaicing, automatic white balance, etc. The processing unit 550 controls the filter control unit 540 to periodically change the filter characteristics according to the image frame exposure information sent by the image sensor unit 530, so that the image sensor unit 530 periodically obtains infrared fill light, and then alternately generates fill light frames and visible light frames. The synthesis of visible light frames and fill light frames by the processing unit 550 refers to generating a synthesized frame through visible light frames and fill light frames, the brightness information of the synthesized frame comes from the fill light frame, and the chromaticity information comes from the visible light frame. The specific synthesis method is as follows: Figure 2a and Figure 2b The functions of the processing unit 550 may be implemented by one processor or by multiple processors. In some possible implementations, the functions of the processing unit 550 may be partially or completely integrated into the image sensing unit 530, which is not limited here.

[0083] The ambient light detection unit 560 is used to detect the ambient light illumination and provide the illumination information to the processing unit 550. In some possible implementations, the function of the ambient light detection unit 560 may also be implemented by the image sensing unit 530, which is not limited here.

[0084] See also Figure 7 , is a schematic diagram of the logical structure of a possible embodiment of the time-division multiplexing fill-light imaging terminal device of the present invention.

[0085] The terminal device 600 includes an imaging module 610 and a main processing module 620. In some possible implementations, it may also include a communication module 630. The specific implementation of the terminal device 600 includes but is not limited to a mobile phone with a shooting function, a tablet computer, a digital camera, etc.

[0086] The imaging module 610 is used to generate a synthetic frame using the time division multiplexing fill light imaging method described in this application. For specific synthetic frame generation methods, please refer to Figure 3 or Figure 4 The relevant description of the internal structure is not repeated here. The implementation of the internal structure includes but is not limited to the following Figure 1 , Figure 5 or Figure 6 The specific forms described in the above are not described in detail here. After acquiring the synthesized frame, the imaging module 610 may send the synthesized frame to the main processing module 620 for further processing, or send it to the communication module 630 for communication.

[0087] The main processing module 620 is used to perform image processing on the synthesized frame, including but not limited to geometric transformation, color adjustment, image splicing, etc. After completing the image processing, the main processing module can send the processed image to the communication module 630 for communication.

[0088] The communication module 630 is used to send the image or video corresponding to the synthesized frame to other devices through interface transmission or wireless transmission.

[0089] The functions of each processor mentioned in the embodiments of the present invention can be implemented in the form of a complete hardware embodiment, a complete software embodiment (including firmware, resident software and microcode, etc.) or a combination of software and hardware embodiments, wherein all of these forms can generally be referred to as "circuit", "module" or "system". Each processor can be implemented by a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP) and a single or combined general-purpose processor, as well as associated software, firmware and glue logic. All processors can include one or more processors. When multiple processors are included, the multiple processors can be integrated on the same chip or can be independent chips. A processor can include one or more physical cores, wherein the physical core is the smallest processing module.

[0090] It should be understood that the references to "one embodiment", "an embodiment", "an implementation" or "some implementations" throughout the specification mean that the specific features, structures or characteristics associated with the embodiment are included in at least one embodiment of the present invention. Therefore, the "in one embodiment" or "in an embodiment" appearing in various places throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics may be combined in one or more embodiments in any suitable manner.

[0091] In the embodiments provided in the present application, it should be understood that "B corresponding to A" means that B is associated with A, and B can be determined according to A. However, it should also be understood that determining B according to A does not mean determining B only according to A, and B can also be determined according to A and / or other information.

[0092] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. In addition, each functional unit in each embodiment of the present invention can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

Claims

1. A fill light imaging device, characterized in that: It includes an image sensor, a fill light unit and a processor, wherein: The processor is used to control the filter switching device to place the single-pass filter on the imaging light path of the optical unit when the fill light imaging mode is not turned on; the single-pass filter is used to filter out other types of light in the imaging light path except visible light; The processor is used to control the filter switching device to place a double-pass filter on the imaging light path of the optical unit when the fill light imaging mode is turned on; the double-pass filter is used to filter out other light in the imaging light path except visible light and fill light; and is also used to control the on or off of the fill light unit; When the fill light unit is turned on, the image sensor generates a fill light frame; When the fill light unit is turned off, the image sensor generates a visible light frame; The processor is specifically used for: The fill light unit is controlled to be turned on or off according to a frame interrupt signal, and the frame interrupt signal is used to indicate the start and end of frame exposure.

2. The fill light imaging device according to claim 1, characterized in that: The processor is specifically used for: When a frame interruption signal indicating a frame start signal is detected, turning on the fill light unit; When a frame interruption signal indicating a frame end signal is detected, turning off the fill light unit; or, When a frame interruption signal indicating a frame end signal is detected, turning on the fill light unit; When a frame interruption signal indicating a frame start signal is detected, the fill light unit is turned off.

3. The fill light imaging device according to claim 1 or 2, characterized in that: The processor is further configured to: Adjacent or continuous visible light frames and fill light frames are synthesized to obtain a synthesized frame.

4. The fill light imaging device according to claim 3, characterized in that: The processor is specifically used for: A composite frame is obtained by synthesizing an adjacent visible light frame and a fill light frame; or, One or more continuous visible light frames are synthesized with one or more fill light frames to obtain a synthesized frame.

5. The fill light imaging device according to claim 3, characterized in that: The color information of the synthesized frame adopts the color information of the visible light frame, and the brightness information of the synthesized frame adopts the brightness information of the fill light frame.

6. The fill light imaging device according to claim 3, characterized in that: The processor is further configured to: In the fill light imaging mode, the image frame acquisition frame rate of the image sensor is increased.

7. The fill light imaging device according to claim 6, characterized in that: The processor is specifically used for: In the fill light imaging mode, the image frame acquisition frame rate of the image sensor is increased to n times that when the fill light imaging mode is not turned on, where n is the sum of the number of the visible light frames and the fill light frames required to generate one synthetic frame.

8. The fill light imaging device according to claim 1 or 2, characterized in that: The case where the fill light imaging mode is not turned on specifically includes: when the ambient light illumination is equal to or less than a preset value; or, When the gain value of the image sensor is equal to or less than the preset value.

9. The fill light imaging device according to claim 1 or 2, characterized in that: The fill light imaging device further includes an ambient light detection unit for detecting the ambient light illumination; the fill light imaging mode is activated specifically including: When the ambient light detection unit detects that the ambient light illumination is equal to or less than a preset value, the fill light imaging mode is turned on; or, The image sensor is also used to detect the ambient light illumination through the gain value of the image sensor; the fill light imaging mode is turned on specifically including: When it is detected that the gain value of the image sensor is equal to or less than a preset value, the fill light imaging mode is turned on.

10. A fill light imaging device, characterized in that: It includes an image sensor, a fill light unit and a processor, wherein: The processor is used to control the filter switching device to place the double-pass filter on the imaging light path of the optical unit when the fill light imaging mode is turned on; the double-pass filter is used to filter out other light in the imaging light path except visible light and fill light; The processor is further configured to increase the image frame acquisition frame rate of the image sensor in the fill light imaging mode; The processor is further used to control the on or off of the fill light unit; When the fill light unit is turned on, the image sensor generates a fill light frame; When the fill light unit is turned off, the image sensor generates a visible light frame; The processor is specifically used for: The fill light unit is controlled to be turned on or off according to a frame interrupt signal, and the frame interrupt signal is used to indicate the start and end of frame exposure.

11. The fill light imaging device according to claim 10, characterized in that: When the image frame acquisition frame rate of the image sensor is increased, the number of synthetic frames generated per unit time increases.

12. The fill light imaging device according to claim 11, characterized in that: The processor is specifically used for: In the fill light imaging mode, the image frame acquisition frame rate of the image sensor is increased to n times that when the fill light imaging mode is not turned on, where n is the sum of the number of the visible light frames and the fill light frames required to generate one synthetic frame.

13. The fill light imaging device according to claim 10, characterized in that: The processor is specifically used for: When a frame interruption signal indicating a frame start signal is detected, turning on the fill light unit; When a frame interruption signal indicating a frame end signal is detected, turning off the fill light unit; or, When a frame interruption signal indicating a frame end signal is detected, turning on the fill light unit; When a frame interruption signal indicating a frame start signal is detected, the fill light unit is turned off.

14. The fill light imaging device according to any one of claims 10 to 13, characterized in that: The processor is further configured to: A composite frame is obtained by synthesizing an adjacent visible light frame and a fill light frame; or, One or more continuous visible light frames are synthesized with one or more fill light frames to obtain a synthesized frame.

15. The fill light imaging device according to claim 14, characterized in that: The color information of the synthesized frame adopts the color information of the visible light frame, and the brightness information of the synthesized frame adopts the brightness information of the fill light frame.

16. The fill light imaging device according to any one of claims 10 to 13, characterized in that: The fill light imaging device further includes an ambient light detection unit for detecting the ambient light illumination; the fill light imaging mode is activated specifically including: When the ambient light detection unit detects that the ambient light illumination is equal to or less than a preset value, the fill light imaging mode is turned on; or, The image sensor is also used to detect the ambient light illumination through the gain value of the image sensor; the fill light imaging mode is turned on specifically including: When it is detected that the gain value of the image sensor is equal to or less than a preset value, the fill light imaging mode is turned on.

17. The fill light imaging device according to any one of claims 10 to 13, characterized in that: The processor is also used to control the filter switching device to place a single-pass filter on the imaging light path of the optical unit when the fill light imaging mode is not turned on; the single-pass filter is used to filter out other types of light in the imaging light path except visible light.

18. A method for fill light imaging, characterized in that: include: When the fill light imaging mode is not turned on, a single-pass filter is placed on the imaging light path of the optical unit; the single-pass filter is used to filter out other types of light in the imaging light path except visible light; When the fill light imaging mode is turned on, a double-pass filter is placed on the imaging light path of the optical unit; the double-pass filter is used to filter out other light in the imaging light path except visible light and fill light; Alternately turn on or off the fill light unit; When the fill light unit is turned on, the image sensor generates a fill light frame; When the fill light unit is turned off, the image sensor generates a visible light frame; The alternately turning on or off the fill light unit specifically includes: The fill light unit is alternately turned on or off according to a frame interrupt signal, and the frame interrupt signal is used to indicate the start and end of frame exposure.

19. The method for supplementary light imaging according to claim 18, characterized in that: The alternately turning on or off the fill light unit according to the frame interrupt signal comprises: When a frame interruption signal indicating a frame start signal is detected, turning on the fill light unit; When a frame interruption signal indicating a frame end signal is detected, turning off the fill light unit; or, When a frame interruption signal indicating a frame end signal is detected, turning on the fill light unit; When a frame interruption signal indicating a frame start signal is detected, the fill light unit is turned off.

20. The method for supplementary light imaging according to claim 18, wherein: The method further comprises: A composite frame is obtained by synthesizing an adjacent visible light frame and a fill light frame; or, One or more continuous visible light frames are synthesized with one or more fill light frames to obtain a synthesized frame.

21. The method for supplementary light imaging according to claim 20, characterized in that: The color information of the synthesized frame adopts the color information of the visible light frame, and the brightness information of the synthesized frame adopts the brightness information of the fill light frame.

22. The method for supplementary light imaging according to claim 20, characterized in that: The method further comprises: In the fill light imaging mode, the image frame acquisition frame rate of the image sensor is increased.

23. The method for supplementary light imaging according to claim 22, characterized in that: In the fill light imaging mode, increasing the image frame acquisition frame rate of the image sensor includes: In the fill light imaging mode, the image frame acquisition frame rate of the image sensor is increased to n times that when the fill light imaging mode is not turned on, where n is the sum of the number of the visible light frames and the fill light frames required to generate one synthetic frame.

24. The method for supplementary light imaging according to any one of claims 18 to 23, characterized in that: The method further comprises: When it is detected that the ambient light illumination is equal to or less than a preset value, the fill light imaging mode is turned on; or, When it is detected that the gain value of the image sensor is equal to or less than a preset value, the fill light imaging mode is turned on.

25. A method for fill light imaging, characterized in that: include: When the fill light imaging mode is turned on, a double-pass filter is placed on the imaging light path of the optical unit; the double-pass filter is used to filter out other light in the imaging light path except visible light and fill light; In the fill light imaging mode, increasing the image frame acquisition frame rate of the image sensor; Alternately turn on or off the fill light unit; When the fill light unit is turned on, the image sensor generates a fill light frame; When the fill light unit is turned off, the image sensor generates a visible light frame; The alternately turning on or off the fill light unit specifically includes: The fill light unit is alternately turned on or off according to a frame interrupt signal, and the frame interrupt signal is used to indicate the start and end of frame exposure.

26. The method for supplementary light imaging according to claim 25, characterized in that: When the image frame acquisition frame rate of the image sensor is increased, the number of synthetic frames generated per unit time increases.

27. The method for supplementary light imaging according to claim 26, characterized in that: In the fill light imaging mode, increasing the image frame acquisition frame rate of the image sensor includes: In the fill light imaging mode, the image frame acquisition frame rate of the image sensor is increased to n times that when the fill light imaging mode is not turned on, where n is the sum of the number of the visible light frames and the fill light frames required to generate one synthetic frame.

28. The method for supplementary light imaging according to claim 25, characterized in that: The alternately turning on or off the fill light unit according to the frame interrupt signal comprises: When a frame interruption signal indicating a frame start signal is detected, turning on the fill light unit; When a frame interruption signal indicating a frame end signal is detected, turning off the fill light unit; or, When a frame interruption signal indicating a frame end signal is detected, turning on the fill light unit; When a frame interruption signal indicating a frame start signal is detected, the fill light unit is turned off.

29. The method for supplementary light imaging according to any one of claims 25 to 28, characterized in that: The method further comprises: A composite frame is obtained by synthesizing an adjacent visible light frame and a fill light frame; or, One or more continuous visible light frames are synthesized with one or more fill light frames to obtain a synthesized frame.

30. The method for supplementary light imaging as claimed in claim 29, characterized in that: The color information of the synthesized frame adopts the color information of the visible light frame, and the brightness information of the synthesized frame adopts the brightness information of the fill light frame.

31. The method for supplementary light imaging according to any one of claims 25 to 28, characterized in that: The method further comprises: When it is detected that the ambient light illumination is equal to or less than a preset value, the fill light imaging mode is turned on; or, When it is detected that the gain value of the image sensor is equal to or less than a preset value, the fill light imaging mode is turned on.

32. The method for supplementary light imaging according to any one of claims 25 to 28, characterized in that: The method further comprises: When the fill light imaging mode is not turned on, a single-pass filter is placed on the imaging light path of the optical unit; the single-pass filter is used to filter out other types of light in the imaging light path except visible light.

33. A fill light imaging device, characterized in that: It includes an image sensor, a fill light unit and a processor, wherein: The fill light unit is used to provide fill light for the photographed object; The processor is used for controlling the filter switching device to switch the single-pass filter or the double-pass filter to be placed on the imaging light path of the optical unit when the fill light unit is turned on, so that the image sensor generates visible light frames or fill light frames alternately; The processor is specifically used for: The fill light unit is controlled to be turned on or off according to a frame interrupt signal, and the frame interrupt signal is used to indicate the start and end of frame exposure.

34. The supplementary light imaging device according to claim 33, characterized in that: The processor controls the filter switching device to switch the single-pass filter to be placed on the imaging light path of the optical unit, and the image sensor generates the visible light frame; The processor controls the filter switching device to switch the dual-pass filter to be placed on the imaging light path of the optical unit, and the image sensor generates the fill light frame.

35. The supplementary light imaging device according to claim 33 or 34, characterized in that: The visible light frame and the fill light frame are used to synthesize to obtain a synthesized frame.

36. A method for fill light imaging, characterized in that: include: When the fill light unit is turned on, the filter switching device is controlled to place a single-pass filter or a double-pass filter on the imaging light path of the optical unit, so that the image sensor generates visible light frames or fill light frames alternately; The control filter switching device places a single-pass filter or a double-pass filter on the imaging light path of the optical unit so that the image sensor alternately generates the visible light frame or the fill light frame, specifically comprising: Controlling the filter switching device to switch the single-pass filter to be placed on the imaging light path of the optical unit, so that the image sensor generates the visible light frame; Controlling the filter switching device to switch the dual-pass filter to be placed on the imaging light path of the optical unit, and the image sensor to generate the fill light frame; The fill light unit is alternately turned on or off according to a frame interrupt signal, and the frame interrupt signal is used to indicate the start and end of frame exposure.

37. The method for supplementary light imaging as claimed in claim 36, characterized in that: The visible light frame and the fill light frame are used to synthesize to obtain a synthesized frame.

Images