Imaging apparatus, imaging system, and imaging method

Abstract

The invention provides an imaging device which is applied to the field of image acquisition. The imaging device comprises a lens, a motor, a color wheel and a rolling shutter sensor. The lens is used for receiving the first incident light and transmitting the first incident light to the color wheel. The motor drives the color wheel to rotate. The color wheel sequentially comprises an infrared light cut-off area, a first shielding area, an infrared light penetrating area and a second shielding area in the rotating direction. The color wheel is used for intermittently transmitting the first visible light and the first infrared light to the rolling shutter sensor. The rolling shutter sensor is used for obtaining a first visible light image frame according to the first visible light and obtaining a first infrared light image frame according to the first infrared light. According to the technical scheme provided by the invention, the visible light image frame and the infrared light image frame can be obtained through the imaging sensor in a time division multiplexing manner, so that the manufacturing cost of the imaging device is reduced.

Description

Technical Field

[0001] The present application relates to the field of image acquisition, and in particular to an imaging device, an imaging system and an imaging method. Background Art

[0002] In traffic capture cameras, since it is necessary to capture high-speed moving vehicles, it is impossible to increase the amount of light entering by increasing the exposure time. For this reason, fill light is required through a fill light device. The fill light device outputs visible light and infrared light. The lens in the capture camera receives the light to be imaged. The light to be imaged includes reflected light of visible light and reflected light of infrared light. The capture camera splits the light to be imaged into visible light and infrared light through a beam splitter prism. The capture camera converts infrared light into infrared light image frames through an infrared light imaging sensor, and converts visible light into visible light image frames through a visible light imaging sensor. In practical applications, the capture camera requires two imaging sensors, which leads to a high manufacturing cost of the capture camera. Summary of the invention

[0003] The present application provides an imaging device, an imaging system and an imaging method. By rotating a color wheel, visible light image frames and infrared light image frames can be acquired by time-division multiplexing through one imaging sensor, thereby reducing the manufacturing cost of the imaging device.

[0004] In a first aspect, the present application provides an imaging device. The imaging device includes a lens, a motor, a color wheel, and a rolling shutter sensor. The lens is used to receive a first incident light and transmit the first incident light to the color wheel. The motor is used to drive the color wheel to rotate. The color wheel is located on the light transmission path between the lens and the rolling shutter sensor. The color wheel includes an infrared light cut-off area, a first shielding area, an infrared light penetration area, and a second shielding area in order according to the rotation direction. The first shielding area and the second shielding area shield visible light and infrared light. The infrared light cut-off area shields infrared light. The infrared light penetration area is a visible light cut-off area or a full-pass area. The visible light cut-off area shields visible light. The full-pass area allows visible light and infrared light to pass. The color wheel is used to receive the first incident light. When the first incident light irradiates the infrared light cut-off area, the color wheel is used to shield the infrared light in the first incident light and transmit the first visible light to the rolling shutter sensor. When the first incident light irradiates the second shielding area, the color wheel is used to shield the infrared light and visible light in the first incident light and stop transmitting the light beam to the rolling shutter sensor. When the first incident light irradiates the visible light cut-off area, the color wheel is used to block the visible light in the first incident light and transmit the first infrared light to the rolling shutter sensor. When the first incident light irradiates the first blocking area, the color wheel is used to block the infrared light and visible light in the first incident light and stop transmitting the light beam to the rolling shutter sensor. Therefore, the color wheel is used to intermittently transmit the first visible light and the first infrared light to the rolling shutter sensor. The rolling shutter sensor is used to obtain a first visible light image frame according to the first visible light and obtain a first infrared light image frame according to the first infrared light. Among them, the first visible light image frame is an image frame obtained according to the first visible light. The first infrared light image frame is an image frame obtained according to the first infrared light.

[0005] In an optional manner of the first aspect, the lens is used to receive a first incident light in a first time period. The lens is also used to receive a second incident light in a second time period, and transmit the second incident light to the color wheel. The intensity of the visible light in the second time period is greater than the intensity of the visible light in the first time period. For example, the first time period is night, and the second time period is day. The color wheel is used to receive the second incident light, and intermittently transmit the second visible light to the rolling shutter sensor. The rolling shutter sensor is used to obtain a second visible light image frame based on the second visible light. In the second time period, the imaging device does not acquire infrared light image frames. In the present application, the rolling shutter sensor only generates visible light image frames in the second time period, thereby reducing the operating power consumption of the imaging device.

[0006] In an optional manner of the first aspect, the second visible light corresponds to the infrared light cutoff area. In practical applications, the second visible light can also correspond to the infrared light penetration area. However, the size of the infrared light penetration area and the size of the infrared light cutoff area may be different, thereby affecting the exposure time of the second visible light image frame. In addition, the size of the first blocking area and the size of the second blocking area may also be different, thereby affecting the reading time of the second visible light image frame. Therefore, in the present application, by making the second visible light correspond to the infrared light cutoff area, the imaging quality of the second visible light image frame can be improved.

[0007] In an optional manner of the first aspect, the imaging device further includes a dual filter, and the dual filter is located on the light transmission path between the lens and the color wheel. The dual filter is an all-pass filter in the first time period, and the dual filter is an infrared light cutoff filter in the second time period. When the infrared light penetration area is the visible light cutoff area, in the second time period, due to the existence of the dual filter and the visible light cutoff area, the visible light cutoff area is equivalent to the blocking area. At this time, the color wheel includes three continuous blocking areas and an infrared light cutoff area. The angles of the three continuous blocking areas are larger than the angle of the second blocking area, thereby providing more reading time for the second visible light image frame and improving the imaging quality of the second visible light image frame.

[0008] In an optional manner of the first aspect, the rotation speed of the color wheel in the second time period is twice the rotation speed of the color wheel in the first time period. In practical applications, the maximum output frame rate of the rolling shutter sensor is generally fixed. In the present application, by controlling the rotation speed of the color wheel, the output frame rate of the rolling shutter sensor in the first time period and the second time period is the same. Therefore, the present application can improve the use efficiency of the rolling shutter sensor.

[0009] In an optional manner of the first aspect, the exposure time of the first visible light image frame is greater than the exposure time of the first infrared light image frame. By reducing the exposure time of the infrared light image frame, the imaging device can reduce the fill light time requirement of the fill light device, thereby reducing the manufacturing cost of the imaging system.

[0010] In an optional manner of the first aspect, the angle of the infrared light cut-off area is greater than the angle of the infrared light penetration area. When the infrared light cut-off area and the infrared light penetration area are fan-shaped, the angle of the infrared light cut-off area and the angle of the infrared light penetration area refer to the size of the central angle of the fan-shaped area. By reducing the angle of the infrared light penetration area, the angle of the shielding area can be increased. The angle of the shielding area is related to the reading time of the image frame. Therefore, by increasing the angle of the shielding area, the imaging quality of the image frame can be improved.

[0011] In an optional manner of the first aspect, the angle of the infrared light penetration area is between 10 and 15 degrees, and the angle of the infrared light cutoff area is between 20 and 30 degrees.

[0012] In an optional manner of the first aspect, the angle of the first shielding area is smaller than the angle of the second shielding area. By reducing the angle of the first shielding area, the interval between the first visible light image frame and the first infrared light image frame obtained by the rolling shutter sensor can be reduced. The smaller the interval between the two image frames, the more conducive to subsequent image fusion. Therefore, the present application can simplify the difficulty of image fusion.

[0013] In an optional manner of the first aspect, the infrared light penetration region is a visible light cutoff region. By shielding the visible light, the influence of the visible light on the first infrared light image frame can be reduced. Therefore, the present application can improve the imaging quality of the first infrared light image frame.

[0014] In an optional manner of the first aspect, the imaging device further includes a processing circuit. The processing circuit is used to perform image fusion on the first visible light image frame and the first infrared light image frame to obtain a target image frame.

[0015] In an optional manner of the first aspect, the processing circuit is used to perform image fusion on a region of interest (ROI) of a first infrared light image frame and a first visible light image frame. The angle of the first occluded area is between t1 and t2 degrees. Wherein, t1=X-7, t2=X+20, X=ω×d×Δt. ω represents the rotation speed of the color wheel. d represents the number of pixel rows included in the ROI. Δt represents the readout time of each row of pixels. The readout time of the ROI is less than the readout time of the first infrared light image frame. By reducing the readout time, the angle of the first occluded area can be reduced. By reducing the angle of the first occluded area, the interval between the first visible light image frame and the ROI obtained by the rolling shutter sensor can be reduced. The smaller the interval between the two image frames, the more conducive to subsequent image fusion. Therefore, the present application can simplify the difficulty of image fusion.

[0016] In an optional manner of the first aspect, the processing circuit is used to perform image fusion on the W pixels or G pixels in the region of interest ROI of the first infrared light image frame and the first visible light image frame. The angle of the first occlusion area is between t1 and t2 degrees. Wherein, t1=X-7, t2=X+20, X=ω×d / 2×Δt. ω represents the rotation speed of the color wheel. d represents the number of pixel rows included in the ROI. Δt represents the readout time for an odd row of pixels and an even row of pixels to be merged into one row. By merging and reading out the W pixels or G pixels of the odd rows and the even rows, the readout time of the ROI can be reduced. By reducing the readout time, the angle of the first occlusion area can be reduced. By reducing the angle of the first occlusion area, the interval between the first visible light image frame and the ROI obtained by the rolling shutter sensor can be reduced. The smaller the interval between the two image frames, the more conducive to subsequent image fusion. Therefore, the present application can simplify the difficulty of image fusion.

[0017] In an optional manner of the first aspect, the motor is used to drive the color wheel to rotate at a constant speed. The constant speed rotation can reduce the requirements on the motor, thereby reducing the cost of the imaging device.

[0018] The second aspect of the present application provides an imaging system. The imaging system includes a fill light and the imaging device described in the first aspect or any one of the first aspects, the fill light is used to provide infrared light for the imaging device. The first incident light received by the imaging device includes reflected light of the infrared light.

[0019] The third aspect of the present application provides an imaging method. The imaging method is applied to an imaging device. The imaging method includes the following steps: the imaging device receives first incident light through a lens. The imaging device rotates a color wheel. The color wheel is located on the light transmission path between the lens and the rolling shutter sensor. The color wheel includes an infrared light cutoff area, a first blocking area, an infrared light penetration area, and a second blocking area in sequence according to the rotation direction. The imaging device intermittently transmits a first visible light and a first infrared light to the rolling shutter sensor through the color wheel. Among them, the first visible light corresponds to the infrared light cutoff area. The first infrared light corresponds to the infrared light penetration area. The imaging device obtains a first visible light image frame according to the first visible light, and obtains a first infrared light image frame according to the first infrared light.

[0020] In an optional manner of the third aspect, the imaging device receives the first incident light in the first time period through the lens. The imaging method further includes the following steps: the imaging device receives the second incident light in the second time period through the lens. The intensity of the visible light in the second time period is greater than the intensity of the visible light in the first time period. The imaging device intermittently transmits the second visible light to the rolling shutter sensor through the color wheel. The imaging device obtains a second visible light image frame based on the second visible light.

[0021] In an optional manner of the third aspect, the imaging method further includes the following steps: the imaging device performs image fusion on the first visible light image frame and the first infrared light image frame to obtain a target image frame.

[0022] In an optional manner of the third aspect, the imaging device performs image fusion on the region of interest ROI of the first infrared light image frame and the first visible light image frame. The angle of the first occlusion area is between t1 and t2 degrees. Wherein, t1=X-7, t2=X+20, X=ω×d×Δt. ω represents the rotation speed of the color wheel. d represents the number of pixel rows included in the ROI. Δt represents the readout time of each row of pixels.

[0023] In an optional manner of the third aspect, the imaging device performs image fusion on the W pixels or G pixels in the region of interest ROI of the first infrared light image frame and the first visible light image frame. The angle of the first occlusion area is between t1 and t2 degrees. Wherein, t1=X-7, t2=X+20, X=ω×d / 2×Δt. ω represents the rotation speed of the color wheel. d represents the number of pixel rows included in the ROI. Δt represents the readout time of an odd-numbered row of pixels and an even-numbered row of pixels being merged into one row. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A first structural schematic diagram of an imaging device provided in an embodiment of the present application;

[0025] Figure 2 An exposure schematic diagram of a rolling shutter sensor provided in an embodiment of the present application;

[0026] Figure 3a A first structural schematic diagram of a color wheel and rolling shutter sensor provided in an embodiment of the present application;

[0027] Figure 3b A second structural schematic diagram of a color wheel and rolling shutter sensor provided in an embodiment of the present application;

[0028] Figure 3c A third structural schematic diagram of the color wheel and rolling shutter sensor provided in an embodiment of the present application;

[0029] Figure 3d A fourth structural schematic diagram of a color wheel and rolling shutter sensor provided in an embodiment of the present application;

[0030] Figure 4 A second structural schematic diagram of an imaging device provided in an embodiment of the present application;

[0031] Figure 5a A schematic diagram of the exposure time of a rolling shutter sensor in a first time period provided in an embodiment of the present application;

[0032] Figure 5b A schematic diagram of the exposure time of the rolling shutter sensor provided in an embodiment of the present application in a second time period;

[0033] Figure 6 A first structural schematic diagram of a color wheel provided in an embodiment of the present application;

[0034] Figure 7a A second structural schematic diagram of a color wheel provided in an embodiment of the present application;

[0035] Figure 7b A third structural schematic diagram of a color wheel provided in an embodiment of the present application;

[0036] Figure 8 A fourth structural schematic diagram of a color wheel provided in an embodiment of the present application;

[0037] Fig. 9 A schematic diagram of the structure of a rolling shutter sensor provided in an embodiment of the present application;

[0038] Fig.10 A fifth structural schematic diagram of a color wheel provided in an embodiment of the present application;

[0039] Fig.11 A schematic diagram of the process of image fusion of RGBW pixels provided in an embodiment of the present application;

[0040] Fig.12 A schematic diagram of the process of image fusion of RGB pixels provided in an embodiment of the present application;

[0041] Fig.13 A second structural schematic diagram of a rolling shutter sensor provided in an embodiment of the present application;

[0042] Fig.14 A sixth structural schematic diagram of a color wheel provided in an embodiment of the present application;

[0043] Fig.15 A schematic diagram of the structure of an imaging system provided in an embodiment of the present application;

[0044] Fig.16 A schematic diagram of the process of the imaging method provided in the embodiment of the present application. DETAILED DESCRIPTION

[0045] The present application provides an imaging device, an imaging system, and an imaging method. By rotating the color wheel, visible light image frames and infrared light image frames can be acquired by time-division multiplexing through an imaging sensor, thereby reducing the manufacturing cost of the imaging device. It should be understood that the "first", "second", "target", etc. used in the present application are only used for the purpose of distinguishing descriptions, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying order. In addition, for the sake of simplicity and clarity, reference numbers and / or letters are repeated in multiple figures of the present application. Repetition does not indicate a strict limiting relationship between various embodiments and / or configurations.

[0046] The imaging device provided in the present application is applied to the field of image acquisition. In the field of image acquisition, a snapshot camera can obtain two imaging frames through two imaging sensors. The two imaging frames include infrared light image frames and visible light image frames. However, the manufacturing cost of the imaging sensor is high, which leads to a high manufacturing cost of the snapshot camera.

[0047] To this end, the present application provides an imaging device, which can also be called a snapshot camera, a video camera, or a camcorder. Figure 1 This is a first structural schematic diagram of an imaging device provided in an embodiment of the present application. Figure 1As shown, the imaging device 100 includes a lens 104, a motor 102, a color wheel 101 and a rolling shutter sensor 103. The lens 104 is used to receive a first incident light and transmit the first incident light to the color wheel 101. The first incident light includes visible light and infrared light. The incident light can also be called light to be imaged, a light beam or an incident light beam, etc. For example, the first incident light can also be called a first light to be imaged. The motor 102 is used to drive the color wheel 101 to rotate. The color wheel 101 is located on the light transmission path between the lens 104 and the rolling shutter sensor 103. The color wheel 101 includes an infrared light cut-off area, a first shielding area, an infrared light penetration area and a second shielding area in sequence according to the rotation direction. The first shielding area and the second shielding area block visible light and infrared light. The infrared light cut-off area blocks infrared light. The infrared light penetration area is a visible light cut-off area or a full-pass area. The visible light cut-off area blocks visible light. The full-pass area allows visible light and infrared light to pass. In the subsequent examples, the infrared light penetration area will be described as the visible light cut-off area. The color wheel 101 is used to receive the first incident light and intermittently transmit the first visible light and the first infrared light to the rolling shutter sensor 103. Among them, when the first incident light is irradiated to the infrared light cut-off area of ​​the color wheel 101, the color wheel 101 is used to block the infrared light in the first incident light and transmit the first visible light to the rolling shutter sensor 103. Therefore, the first visible light corresponds to the infrared light cut-off area. When the first incident light is irradiated to the second blocking area of ​​the color wheel 101, the color wheel 101 is used to block the infrared light and visible light in the first incident light and stop transmitting the light beam to the rolling shutter sensor 103. When the first incident light is irradiated to the visible light cut-off area of ​​the color wheel 101, the color wheel 101 is used to block the visible light in the first incident light and transmit the first infrared light to the rolling shutter sensor 103. Therefore, the first infrared light corresponds to the infrared light penetration area. When the first incident light irradiates the first shielding area of ​​the color wheel 101, the color wheel 101 is used to shield the infrared light and visible light in the first incident light, and stop transmitting the light beam to the rolling shutter sensor 103. Therefore, the color wheel 101 is used to intermittently transmit the first visible light and the first infrared light to the rolling shutter sensor 103. It should be understood that when the infrared light penetration area is a full-pass area, the light beam output by the color wheel 101 through the full-pass area includes both visible light and infrared light. At this time, the first infrared light can also be called the first incident light. The rolling shutter sensor 103 is also called a rolling shutter sensor or a rolling shutter image sensor. The rolling shutter sensor 103 is used to obtain a first visible light image frame according to the first visible light, and obtain a first infrared light image frame according to the first infrared light. Among them, the first visible light image frame is an image frame obtained according to the first visible light. The first infrared light image frame is an image frame obtained according to the first infrared light. In the embodiment of the present application, by rotating the color wheel 101 , the rolling shutter sensor 103 can intermittently receive the first visible light and the first infrared light, thereby acquiring the visible light image frame and the infrared light image frame in a time division multiplexing manner.Therefore, the embodiments of the present application can reduce the manufacturing cost of the imaging device.

[0048] The rolling shutter sensor 103 in the embodiment of the present application can have a global reset function. Figure 2 This is a schematic diagram of the exposure of the rolling shutter sensor provided in the embodiment of the present application. Figure 2 As shown, the horizontal axis represents time and the vertical axis represents the row of pixels. At time H1, the rolling shutter sensor 103 performs a global reset. The global reset action can be triggered by an external signal or by an internal timer in the rolling shutter sensor 103. After the global reset is performed at time H1, all pixel rows on the rolling shutter sensor 103 start to be exposed at the same time. When the time H2 is reached, the exposure of the pixels in the first row ends. The imaging device 100 starts to read out the information carried by the pixels in the first row. The time required for the imaging device 100 to read the information carried by a row of pixels is Δt. After reading the information carried by the pixels in the first row, the imaging device 100 starts to read the information carried by the pixels in the second row. The imaging device 100 repeats the above actions until the information carried by the pixels in the last row is read. It can be seen that the actual exposure time between two adjacent rows of pixels differs by Δt. The actual exposure time of the 0th row and the Nth row differs by K1. Among them, K1=(N+1)*Δt. At this time, the actual exposure time of each row is different. Straight line 201 represents the actual exposure time of each row when the color wheel 101 is not introduced. In the embodiment of the present application, by rotating the color wheel 101, the difference in actual exposure time between two adjacent rows of pixels can be reduced. For example, at time H2, a light shielding device (first shielding area and second shielding area) is used to quickly shield the rolling shutter sensor 103, which is equivalent to the exposure of all pixel rows on the rolling shutter sensor 103 being terminated almost at the same time, thus achieving an exposure effect similar to a global shutter. Straight line 202 represents the actual exposure time of each row after the color wheel 101 is introduced.

[0049] Figure 3a The first structural diagram of the color wheel and rolling shutter sensor provided in the embodiment of the present application. The length of the rolling shutter sensor 103 is b, and the width is a. The position of the rolling shutter sensor 103 relative to the color wheel 101 is as follows: Figure 3a As shown. Figure 1 As can be seen from the description, the motor 102 is used to rotate the color wheel 101. In the following examples, the color wheel 101 is described as rotating in a clockwise direction. Figure 3a As shown, the angle of the infrared light cut-off area is α, the angle of the first shielding area is γ', and the angle of the infrared light penetrating area is α'. The angle of the second shielding area is Υ. The radius of the color wheel 101 is R. When the relative position of the rolling shutter sensor 103 and the color wheel 101 is as shown in FIG. Figure 3aAs shown in , the rolling shutter sensor 103 occupies an area of ​​the infrared light cutoff region at an angle β.

[0050] Figure 3b This is a second structural diagram of the color wheel and rolling shutter sensor provided in the embodiment of the present application. Figure 3b As shown, in Figure 3a Based on the rotation of the color wheel 101 clockwise, the relative position of the rolling shutter sensor 103 and the color wheel 101 will change. Figure 3a The rolling shutter sensors 103 in FIG. 1 just leave the first shielding area and enter the infrared light cutoff area. Therefore, Figure 3a The rolling shutter sensor 103 in is the earliest time at which the rolling shutter sensor 103 can perform exposure. The actual exposure start time of the rolling shutter sensor 103 is equal to or later than the earliest time. Figure 3b The rolling shutter sensor 103 in the example has started exposure or has not started exposure. The actual exposure start time of the rolling shutter sensor 103 is also referred to as the time when the rolling shutter sensor 103 performs a global reset.

[0051] Figure 3c This is a third structural diagram of the color wheel and rolling shutter sensor provided in the embodiment of the present application. Figure 3c As shown, in Figure 3b On the basis of , by continuing to rotate the color wheel 101 clockwise, the relative position of the rolling shutter sensor 103 and the color wheel 101 further changes. At this time, the first row of pixels of the rolling shutter sensor 103 is about to contact the boundary line between the infrared light cut-off area and the second shielding area. This moment is called moment rd1. Figure 3c The exposure ends at the position in. Figure 3c The position of the rolling shutter sensor 103 in is the exposure end time of the rolling shutter sensor 103. The maximum exposure angle of the rolling shutter sensor 103 is α-β. Therefore, the maximum exposure angle is related to α and β. The maximum exposure angle and the maximum exposure time can be converted by the rotation speed of the color wheel 101.

[0052] Figure 3d This is a fourth structural diagram of a color wheel and rolling shutter sensor provided in an embodiment of the present application. Figure 3d As shown, in Figure 3c On the basis of, by continuing to rotate the color wheel 101 clockwise, the relative position of the rolling shutter sensor 103 and the color wheel 101 further changes. At this time, the last row of pixels of the rolling shutter sensor 103 is located at the boundary line rd1. Figure 3c to Figure 3d Rotated by angle β. Figure 3c to Figure 3dDuring the process, some rows of pixels in the rolling shutter sensor 103 are blocked by the second blocking area, and some rows of pixels are not blocked by the second blocking area. This period of time is referred to as transition time t2. After time rd1, the imaging device 100 starts to read the information in the rolling shutter sensor 103, that is, the information of the first visible light image frame. The end time of reading the information is earlier than the time when the rolling shutter sensor 103 enters the infrared light penetration area. The time when the rolling shutter sensor 103 enters the infrared light penetration area is also called the earliest time when the rolling shutter sensor 103 can expose the first infrared light image frame.

[0053] according to Figure 3a From the description, we can see that Figure 3a The rolling shutter sensor 103 in is the earliest moment when the rolling shutter sensor 103 can be exposed. When the position of the rolling shutter sensor 103 is fixed, the color wheel 101 rotates to a fixed angle to correspond to this moment. Therefore, the motor 102 can output a signal when it rotates to this angle, indicating that this time corresponds to the earliest moment when the rolling shutter sensor 103 can be exposed. Alternatively, the imaging device 100 can also include a position sensor. The position sensor is used to determine whether the color wheel 101 rotates to this angle. Similarly, according to Figure 3c As can be seen from the description, the rd1 moment is the moment when the rolling shutter sensor 103 can start and end exposure. When the rolling shutter sensor 103 is fixed in position, the color wheel 101 rotates to a fixed angle corresponding to the moment. The imaging device 100 can determine whether the color wheel 101 is at the angle corresponding to the rd1 moment through the motor 102 or the position sensor. The processing circuit is used to control the rolling shutter sensor 103 to start or end exposure according to the position or angle information output by the position sensor or the motor 102.

[0054] The foregoing Figure 3a to Figure 3d The process of the rolling shutter sensor 103 acquiring the first visible light image frame is described. For the convenience of description, the infrared light cut-off area and the second shielding area are referred to as the visible light period, and the infrared light penetration area and the first shielding area are referred to as the infrared light period. It should be understood that there are similarities between the description of the infrared light period and the description of the visible light period. Therefore, the description of the infrared light period can refer to the description of the visible light period.

[0055] According to the above description, the imaging device 100 can obtain the first visible light imaging frame in the visible light period and the first infrared light imaging frame in the infrared light period. The imaging device 100 can fuse the first visible light imaging frame and the first infrared light imaging frame. Figure 4 This is a second structural schematic diagram of the imaging device provided in the embodiment of the present application. Figure 4 As shown, in Figure 1On the basis of, the imaging device 100 also includes a processing circuit 401. The processing circuit 401 is a central processing unit (CPU) or a network processor (NP), etc. The processing circuit 401 may further include a hardware chip or other general-purpose processor. The above-mentioned hardware chip may be an application specific integrated circuit (ASIC), a system on chip (SOC) chip, a programmable logic device (PLD) or a combination thereof. The processing circuit 401 is used to perform image fusion on the first visible light image frame and the first infrared light image frame to obtain a target image frame.

[0056] According to the foregoing description, the imaging device 100 can obtain the target image frame by image fusion. However, in some scenarios, the imaging device 100 may only need to output a visible light image frame or an infrared light image frame. Therefore, in different time periods, the imaging device 100 can perform different functions. For example, in a first time period, the imaging device 100 is used to obtain a target image frame. In a second time period, the imaging device 100 obtains a second visible light image frame, but does not obtain a second infrared light image frame. This is described below with an example.

[0057] The first time period is night, and the second time period is day. At this time, the intensity of visible light in the second time period is greater than the intensity of visible light in the first time period. In the first time period, the intensity of visible light is low. The imaging device 100 is used according to the above Figure 1, a first visible light image frame and a first infrared light image frame are obtained, and the first visible light image frame and the first infrared light image frame are image-fused to obtain a target image frame. In the second time period, the lens 104 is also used to receive the second incident light in the second time period and transmit the second incident light to the color wheel 101. The color wheel 101 is used to receive the second incident light and intermittently transmit the second visible light to the rolling shutter sensor 103. The rolling shutter sensor 103 is used to obtain a second visible light image frame according to the second visible light. In the first time period, the first visible light image frame corresponds to the infrared light cut-off area, and the first infrared light image frame corresponds to the infrared light penetration area. In the second time period, when the infrared light penetration area is the full-pass area, the second visible light image frame can correspond to the infrared light cut-off area or the infrared light penetration area, and when the infrared light penetration area is the visible light cut-off area, the second visible light image frame corresponds to the infrared light cut-off area. In the subsequent examples, the second visible light image frame corresponding to the infrared light cut-off area is described as an example. In the second time period, the imaging device 100 does not obtain an infrared light image frame. For example, when the second visible light image frame corresponds to the infrared light cut-off region, the rolling shutter sensor 103 does not perform exposure after entering the infrared light penetration region.

[0058] Figure 5a Schematic diagram of the exposure time of the rolling shutter sensor in the first time period provided in the embodiment of the present application. Assuming that the acquisition frequency of the rolling shutter sensor 103 is 50 frames per second, the rotation speed of the color wheel 101 is 25 revolutions per second. The time length for the color wheel 101 to rotate one circle is 40 milliseconds. The first visible light image frame and the first infrared light image frame are generated alternately. The exposure time length of the first visible light image frame is t0. The exposure end time of the first visible light image frame is rd1, that is, the moment when the imaging device 100 starts to read the first row of pixel information of the first visible light image frame. The exposure time length of the first infrared light image frame is t1. The exposure end time of the first visible light image frame is rd0, that is, the moment when the imaging device 100 starts to read the first row of pixel information of the first infrared light image frame. rd0 refers to the moment when the first row of pixels of the rolling shutter sensor 103 is about to contact the boundary line between the infrared light penetration area and the first shielding area. When the infrared light blocking area and the infrared light penetrating area are symmetrically arranged about one center line of the color wheel 101, and the first blocking area and the second blocking area are symmetrically arranged about another center line of the color wheel 101, the interval between rd0 and rd1 is 20 milliseconds. Figure 5aAs shown, after the infrared light exposure time t1, the rolling shutter sensor 103 starts to read the information of the first infrared light image frame and enters the transition time t2. After reading the information of the first infrared light image frame, the rolling shutter sensor 103 enters the visible light exposure time t0. After the visible light exposure time t0, the rolling shutter sensor 103 starts to read the information of the first visible light image frame and enters the transition time t2. After reading the information of the first visible light image frame, the rolling shutter sensor 103 enters the infrared light exposure time t1. Repeating the above steps, the imaging device 100 periodically acquires the first infrared light image frame and the first visible light image frame.

[0059] Figure 5b Schematic diagram of the exposure time of the rolling shutter sensor in the second time period provided in the embodiment of the present application. Assuming that the acquisition frequency of the rolling shutter sensor 103 is 50 frames per second, the rotation speed of the color wheel 101 is 50 revolutions per second. The time length for the color wheel 101 to rotate one circle is 20 milliseconds. The exposure time length of the second visible light image frame is t0. The exposure end time of the second visible light image frame is rd1. The interval between two adjacent rd1 is 20 milliseconds. Figure 5b As shown, after the visible light exposure time t0, the rolling shutter sensor 103 starts to read the information of the second visible light image frame and enters the transition time t2. After reading the information of the first visible light image frame, the rolling shutter sensor 103 enters the visible light exposure time t0 again. Repeating the above steps, the imaging device 100 periodically acquires the second visible light image frame.

[0060] In the above example, the rotation speed of the color wheel 101 in the second time period is twice the rotation speed of the color wheel 101 in the first time period. At this time, the output frame rate of the rolling shutter sensor 103 in the first time period and the second time period is the same. It should be understood that in practical applications, those skilled in the art can adjust the rotation speed of the color wheel 101 in any one or more of the two time periods as required. For example, the rotation speed of the color wheel 101 in the second time period is the same as the rotation speed of the color wheel 101 in the first time period. At this time, the output frame rate of the rolling shutter sensor 103 in the first time period is twice the output frame rate of the rolling shutter sensor 103 in the second time period.

[0061] According to the aforementioned Figure 5bAs can be seen from the description, the second visible light image frame can correspond to the infrared light cut-off area, that is, the second visible light corresponds to the infrared light penetration area. Therefore, the color wheel 101 does not need to output a light beam in the infrared light penetration area. For this reason, when the infrared light penetration area is the visible light cut-off area, the imaging device 100 can also include a double filter. The double filter is located on the light transmission path between the lens 104 and the color wheel 101. In the second time period, the double filter is an infrared light cut-off plate. Therefore, in the second time period, the infrared light in the second imaging light will be blocked by the double filter. The visible light in the second imaging light will be blocked by the visible light cut-off area. The visible light cut-off area of ​​the color wheel 101 does not output infrared light and visible light. At this time, the visible light cut-off area is equivalent to another shielding area. Another shielding area, the first shielding area, and the second shielding area form a larger shielding area. The size of the shielding area will affect the information reading time of the second visible light image frame. The reading time affects the imaging quality of the second visible light image frame. Therefore, by adding a double filter, the imaging quality of the second visible light image frame can be improved. In order to ensure that the imaging device 100 can work normally in the first time period, the dual filter is an all-pass filter in the first time period. Both the infrared light and the visible light in the first imaging light can penetrate the all-pass filter. The dual filter can also include a polarizer in the second time period. The polarizer is used to filter the polarized light with a polarization direction of a certain angle in the second incident light, thereby improving the imaging quality. For example, the polarizer is used to filter the reflected light of the front windshield of the vehicle in the second incident light, so as to better capture the face of the driver in the vehicle.

[0062] The specific design of the color wheel 101 provided in the embodiment of the present application is described below by way of example.

[0063] In the first example, it is assumed that the frame rate of the target image frame output by the imaging device 100 is 25 frames per second. Each time the color wheel 101 rotates one circle, the imaging device 100 outputs one first visible light image frame and one first infrared light image frame. The output frame rate of the rolling shutter sensor 103 is 50 frames per second. One first visible light image frame and one first infrared light image frame are used to obtain one target image frame. Therefore, the rotation speed of the color wheel 101 is 25 circles per second. According to Figure 3a From the structure of the color wheel 101 , it can be seen that the values ​​of the color wheel 101 have the following relationship.

[0064] ω=50π / s Formula 1

[0065] α+Υ+α'+Υ'=2π Formula 2

[0066] α+γ=π Formula 3

[0067] α=ω*t0+β Formula 4

[0068] α'=ω*t1+β Formula 5

[0069] R≈(b / 2) / sin(β / 2))+c Formula 6

[0070] β=ωt Formula 7

[0071] Wherein, t represents the time for the color wheel 101 to rotate one circle. ω represents the rotation speed of the color wheel 101. “*” represents multiplication. R represents the radius of the color wheel 101. Assume that R is equal to 44.5 mm. c represents the length of the rolling shutter sensor 103. Assume that c is equal to 7.7 mm. b represents the width of the rolling shutter sensor 103. Assume that b is equal to 4.4 mm. t0 is the visible light exposure time. t1 is the infrared light exposure time. Assume that t0 and t1 are both equal to 2 milliseconds. According to formulas 1 to 7, it can be obtained that β is equal to 8.28 degrees. t is equal to 0.92 milliseconds. When the infrared light blocking area and the infrared light penetrating area are symmetrically arranged about one center line of the color wheel 101, and the first blocking area and the second blocking area are symmetrically arranged about another center line of the color wheel 101, α and α' are equal to 26.28 degrees. Υ and Υ' are equal to 153.72 degrees. Figure 6 This is a first structural diagram of a color wheel provided in an embodiment of the present application. Figure 6 As shown, in the color wheel 101, the infrared light blocking area and the infrared light penetration area are symmetrically arranged about a center line of the color wheel 101, and the first blocking area and the second blocking area are symmetrically arranged about another center line of the color wheel 101. The angles of the infrared light blocking area and the infrared light penetration area are both 26.28 degrees. The angles of the first blocking area and the second blocking area are both 153.72 degrees.

[0072] In the second example, when the infrared light intensity in the environment is weak, the imaging device 100 can be supplemented with an infrared fill light. The infrared light output by the infrared fill light irradiates the object to be imaged. The light to be imaged reflects the reflected light of the infrared light to the imaging device 100. At this time, the first incident light includes the reflected light of the infrared light. The fill light duration of the infrared fill light may be short. For example, when the infrared fill light is an infrared gas flash light, the flash duration is short, usually only 200 to 300 microseconds. At this time, the angle of the infrared light penetration area can be smaller than the angle of the infrared light blocking area. Assuming that t1 in the first example is equal to 0.5 milliseconds, the requirement of symmetry is eliminated, and other conditions remain unchanged. At this time, according to formulas 1 to 7, β is equal to 8.28 degrees. t is equal to 0.92 milliseconds. α is equal to 26.28 degrees. α' is equal to 12.78 degrees. Υ is equal to 167.22 degrees. γ' is equal to 153.72 degrees. Figure 7a The second structural diagram of the color wheel provided in the embodiment of the present application. The specific parameters of the color wheel 101 are shown in the figure. In practical applications, the angles of γ and γ' can be interchanged. After the angles of γ and γ' are interchanged, γ is equal to 153.72 degrees. γ' is equal to 167.22 degrees. Figure 7bThis is a third structural diagram of the color wheel provided in the embodiment of the present application. Specific parameters of the color wheel 101 are shown in the figure.

[0073] It should be understood that the values ​​of the color wheel 101 are only specific examples provided in the embodiments of the present application. In practical applications, those skilled in the art can adjust the values ​​of the color wheel 101 according to needs. For example, the angle α' of the infrared light penetration area is between 10 and 15 degrees. The angle α of the infrared light cutoff area is between 20 and 30 degrees.

[0074] In the third example, the smaller value of γ and γ' represents the frame interval between the first visible light image frame and the first infrared light image frame. The smaller the frame interval, the stronger the correlation between the two frames of image, and the more conducive to image fusion. Therefore, the embodiment of the present application can reduce the difficulty of image fusion by reducing the smaller value of γ and γ'. Among them, the value of γ is related to the duration of the rolling shutter sensor 103 reading the first visible light image frame. When the duration of the rolling shutter sensor 103 reading the first visible light image frame is longer, the value of γ is larger. When the exposure time of the first row of pixels in the first infrared light image frame is equal to the readout time of the last row of pixels in the first visible light image frame, the minimum value of γ can be obtained. Assume that one image frame of the rolling shutter sensor 103 includes N rows of pixels. The data reading time of each row is Δt, that is, the readout time required for all rows of pixels in one image frame is H1. H1=N*Δt. The rolling shutter sensor 103 starts to read the information of the first row of pixels in the first visible light image frame at time rd1. After the time H1, the curtain shutter sensor 103 reads the information of the last row of pixels in the first visible light image frame and begins to enter the infrared light exposure time t1. Therefore, γ = ω*N*P1. Assuming that N is equal to 2204 and Δt is equal to 7.33 microseconds. Therefore, the corresponding size of the color wheel 101 can be obtained by calculation.

[0075] H1=N*Δt=2204*7.33us=16155.32us≈16.16ms

[0076] γ=ω*N*Δt=50*180°*16.162 / 1000=145.44°

[0077] γ'=153.72°+(180°-γ-α')=153.72°+(180°-ω*N*Δt-29°)

[0078] According to the calculation in the second example, α' is equal to 12.78 degrees. Therefore, the value of γ' can be obtained by the above formula.

[0079] γ'=153.72°+(180°-145.44°-12.78°)=175.5°

[0080] Thus, in the third example, the color wheel 101 has β equal to 8.28 degrees. t equal to 0.92 milliseconds. α equal to 26.28 degrees. α' equal to 12.78 degrees. γ equal to 145.44 degrees. γ' equal to 175.5 degrees. Figure 8 This is a fourth structural diagram of a color wheel provided in an embodiment of the present application. The specific parameters of the color wheel 101 are shown in the figure. Figure 7b The smaller value of γ and γ' is 153.72. Figure 8 The smaller value of γ and γ' is 145.44 degrees. Therefore, the embodiment of the present application can reduce the difficulty of image fusion.

[0081] In a fourth example, the processing circuit can use the region of interest (ROI) of the first infrared image frame and the first visible light image frame for image fusion. The number of pixels of the ROI is less than the number of pixels of the first visible light image frame. For example, Fig. 9 This is a schematic diagram of the structure of a rolling shutter sensor provided in an embodiment of the present application. Fig. 9 As shown, the first visible light image frame includes N rows of pixels. The ROI of the first infrared light image frame is located at the hth to h+dth rows of pixels in the N rows of pixels. The ROI includes d rows of pixels. d is less than N. In the field of traffic capture, the ROI is generally used to cover the position of the driver in the vehicle. Therefore, it can be assumed that d=720. Among them, h is related to the window position. The window position can be calculated and estimated by detecting the vehicle position and speed information. The readout time of the ROI is H2 microseconds.

[0082] H2=d*P1=720*7.33u=5277.6, where 5277.6 microseconds is approximately equal to 5.28 milliseconds.

[0083] γ'=ω*d*Δt=50*180°*5.28 / 1000=47.52°

[0084] As in the second example above, assume that t1 is equal to 0.5 milliseconds and t0 is equal to 2 milliseconds. α is equal to 26.28 degrees. α' is equal to 12.78 degrees. Therefore, the value of γ can be obtained. γ=360°-12.78°-47.52°-26.28°=273.42°. Fig.10 This is the fifth structural diagram of the color wheel provided in the embodiment of the present application. The specific parameters of the color wheel 101 are shown in the figure. Figure 8 The smaller value of γ and Υ' is 145.44 degrees. Fig.10 The smaller value of γ and γ' is 47.52 degrees. Therefore, the embodiment of the present application can reduce the difficulty of image fusion.

[0085] It should be understood that the values ​​of the above-mentioned color wheel 101 are only specific examples provided in the embodiments of the present application. In practical applications, those skilled in the art can adjust the values ​​of the color wheel 101 as needed. For example. The angle of the first occlusion area Y' is between t1 and t2 degrees. Wherein, t1=X-7, t2=X+20, X=ω×d×Δt, ω represents the rotation speed of the color wheel, d represents the number of pixel rows included in the ROI, and Δt represents the readout time of each row of pixels. In the fourth example, t1 is equal to 40.52 degrees. t2 is equal to 67.52 degrees.

[0086] In the fifth example, the processing circuit is used to perform image fusion on the W pixel or the G pixel in the first infrared light image frame and the first visible light image frame, thereby reducing the value of Y'.

[0087] Fig.11 The following is a schematic diagram of the process of image fusion of RGBW pixels provided in the embodiment of the present application. Fig.11 As shown, in the visible light image frame period, the imaging device 100 reads the RGB pixels (red, green and blue pixels) and W pixels (white pixels) of the first visible light image frame in the rolling shutter sensor 103. In the infrared light image frame period, the imaging device 100 reads the W pixels of the first infrared light image frame, and does not read the RGB pixels of the first infrared light image frame. In step 1103, the imaging device 100 performs ROI area configuration through the W pixels of the first infrared light image frame and the W pixels of the first visible light image frame to determine the target area corresponding to the ROI of the first infrared light image frame in the first visible light image frame. In step 1104, the imaging device 100 is used to perform ROI area fusion on the target area extracted from the RGB pixels of the first visible light image frame and the ROI of the first infrared light image frame, and output the first image. In step 1101, the imaging device 100 is used to perform black and white and color fusion processing on the RGB pixels and W pixels of the first visible light image frame, and output the second image. In step 1105, the imaging device 100 is used to perform ROI large image fusion on the first image and the second image to obtain a target image frame. In step 1102, the imaging device 100 is used to perform image signal processing (ISP) on the first image to obtain a video stream. It should be understood that in actual applications, in the infrared light image frame period, the imaging device 100 can read the G pixel of the first infrared light image frame, and does not read the R pixel, B pixel and W pixel of the first infrared light image frame. The imaging device 100 performs image fusion through the G pixel of the first infrared light image frame and the first visible light image frame.

[0088] Fig.12 The schematic diagram of the process of image fusion of RGB pixels provided in the embodiment of the present application. Fig.12As shown, in the visible light image frame period, the imaging device 100 reads the RGB pixels of the first visible light image frame in the rolling shutter sensor 103. In the infrared light image frame period, the imaging device 100 reads the G pixels of the first infrared light image frame, and does not read the R pixels and B pixels of the first infrared light image frame. In step 1202, the imaging device 100 performs ROI area configuration through the G pixels of the first infrared light image frame and the RGB pixels of the first visible light image frame, so as to determine the target area corresponding to the ROI of the first infrared light image frame in the first visible light image frame. In step 1203, the imaging device 100 is used to perform ROI area fusion on the target area extracted from the RGB pixels of the first visible light image frame and the ROI of the first infrared light image frame, and output the first image. In step 1204, the imaging device 100 is used to perform ROI large image fusion on the first image and the first visible light image frame to obtain the target image frame. In step 1201, the imaging device 100 is used to perform ISP processing on the first visible light image frame to obtain a video stream.

[0089] Fig.13 This is a second structural diagram of a rolling shutter sensor provided in an embodiment of the present application. Fig.13 As shown, when the rolling shutter sensor 103 includes RGBW pixels, the gray squares represent W pixels or G pixels. Wherein, when the gray squares represent W pixels, the white squares represent R pixels, B pixels and G pixels. When the gray squares represent G pixels, the white squares represent R pixels, W pixels and B pixels. When the rolling shutter sensor 103 includes RGB pixels, the gray squares represent G pixels, and the white squares represent R pixels and B pixels. By merging and reading out the W pixels or G pixels of the even rows and the odd rows, the readout time length of the first infrared light image frame can be reduced without increasing the ADC circuit of the rolling shutter sensor 103. By reducing the readout time length, the value of Y' can be reduced.

[0090] Based on the fourth example above, the readout time of ROI is H3 microseconds. H3 = d / 2*Δt = 360*7.33us = 2638.8, where 2638.8 microseconds is approximately equal to 2.63 milliseconds. γ' = ω*d*Δt = 50*180°*2.63 / 1000 = 23.67°. As in the second example above, assume that t1 is equal to 0.5 milliseconds and t0 is equal to 2 milliseconds. α is equal to 26.28 degrees. α' is equal to 12.78 degrees. Therefore, the value of γ can be obtained. γ = 360°-12.78°-23.67°-26.28° = 297.27°. Fig.14 This is the sixth structural diagram of the color wheel provided in the embodiment of the present application. The specific parameters of the color wheel 101 are shown in the figure. Fig.10 The smaller value of γ and γ' is 47.52°. Fig.14The smaller value of γ and γ' is 23.67 degrees. Therefore, the embodiment of the present application can further reduce the difficulty of image fusion.

[0091] It should be understood that the values ​​of the above-mentioned color wheel 101 are only specific examples provided in the embodiments of the present application. In practical applications, those skilled in the art can adjust the values ​​of the color wheel 101 as needed. For example. The angle γ' of the first occlusion area is between t1 and t2 degrees, where t1=X-7, t2=X+20, and X=ω×d / 2×Δt. ω represents the rotation speed of the color wheel, d represents the number of pixel rows included in the ROI, and Δt represents the readout time for an odd row of pixels and an even row of pixels to be merged into one row. In the fifth example, t1 is equal to 16.67 degrees. t2 is equal to 43.67 degrees.

[0092] It should be understood that Fig.13 In the example, the processing circuit is used to perform image fusion on the W pixel or G pixel of the ROI in the first infrared light image frame and the first visible light image frame. In practical applications, the processing circuit can also be used to perform image fusion on the W pixel or G pixel in the first infrared light image frame and the first visible light image frame. At this time, the first infrared light image frame also includes N rows of pixels. The readout time of the W pixel or G pixel in the first infrared light image frame is H4 microseconds. H4=N / 2*Δt.

[0093] It should be understood that in the aforementioned example, the shape of the color wheel 101 is circular. In practical applications, the shape of the color wheel 101 can also be other shapes, such as a positive direction or a triangle. Similarly, in the aforementioned example, the four areas of the color wheel 101 are all fan-shaped. The four areas include an infrared light cut-off area, a first shielding area, an infrared light penetration area, and a second shielding area. In practical applications, the shape of each of the four areas can also be other shapes, such as a triangle or a rectangle.

[0094] It should be understood that in the aforementioned example, the color wheel 101 rotates at a constant speed. In practical applications, the rotation mode of the color wheel 101 may not be constant speed. For example, when the rolling shutter sensor 103 is located in the infrared light cut-off area and the infrared light penetration area, the rotation speed of the color wheel 101 is relatively low; when the rolling shutter sensor 103 is located in the first shielding area and the second shielding area, the rotation speed of the color wheel 101 is relatively high.

[0095] It should be understood that in the aforementioned example, the color wheel 101 includes four areas. In practical applications, the color wheel 101 can include more areas. For example, the color wheel 101 includes 6 areas, 8 areas, or 16 areas, etc. When the color wheel 101 includes 6 areas, the 6 areas can include 3 shielding areas, 2 infrared light cut-off areas, and 1 infrared light penetration area. At this time, the imaging device 100 can output 2 first visible light imaging frames and 1 first infrared light imaging frame during one rotation of the color wheel 101. When the color wheel 101 includes 8 areas, the 8 areas can include 4 shielding areas, 2 infrared light cut-off areas, and 2 infrared light penetration areas. At this time, the imaging device 100 can output 2 first visible light imaging frames and 2 first infrared light imaging frames during one rotation of the color wheel 101.

[0096] In other embodiments, the imaging device 100 may further include a memory. The memory may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), or a flash memory. The volatile memory may be a random access memory (RAM). The memory is used to store the first visible light image frame and / or the first infrared light imaging frame.

[0097] The imaging device provided by the present application is described above. The imaging system and imaging method provided by the present application are described below. Fig.15 This is a schematic diagram of the structure of the imaging system provided in the embodiment of the present application. Fig.15 As shown, the imaging system 1500 includes a fill light 1501 and an imaging device 100. The fill light 1501 includes a white light emitting diode (LED) strobe light and / or an infrared white light integrated gas strobe light. The light emitted by the white light LED strobe light is visible light. The infrared white light integrated gas strobe light is used to provide infrared light, and can also be used to provide visible light. The processing circuit in the imaging device 100 can be used to control the fill light 1501 to output infrared light and / or visible light. The imaging device 100 is used to receive a first imaging light, and obtain a first visible light image frame and a first infrared light image frame according to the first imaging light. The first incident light includes reflected light of infrared light and / or reflected light of visible light. For the description of the imaging device 100, reference can be made to the aforementioned Figure 1 to Figure 14 A description of any graph. Fig.16 Schematic diagram of the process of the imaging method provided in the embodiment of the present application. Fig.16 As shown, the imaging method includes the following steps.

[0098] In step 1601, an imaging device receives first incident light through a lens.

[0099] The first incident light includes infrared light and visible light. The infrared light and / or visible light is fill light provided by ambient light or fill light. The lens can include multiple optical elements. The multiple optical elements are used to focus, change the transmission direction, collimate or eliminate aberration of the first incident light beam. The imaging device can also be called a snapshot camera, a video camera or a camcorder.

[0100] In step 1602, the imaging device rotates a color wheel. The color wheel is located on the light transmission path between the lens and the rolling shutter sensor. The color wheel includes an infrared light cutoff area, a first shielding area, an infrared light penetration area, and a second shielding area in sequence according to the rotation direction.

[0101] The first shielding area and the second shielding area shield visible light and infrared light. The infrared light cut-off area shields infrared light. The infrared light penetration area is a visible light cut-off area or a full-pass area. The visible light cut-off area shields visible light. The full-pass area allows visible light and infrared light to pass through. The imaging device can include a motor. The imaging device drives the color wheel to rotate through the motor. The light transmission path between the lens and the rolling shutter sensor can also include other optical elements, such as a reflector. The reflector is used to change the transmission direction of the first incident light or the first visible light.

[0102] In step 1603, the imaging device intermittently transmits the first visible light and the first infrared light to the rolling shutter sensor through the color wheel, wherein the first visible light corresponds to the infrared light cut-off region and the first infrared light corresponds to the infrared light penetration region.

[0103] When the first incident light is irradiated to the infrared light cut-off area, the infrared light in the first incident light is blocked, and the color wheel outputs the first visible light. Therefore, the first visible light corresponds to the infrared light cut-off area. When the first incident light is irradiated to the second blocking area, the infrared light and visible light in the first incident light are blocked. When the first incident light is irradiated to the visible light cut-off area, the visible light in the first incident light is blocked, and the color wheel outputs the first infrared light. Therefore, the first visible light corresponds to the infrared light cut-off area. It should be understood that when the infrared light penetration area is a full-pass area, the first infrared light output by the color wheel may also include visible light. When the first incident light is irradiated to the first blocking area, the infrared light and visible light in the first incident light are blocked. Therefore, the color wheel intermittently outputs the first visible light and the first infrared light.

[0104] In step 1604, the imaging device obtains a first visible light image frame according to the first visible light, and obtains a first infrared light image frame according to the first infrared light. The imaging device includes a rolling shutter sensor. After the shutter is opened, the pixels on the rolling shutter sensor are exposed row by row, and the exposure starting points of pixels in different rows are the same. The rolling shutter sensor intermittently receives the first visible light and the first infrared light, and obtains the first visible light image frame and the first infrared light image frame by time division multiplexing.

[0105] It should be understood that Fig.16 The imaging method described in the foregoing Figure 1 to Figure 14 Therefore, for the description of the imaging method, reference can be made to the aforementioned Figure 1 to Figure 14 Any figure describes the imaging device. For example, the imaging device receives the first incident light in the first time period through the lens. The imaging method also includes the following steps: the imaging device receives the second incident light in the second time period through the lens. The intensity of the visible light in the second time period is greater than the intensity of the visible light in the first time period. The imaging device intermittently transmits the second visible light to the rolling shutter sensor through the color wheel. The imaging device obtains a second visible light image frame based on the second visible light. For another example, the imaging device performs image fusion on the first visible light image frame and the first infrared light image frame to obtain a target image frame. For another example, the imaging device performs image fusion on the region of interest ROI of the first infrared light image frame and the first visible light image frame. The angle of the first occlusion area is between t1 and t2 degrees. Among them, t1=X-7, t2=X+20, X=ω×d×Δt. ω represents the rotation speed of the color wheel. N represents the number of pixel rows included in the ROI. P1 represents the readout time of each row of pixels.

[0106] According to the aforementioned Figure 1 to Figure 14 As can be seen from the description, the imaging device includes multiple components. For example, the imaging device includes a lens, a motor, a color wheel, a rolling shutter sensor, and a processing circuit. Those skilled in the art can conceive of implementing different steps in the above-mentioned imaging method by one or more components in the imaging device. For example, for step 1602, the processing circuit is used to generate a drive signal; the motor is used to rotate according to the drive signal and drive the color wheel to rotate. For example, for step 1604, the rolling shutter sensor is used to intermittently receive the first visible light and the first infrared light, and obtain the first visible light image frame and the first infrared light image frame by time division multiplexing.

[0107] The above are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be covered by the protection scope of the present application.

Claims

1. An imaging device, characterized in that: Includes lens, motor, color wheel and rolling shutter sensor, where: The lens is used to receive first incident light and transmit the first incident light to the color wheel, wherein the first incident light includes visible light and infrared light; The motor is used to drive the color wheel to rotate; The color wheel is located on the light transmission path between the lens and the rolling shutter sensor, and the color wheel includes an infrared light cut-off area, a first blocking area, an infrared light penetration area, and a second blocking area in sequence according to the rotation direction. The color wheel is used to receive the first incident light and intermittently transmit the first visible light and the first infrared light to the rolling shutter sensor, wherein the first visible light corresponds to the infrared light cut-off area, and the first infrared light corresponds to the infrared light penetration area; The rolling shutter sensor is used to obtain a first visible light image frame according to the first visible light, and to obtain a first infrared light image frame according to the first infrared light.

2. The imaging device according to claim 1, characterized in that The lens is used to receive the first incident light, including: the lens is used to receive the first incident light in the first time period; The lens is further used to receive second incident light in a second time period, and transmit the second incident light to the color wheel, wherein the intensity of the visible light in the second time period is greater than the intensity of the visible light in the first time period; The color wheel is used to receive the second incident light and intermittently transmit the second visible light to the rolling shutter sensor; The rolling shutter sensor is used to obtain a second visible light image frame according to the second visible light.

3. The imaging device according to claim 2, characterized in that The second visible light corresponds to the infrared light cut-off region.

4. The imaging device according to claim 2 or 3, characterized in that: The imaging device further comprises a dual filter, wherein the dual filter is located on the light transmission path between the lens and the color wheel; The dual filter is an all-pass filter in the first time period, and is an infrared light cutoff filter in the second time period.

5. The imaging device according to any one of claims 2 to 4, characterized in that: The rotation speed of the color wheel in the second time period is twice the rotation speed of the color wheel in the first time period.

6. The imaging device according to any one of claims 1 to 5, characterized in that: The exposure time length of the first visible light image frame is greater than the exposure time length of the first infrared light image frame.

7. The imaging device according to any one of claims 1 to 6, characterized in that: The angle of the infrared light cut-off area is greater than the angle of the infrared light penetration area.

8. The imaging device according to claim 7, characterized in that: The angle of the infrared light penetration area is between 10 and 15 degrees, and the angle of the infrared light cut-off area is between 20 and 30 degrees.

9. The imaging device according to any one of claims 1 to 8, characterized in that: The angle of the first shielding area is smaller than the angle of the second shielding area.

10. The imaging device according to any one of claims 1 to 9, characterized in that: The infrared light penetration region is a visible light cut-off region.

11. The imaging device according to any one of claims 1 to 10, characterized in that: The imaging device further comprises a processing circuit, wherein: The processing circuit is used for performing image fusion on the first visible light image frame and the first infrared light image frame to obtain a target image frame.

12. The imaging device according to claim 11, characterized in that The processing circuit is used to perform image fusion on the first visible light image frame and the first infrared light image frame, including: the processing circuit is used to perform image fusion on the region of interest ROI of the first infrared light image frame and the first visible light image frame; The angle of the first shading area is between t1 and t2 degrees, wherein t1=X-7, t2=X+20, X=ω×d×Δt, ω represents the rotation speed of the color wheel, d represents the number of pixel rows included in the ROI, and Δt represents the readout time of each row of pixels.

13. The imaging device according to claim 11, characterized in that The processing circuit is used to perform image fusion on the first visible light image frame and the first infrared light image frame, including: the processing circuit is used to perform image fusion on the W pixel or the G pixel in the ROI of the first infrared light image frame and the first visible light image frame; The angle of the first occluding area is between t1 and t2 degrees, wherein t1=X-7, t2=X+20, X=ω×d / 2×Δt, ω represents the rotation speed of the color wheel, d represents the number of pixel rows included in the ROI, and Δt represents the readout time for an odd row of pixels and an even row of pixels to be combined into one row.

14. An imaging system, characterized in that: The invention comprises a fill light and an imaging device as described in any one of claims 1 to 13, wherein the fill light is used to provide infrared light for the imaging device.

15. An imaging method, applied to an imaging device, characterized in that: include: receiving first incident light through the lens; A rotating color wheel, wherein the color wheel is located on a light transmission path between the lens and the rolling shutter sensor, and the color wheel includes an infrared light cutoff area, a first shielding area, an infrared light penetration area, and a second shielding area in sequence according to a rotation direction; intermittently transmitting a first visible light and a first infrared light to the rolling shutter sensor through the color wheel, wherein the first visible light corresponds to the infrared light cut-off region, and the first infrared light corresponds to the infrared light penetration region; A first visible light image frame is obtained according to the first visible light, and a first infrared light image frame is obtained according to the first infrared light.

16. The imaging method according to claim 15, characterized in that: The receiving the first incident light through the lens comprises: receiving the first incident light through the lens in the first time period; The method further comprises: receiving a second incident light through the lens in a second time period, wherein the intensity of the visible light in the second time period is greater than the intensity of the visible light in the first time period; intermittently transmitting a second visible light to the rolling shutter sensor through the color wheel; A second visible light image frame is obtained according to the second visible light.

17. The imaging method according to claim 15 or 16, characterized in that: The method further comprises: The first visible light image frame and the first infrared light image frame are fused to obtain a target image frame.

18. The imaging method according to claim 17, characterized in that: The image fusion of the first visible light image frame and the first infrared light image frame comprises: image fusion of a region of interest ROI of the first infrared light image frame and the first visible light image frame; The angle of the first shading area is between t1 and t2 degrees, wherein t1=X-7, t2=X+20, X=ω×d×Δt, ω represents the rotation speed of the color wheel, d represents the number of pixel rows included in the ROI, and Δt represents the readout time of each row of pixels.

19. The imaging method according to claim 17, characterized in that: The image fusion of the first visible light image frame and the first infrared light image frame comprises: performing image fusion of W pixels or G pixels in the ROI of the first infrared light image frame and the first visible light image frame; The angle of the first occluding area is between t1 and t2 degrees, wherein t1=X-7, t2=X+20, X=ω×d / 2×Δt, ω represents the rotation speed of the color wheel, d represents the number of pixel rows included in the ROI, and Δt represents the readout time for an odd row of pixels and an even row of pixels to be combined into one row.

Images