Imaging device and control method for the imaging device

Abstract

This technology suppresses count loss in photon-counting image sensors, improving image brightness and color reproduction. [Solution] The imaging device includes an image sensor having a light receiving unit that outputs a pulse signal in response to the incidence of photons and a counting unit that counts the pulse signal; an acquisition means for acquiring the blinking frequency and emission period of a light source; and a control means for controlling the exposure period of the image in the image sensor and the counting frequency used for counting by the counting unit based on the acquired blinking frequency and emission period of the light source.

Description

Technical Field

[0001] The present invention relates to an imaging device and a method for controlling the imaging device.

Background Art

[0002] There has been proposed a photon counting type imaging device using a photodiode array in which avalanche photodiodes (APDs) to which a reverse bias voltage greater than the breakdown voltage is applied are arranged in an array to detect the number of photons incident in a certain range. Also, when shooting a moving image under a blinking light source such as a fluorescent lamp, periodic light and dark called flicker may appear in the obtained image. In order to eliminate flicker, a detection generally called flicker detection is executed to detect the blinking period of the blinking light source, and a technique for suppressing the luminance change of the image by adjusting the accumulation period (exposure period) of the imaging device in accordance with this is known.

[0003] [[ID=1:16]]In a photon counting type imaging device, the current flowing due to the avalanche multiplication operation generated when photons (photons) are incident is converted into a change in a voltage signal, the change in the voltage signal is output as a pulse, and a pixel signal is generated by counting the number of pulses. In Patent Document 1, there has been proposed a technique for detecting the presence or absence and characteristics of a flickering light source in a shooting range based on the output frequency of a pulse signal corresponding to the incidence of photons in an imaging under a blinking light source in a photon counting type imaging device. Also, in Patent Document 2, by measuring the charge generated in the photoelectric conversion unit outside the exposure period of the photoelectric conversion unit, the frequency and duty ratio of the light source are specified, the exposure period of the photoelectric conversion unit is adjusted, and a technique capable of reducing the variation in brightness when shooting a subject that repeats light and dark has been proposed.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

[0005] In a photon counting type imaging device, the number of photons that can be counted per unit time is determined by a so-called dead time that is rate-limited by the device structure of the APD. When a plurality of photons enter in a period shorter than the dead time, the number of counted pulses becomes one. Therefore, in the case of a blinking light source, photons enter concentrated in a short time, resulting in many counting losses, and there is a problem that the actual brightness and color are not reproduced in the image. [Means for Solving the Problems]

[0006] The imaging device according to the present invention includes an imaging element having a light receiving unit that outputs a pulse signal in response to the incidence of photons, and a counting unit that counts the pulse signal, an acquisition unit that acquires the blinking frequency and the light emission period of a light source, and control means for controlling an exposure period of an image in the imaging element and a counting frequency for counting in the counting unit based on the acquired blinking frequency and light emission period of the light source. [Effects of the Invention]

[0007] According to the present invention, it is possible to suppress counting loss by controlling the exposure time and the counting frequency in a photon counting type imaging device according to the light emission period of a blinking light source, and to improve the reproducibility of the brightness and color of an image. [Brief Description of the Drawings]

[0008] [Figure 1] It is a diagram showing a configuration example of an imaging device. [Figure 2]This flowchart shows an example of a process performed in an imaging device. [Figure 3] This figure illustrates the emission and counting of a single light source in the first embodiment. [Figure 4] This figure illustrates the operation of photon counting in the first embodiment. [Figure 5] This figure shows an example of light output when the light output of a light source changes over time. [Figure 6] This figure illustrates an example of using a rolling shutter type image sensor in the first embodiment. [Figure 7] This figure illustrates the operation of photon counting in the second embodiment. [Figure 8] This figure illustrates the operation of photon counting in the second embodiment. [Figure 9] This figure illustrates the operation of photon counting in the third embodiment. [Figure 10] This figure illustrates the operation of photon counting in the third embodiment. [Figure 11] This figure illustrates the operation of photon counting in the third embodiment. [Modes for carrying out the invention]

[0009] Embodiments of the present invention will be described below with reference to the drawings. Note that the following embodiments do not limit the invention as defined in the claims. While multiple features are described in the embodiments, not all of these features are necessarily essential to the invention, and the features may be combined in any way. Furthermore, in the accompanying drawings, the same or similar configurations are given the same reference numerals, and redundant descriptions are omitted.

[0010] <First Embodiment> Figure 1 is a block diagram showing an example of the functional configuration of the imaging device 100 in the first embodiment. The imaging device 100 includes a photographic lens 101, an image sensor 102, an image processing unit 103, a memory unit 104, a recording unit 105, a control unit 106, an operation unit 107, a display unit 108, and a recording medium 109.

[0011] The photographic lens 101 is a photographic optical system that forms an optical image of the subject on the imaging surface of the image sensor 102, and has multiple lenses, including a focusing lens. The photographic lens 101 also has an aperture, an ND filter, etc. The photographic lens 101 may be detachable from the imaging device 100, or it may be fixed and non-detachable from the imaging device 100.

[0012] The image sensor 102 is a photon counting type image sensor having a light receiving unit that outputs a pulse signal in response to the incidence of photons and a counting unit that counts the pulse signals. The image sensor 102 has pixels, each having an avalanche photodiode (APD) as a photoelectric conversion element, arranged in two dimensions, and a counter is provided for each pixel to determine and count the presence or absence of photons. Furthermore, as described in Japanese Patent Application Publication No. 2023-61883, the image sensor 102 can recharge the APD to a bias voltage that enables avalanche multiplication in Geiger mode by a clock signal set by the control unit 106.

[0013] The image processing unit 103 applies signal processing to the image data read from the image sensor 102 to generate image data for display and image data for recording. The signal processing applied by the image processing unit 103 includes signal rearrangement, defective pixel correction, noise reduction, color conversion, white balance correction, gamma correction, resolution conversion, data compression, and sharpness adjustment during the image generation process. These are examples of signal processing that the image processing unit 103 can perform and do not limit the signal processing that the image processing unit 103 performs. The image processing unit 103 also outputs information obtained by applying signal processing to the image data to the control unit 106. The image processing unit 103 has a light source characteristic acquisition unit 110. The light source characteristic acquisition unit 110 estimates periodic information related to the flicker phenomenon, such as the blinking frequency, lighting time width, and lighting interval of the light source, and acquires it as characteristic information of the light source. The image processing unit 103 is an example of image processing means, and the light source characteristic acquisition unit 110 is an example of acquisition means. Furthermore, some or all of the functions of the image processing unit 103 may be implemented using dedicated hardware circuits, such as an ASIC (Application-Specific Integrated Circuit) designed to perform a specific function. Alternatively, some or all of the functions of the image processing unit 103 may be implemented by a processor, such as a DSP (Digital Signal Processor), executing software.

[0014] The memory unit 104 is configured using, for example, RAM (Random Access Memory) or ROM (Read Only Memory). The memory unit 104 is used as an image data buffer, as a work area for the image processing unit 103 and the control unit 106, or as video memory for the display unit 108. The memory unit 104 also has a non-volatile storage area. This non-volatile storage area is used to store, for example, programs and settings executed by the control unit 106, settings for the imaging device 100, and UI (User Interface) display data.

[0015] Based on control by the control unit 106, the recording unit 105 performs tasks such as writing and reading data files to and from the recording medium 109, which is a semiconductor memory card.

[0016] The control unit 106 is, for example, a processor and controls the entire imaging device 100. The control unit 106 is an example of a control means. For example, the control unit 106 loads a program stored in the ROM of the memory unit 104 into the system memory of the memory unit 104 and executes it, controlling the operation of each part to realize the functions of the imaging device 100. For example, when an operation of the operation unit 107 is detected, the control unit 106 executes an operation corresponding to the detected operation.

[0017] The operation unit 107 consists of input devices such as switches, buttons, touchpads, and dials, which are used by the user to give instructions to the imaging device 100. Each of the input devices included in the operation unit 107 is assigned a function either fixedly or dynamically. As a result, the input devices function as shutter buttons, video recording / stop buttons, menu buttons, directional keys, select buttons, operation mode switching dials, etc. If the display unit 108 is a touch display, software-reproduced input devices realized through a combination of a touch panel and a GUI (Graphical User Interface) are also included as input devices in the operation unit 107.

[0018] The display unit 108 is, for example, a flat panel display and displays an image based on a display signal supplied from the control unit 106. Based on the display signal supplied from the control unit 106, the display unit 108 displays, for example, a live view image or a menu image. The display unit 108 may also be a touch display.

[0019] Next, the processing performed in the imaging device 100 in this embodiment will be described. Figure 2 is a flowchart showing an example of the processing performed in the imaging device 100. The control unit 106 of the imaging device 100 controls the operation of each part in the device to execute the processing shown in the flowchart in Figure 2. In step S201, the control unit 106 acquires characteristic information of the light source using the light source characteristic acquisition unit 110, etc. In step S202, the control unit 106 determines the accumulation timing and counting frequency of the image sensor 102 based on the characteristic information of the light source acquired in step S201. In step S203, the control unit 106 sets the accumulation timing and counting frequency determined in step S202 to the image sensor 102, and the image sensor 102 performs imaging with that accumulation timing and counting frequency. The processing in each step will be described in detail below.

[0020] In step S201, the light source characteristic acquisition unit 110 estimates the periodic information of the light source, and the control unit 106 acquires the estimated periodic information of the light source as characteristic information of the light source. For example, the light source characteristic acquisition unit 110 stores the average brightness value of the subject image for each of the multiple imaging frames, and estimates the periodic information (flashing frequency, lighting duration, etc.) in the time change of brightness by using the Discrete Fourier Transform (DFT) on the stored data. Note that the estimation of periodic information is not limited to the example described above, and other methods such as the flicker detection method described in International Publication No. 2016 / 132615 may be used. Alternatively, the user of the imaging device 100 may specify the number of light sources and the frequency information of each light source using the operation unit 107, the control unit 106 may notify the light source characteristic acquisition unit 110 of the specified frequency information, and the light source characteristic acquisition unit 110 may identify the periodic information of the light source based on that frequency information.

[0021] In step S202, based on the characteristic information of the light source acquired in step S201, the control unit 106 determines the accumulation timing (exposure period) and the counting frequency in the image sensor 102 according to each light emission period of the light source within the set shutter speed. FIG. 3 is a diagram for explaining one light emission of the light source and its counting in the first embodiment. "Count" in FIG. 3 indicates that in each pixel of the image sensor 102, the photons incident are counted by the counter possessed by the pixel, and it is possible to count one photon with the resolution (1 / f cnt ). The count period t cnt for counting the incident photons is determined based on the light emission period t cnt specified in step S201 and the preset allowable range tε so that |t e - t e | < tε is satisfied. That is, the count period t cnt is determined so that the difference between the light emission period t e and the count period t cnt does not exceed the allowable range tε. The counting frequency f cnt is determined so that the image sensor 102 can count up to a predetermined number of times K within the count period t cnt . That is, the counting frequency f cnt is determined so that the image sensor 102 can count a predetermined number of times within the exposure period. As an example, the counting frequency f cnt is determined by f cnt = K / t cnt . Note that the determination of the counting frequency is not limited to the above-described example, and other methods may be used for determination.

[0022] ​​The imaging in step S203 will be described. Figure 4 is a diagram illustrating the operation of photon counting in one frame in the first embodiment. The image sensor 102 performs an accumulation operation (counting operation, counting operation) in accordance with each emission period 401, 402, and 403 of the light source, based on the accumulation timing (exposure period) determined in step S202 set by the control unit 106. The image sensor 102 also performs counting in each emission period 401, 402, and 403 of the light source at the counting frequency determined in step S202 set by the control unit 106. The image sensor 102 performs counting in accordance with the emission period (counting period t cnt During periods other than the exposure period, the counting operation is stopped. In other words, the image sensor 102 stops the counting operation during periods other than the exposure period. The image sensor 102 uses the sum of the count values ​​K1, K2, and K3 obtained by the counting operations 421, 422, and 423 corresponding to each light emission period 401, 402, and 403 of the light source as the result of the count value (count value) for one frame.

[0023] Note that in Figures 3 and 4, the light output of the light source is assumed to be constant during the emission period, but it is possible to count similarly even when the light output of the light source changes over time. Figure 5 shows an example of the light output when the light output of the light source changes over time. In cases where the light output of the light source changes over time in this way, for example, if the light output 501 is the threshold P th The period of time exceeding this is called the luminescence period t. e It is sufficient to determine that this is the case.

[0024] The above explanation assumes that the image sensor 102 is driven by a global shutter system, but it is also applicable to a rolling shutter system image sensor. Figure 6 is a diagram illustrating the accumulation of each line when a rolling shutter system image sensor is used as the image sensor 102 in the first embodiment. In Figure 6, 601 and 602 indicate the emission period of the light source, and 621-1 indicates the pixel counting operation of line 1 (L1). In imaging in step S203 of Figure 2, the accumulation period T of pixels in each line L1, L2, ..., Lm of the image sensor 102 is as shown in Figure 6. a,1 , T a,2 ,...,T a,m The light source's emission period t e The image sensor 102 is driven by setting it to include at least a portion of the following. Note that since there are differences in the emission time of the blinking light source during the accumulation period of each line, gain correction may be performed by multiplying the pixel count value for each line by a gain according to the emission time of the blinking light source.

[0025] According to this embodiment, based on the flashing frequency and emission period of the flashing light source, the counting frequency for counting can be increased when the flashing light source is emitted, thereby suppressing photon count loss and reducing the degree of color distortion that occurs when imaging with a flashing light source. In this way, by controlling the exposure time and counting frequency in the photon counting type image sensor according to the emission period of the flashing light source, it is possible to suppress count loss and improve the brightness and color reproduction of the image. Furthermore, by increasing the counting frequency according to the emission period of the flashing light source, it is possible to suppress the increase in power consumption due to the increase in counting frequency compared to the case where the counting frequency is increased for the entire period regardless of the emission period of the flashing light source.

[0026] <Second Embodiment> A second embodiment will now be described. In the second embodiment described below, the same configuration and operation as in the first embodiment described above will be omitted, and the differences from the first embodiment will be described. In the second embodiment, imaging of the image sensor 102 is performed under multiple periodic light sources, which is different from the first embodiment.

[0027] In the second embodiment, the image processing unit 103 of the imaging device 100 synthesizes multiple images corresponding to multiple frames obtained by imaging each frame to generate a display image to be displayed on the display unit 108 and a recorded image to be recorded on the recording medium 109 by the recording unit 105. The processing flow from acquisition of light source characteristic information to imaging performed by the imaging device 100 in the second embodiment is the same as the processing in the flowchart shown in Figure 2, so the explanation of redundant processing will be omitted.

[0028] Figure 7 is a diagram illustrating the operation of photon counting in the second embodiment. In step S201, the light source characteristic acquisition unit 110 acquires the blinking period T of each light source as periodic information of the multiple light sources A and B shown in Figure 7. A , T B The control unit 106 estimates the light emission timing and acquires the estimated period information of the light source. Alternatively, the user of the imaging device 100 may specify the number of light sources and the frequency information of each light source using the operation unit 107, and the control unit 106 will notify the light source characteristic acquisition unit 110 of the specified frequency information, and the light source characteristic acquisition unit 110 will identify the period information of the light source based on that frequency information. In the example shown in Figure 7, the blinking period T of light source B B The blinking period T of light source A. A Let's assume it's twice as much. In other words, the blinking frequency f of light source A. A The blinking frequency f of light source B is B Assume it is twice as much. Also, in one frame, light source A emits light four times, and light source B emits light twice.

[0029] In step S202, the control unit 106 determines the accumulation timing and counting frequency on the image sensor 102 in accordance with each light emission period of the light source within the set shutter speed, based on the light source characteristic information acquired in step S201. Alternatively, instead of the light source characteristic information acquired in step S201, frequency information specified by the user of the imaging device 100 via the operation unit 107 may be used as the characteristic information for each light source. Furthermore, in step S202, to account for the difference in brightness of each light source due to differences in blinking frequency, the count values ​​(counted values) K1, K2, ... obtained as counting results in each frame may be multiplied by an appropriate gain. Gain G multiplied by the counting results of light source A and light source B. A , G B For example, the blinking frequency f of each light source. A ,f B It is determined based on the ratio of . In the example shown in Figure 7, for example, G A :G B =f A :f B The ratio becomes 2:1.

[0030] In step S203, the image sensor 102 performs an accumulation operation in accordance with the emission period of each light source, based on the accumulation timing and counting frequency determined in step S202, which are set by the control unit 106. In the second embodiment, the light source that the image sensor 102 counts is changed in adjacent frames. In the example shown in Figure 7, for example, frames <1> Now, let's set light source A to frame <2> Now, let's set light source B to frame <3> So, the light source being counted is changed frame by frame, such as light source A. For the count values ​​(count values) K1, K2, ... obtained as the counting result in each frame, the gain G determined in step S202 is applied according to the type of light source accumulated in each frame. A , G B Multiplication by is also possible. In addition, the image sensor 102 stops counting during periods other than the light emission period to be counted in each frame (other than the exposure period of each frame).

[0031] In the second embodiment, the image processing unit 103 synthesizes the images of adjacent frames obtained in this way to acquire the display image and recorded image for each frame. In the example shown in Figure 7, frames <1> and frame <2> By combining the images acquired in each location, a single image is generated, and the frame <2> and frame <3> A single image is generated by combining the images acquired in each of the following locations. Note that the selection of frames to be combined is not limited to these.

[0032] In the example shown in Figure 7, there are two types of light sources, but it is possible to perform the same counting for three or more types of light sources and generate an image based on the count value obtained as the counting result in each frame. Figure 8 is a diagram illustrating another example of the operation of photon counting in the second embodiment. For example, as shown in Figure 8, if there are three light sources, light source A, light source B, and light source C, the frame <1> Now, let's set light source A to frame <2> Now, let's set light source B to frame <3> So, the light source C is changed on a frame-by-frame basis, and so on. Then, based on the count values ​​K1, K2, ... obtained as the counting result in each frame, the display image or recorded image is obtained by combining multiple frames. In the example shown in Figure 8, for example, three consecutive frames are called frames <1> frame <2> and frame <3> The image processing unit 103 combines the images acquired in each step to obtain a final single display image or recorded image.

[0033] Furthermore, even if the periods of three or more light sources are different, the counting can be performed in the same way as explained with reference to Figures 7 and 8, and an image can be generated based on the count value obtained as a result of the counting in each frame.

[0034] According to this embodiment, similar to the first embodiment, it is possible to suppress count loss in the photon counting type image sensor and suppress the degree of color distortion that occurs when shooting with a flashing light source, thereby improving the brightness and color reproduction of the image. Furthermore, by increasing the counting frequency according to the emission period of the flashing light source, it is possible to suppress the increase in power consumption due to the increase in counting frequency compared to the case where the counting frequency is increased for the entire period regardless of the emission period of the flashing light source. In addition, even if there are multiple flashing light sources, it is possible to acquire images taken with multiple flashing light sources by combining images of multiple frames captured by changing the light source being counted on a frame-by-frame basis.

[0035] <Third Embodiment> A third embodiment will now be described. In the third embodiment described below, descriptions of the same configuration and operation as in the first or second embodiment described above will be omitted, and the differences from the first or second embodiment will be described. The third embodiment differs from the first embodiment in that imaging of the image sensor 102 is performed when both a flashing light source and a steady light source with a constant light output are present.

[0036] In the third embodiment, the image processing unit 103 of the imaging device 100 synthesizes multiple images to generate a display image to be displayed on the display unit 108 and a recording image to be recorded on the recording medium 109 by the recording unit 105. The processing flow from acquiring characteristic information of the light source to imaging performed by the imaging device 100 in the third embodiment is the same as the processing in the flowchart shown in Figure 2, so the explanation of redundant processing will be omitted.

[0037] Figure 9 is a diagram illustrating the operation of photon counting in the third embodiment. In step S201, the light source characteristic acquisition unit 110 estimates the period information of the blinking light source shown in Figure 9, and the control unit 106 acquires the estimated period information of the light source. Alternatively, the user of the imaging device 100 may specify the number of light sources and the frequency information of each light source using the operation unit 107, and the control unit 106 may notify the light source characteristic acquisition unit 110 of the specified frequency information, and the light source characteristic acquisition unit 110 may identify the period information of the light source based on that frequency information.

[0038] In step S202, the control unit 106 determines the counting frequency for the steady light source and the flashing light source. K is the predetermined number of counts that the image sensor 102 can count in one frame. The counting frequency f of the flashing light source. cntp The counting period of the flashing light source is t cntp as, f cntp =K / t cntp This is determined by the counting frequency f of the steady-state light source. cntc The counting period for the steady-state light source is t cntc as, f cntc =K / t cntc This is determined by the following. As shown in Figure 9, the accumulation operation (counting operation) for the steady light source is performed in segments, avoiding the light emission period of the flashing light source, so t cntc =t cntc1 +t cntc2 Therefore, in the example shown in Figure 9, the counting frequency f of the steady-state light source is... cntc is, f cntc =K / (t cntc1 +t cntc2 The counting frequency is determined by the method described above. However, the determination of the counting frequency is not limited to the example described above and may be determined by other methods.

[0039] In step S203, the image sensor 102 performs the counting operation by changing the light source being counted in adjacent frames. In the example shown in Figure 9, the image sensor 102 performs the counting operation based on the accumulation timing and counting frequency determined in step S202, which are set by the control unit 106, for each frame. <1> Next, count (accumulate) during the light emission period of the flashing light source. <1> Therefore, the image sensor 102 stops counting during periods when the flashing light source is not emitting light. Next frame <2> Then, the image sensor 102 performs counting (accumulation) for the steady light source based on the accumulation timing and counting frequency determined in step S202 set by the control unit 106, and stops the counting operation during the light emission period of the flashing light source. From the 3rd frame onward, the light source being counted is changed according to the frame, and the counting (accumulation) for the flashing light source and the counting (accumulation) for the steady light source are repeated.

[0040] In the second embodiment, the image processing unit 103 synthesizes images of adjacent frames obtained by counting for each light source to acquire the display image and recorded image for each frame. In the example shown in Figure 7, frames <1> and frame <2> By combining the images acquired in each frame, a single image is generated. By performing the same combining process for subsequent frames, display images and recorded images can be obtained.

[0041] In the description above, the light source to be counted is changed in adjacent frames. However, in this embodiment, as shown in Figure 10, the frequency to be counted may be changed within a single frame. Figure 10 illustrates another example of the operation of photon counting in the third embodiment. For example, as shown in Figure 10, the frequency counted by the image sensor 102 is changed between the period when the flashing light source is emitting light and the period when only the steady light source is emitting light, thereby preventing count loss during the flashing light source's emission period.

[0042] For example, in the example shown in Figure 10, the counting frequency is the light output P of the flashing light source. pwm and the light output P of the steady-state light source const and is determined based on a predetermined number K as the total count of the image sensor 102 within the frame. In addition, the counting frequency of the steady light source is f cntc The counting frequency of the flashing light source is f cntp In that case, f cntc :f cntp =P pwm :P const Alternatively, the light emission period t of the flashing light source. e and storage period (shutter speed) T sh Using the inverse ratio of f cntc :f cntp =T sh :t e But that's fine too. Or, the accumulation period T sh Instead, the frame period T is based on the frame rate. fps You may use this method. The determination of the counting frequency is not limited to the examples described above and may be determined by other methods.

[0043] Figure 11 illustrates another example of the operation of photon counting in the third embodiment. In the example shown in Figure 10 above, counting is performed continuously during the period when the flashing light source is emitting light and the period when only the steady light source is emitting light. However, as shown in Figure 11, the periods for counting the flashing light source and the steady light source may be divided. The time widths of the flashing light source counting period and the steady light source counting period may be the same or different.

[0044] According to this embodiment, similar to the third embodiment, it is possible to suppress count loss in the photon counting image sensor and reduce the degree of color distortion that occurs when imaging with a flashing light source, thereby improving the brightness and color reproduction of the image. Furthermore, by increasing the counting frequency according to the emission period of the flashing light source, it is possible to suppress the increase in power consumption due to the increase in counting frequency compared to the case where the counting frequency is increased for the entire period regardless of the emission period of the flashing light source. In addition, when both flashing light sources and steady light sources are present, it is possible to perform counting on the steady light source in addition to the flashing light source, thereby enabling imaging that takes the steady light source into consideration.

[0045] Furthermore, each of the embodiments described above is applicable to devices having a photon counting image sensor. For example, the present invention is applicable to any device capable of capturing images, such as a digital camera, mobile phone terminal, portable image viewer, television equipped with an image sensor, digital photo frame, music player, game console, e-book reader, etc.

[0046] (Other embodiments of the present invention) The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by a process in which one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0047] It should be noted that the embodiments described above are merely examples of how the present invention can be implemented, and the technical scope of the present invention should not be interpreted as being limited by them. In other words, the present invention can be implemented in various forms without departing from its technical concept or its main features.

[0048] The disclosure of this embodiment includes the following configurations and methods, etc. (Composition 1) An image sensor having a light-receiving unit that outputs a pulse signal in response to the incidence of photons, and a counting unit that counts the pulse signals, An acquisition means for acquiring the flashing frequency and emission period of a light source, An imaging apparatus characterized by having control means for controlling the exposure period of an image in the image sensor and the counting frequency used for counting in the counting unit, based on the acquired blinking frequency and emission period of the light source. (Configuration 2) The imaging apparatus according to configuration 1, characterized in that the control means controls the exposure period to match the acquired light emission period of the light source. (Composition 3) The imaging apparatus according to configuration 1 or 2, characterized in that the control means controls the exposure period so that the difference between the acquired light emission period of the light source and the exposure period does not exceed an acceptable range. (Composition 4) The imaging apparatus according to any one of configurations 1 to 3, characterized in that the control means controls the counting frequency so that the counting unit can count a predetermined number of times within the exposure period. (Composition 5) The imaging apparatus according to any one of configurations 1 to 4, characterized in that the control means stops counting in the counting unit during periods other than the exposure period. (Composition 6) The imaging apparatus according to any one of configurations 1 to 5, characterized in that the control means makes the counting frequency during the exposure period higher than during periods other than the exposure period. (Composition 7) The acquisition means acquires the flashing frequency and light emission period of the plurality of light sources, The imaging apparatus according to any one of configurations 1 to 6, characterized in that the control means controls the exposure period so that the exposure period matches the emission period of at least one of the plurality of light sources. (Composition 8) The imaging apparatus according to configuration 7, characterized in that it has an image processing means that generates a single image by synthesizing images of multiple frames obtained by changing the light source to be counted from among the multiple light sources on a frame-by-frame basis. (Composition 9) The imaging device according to configuration 8, characterized in that it synthesizes images of the plurality of frames by multiplying them by a gain corresponding to the counted blinking frequency of the light source. (Composition 10) The imaging apparatus according to any one of configurations 1 to 9, characterized in that the control means changes the counting frequency between the light emission period of the flashing light source and the light emission period of the steady light source. (Composition 11) The acquisition means acquires information regarding the light output of the flashing light source and the light output of the steady light source. The imaging apparatus according to configuration 10, characterized in that the control means determines the counting frequency based on the light output of the flashing light source, the light output of the steady light source, and a predetermined sum of the count values ​​from the flashing light source and the steady light source during the exposure period. (Composition 12) The imaging apparatus according to configuration 10, characterized in that the control means determines the counting frequency based on the inverse ratio of the light emission period of the flashing light source, the shutter speed, and a predetermined sum of the count values ​​of the flashing light source and the steady light source within the exposure period. (Composition 13) The aforementioned image sensor is a rolling shutter type image sensor, The imaging apparatus according to any one of configurations 1 to 12, characterized in that the control means controls the exposure period such that the exposure period of each line of the image sensor includes at least a portion of the light emission period of the light source. (Method 1) A control method for an imaging device having an image sensor having a light receiving unit that outputs a pulse signal in response to the incidence of photons, and a counting unit that counts the pulse signal, An acquisition process to acquire the flashing frequency and emission period of a light source, A control method for an imaging device, characterized by comprising a control step of controlling the exposure period of an image in the image sensor and the counting frequency used for counting in the counting unit, based on the acquired blinking frequency and light emission period of the light source. [Explanation of Symbols]

[0049] 100: Imaging device 101: Imaging lens 102: Image sensor 103: Image processing unit 104: Memory unit 105: Recording unit 106: Control unit 107: Operation unit 108: Display unit 110: Light source characteristic acquisition unit

Claims

1. An image sensor having a light-receiving unit that outputs a pulse signal in response to the incidence of photons, and a counting unit that counts the pulse signals, An acquisition means for acquiring the flashing frequency and emission period of a light source, An imaging apparatus characterized by having control means for controlling the exposure period of an image in the image sensor and the counting frequency used for counting in the counting unit, based on the acquired blinking frequency and emission period of the light source.

2. The imaging apparatus according to claim 1, characterized in that the control means controls the exposure period to match the acquired light emission period of the light source.

3. The imaging apparatus according to claim 1, characterized in that the control means controls the exposure period so that the difference between the acquired light emission period of the light source and the exposure period does not exceed an acceptable range.

4. The imaging apparatus according to claim 1, characterized in that the control means controls the counting frequency so that the counting unit can count a predetermined number of times within the exposure period.

5. The imaging apparatus according to claim 1, characterized in that the control means stops counting in the counting unit during periods other than the exposure period.

6. The imaging apparatus according to claim 1, characterized in that the control means makes the counting frequency during the exposure period higher than during periods other than the exposure period.

7. The acquisition means acquires the flashing frequency and light emission period of the plurality of light sources, The imaging apparatus according to claim 1, characterized in that the control means controls the exposure period so that the exposure period matches the emission period of at least one of the plurality of light sources.

8. The imaging apparatus according to claim 7, characterized in that it has an image processing means that generates a single image by synthesizing images of multiple frames obtained by changing the light source to be counted from among the multiple light sources on a frame-by-frame basis.

9. The imaging apparatus according to claim 8, characterized in that it synthesizes images of the plurality of frames by multiplying them by a gain corresponding to the counted blinking frequency of the light source.

10. The imaging apparatus according to claim 1, characterized in that the control means changes the counting frequency between the light emission period of the flashing light source and the light emission period of the steady light source.

11. The acquisition means acquires information regarding the light output of the flashing light source and the light output of the steady light source. The imaging apparatus according to claim 10, characterized in that the control means determines the counting frequency based on the light output of the flashing light source, the light output of the steady light source, and a predetermined sum of the count values ​​from the flashing light source and the steady light source during the exposure period.

12. The imaging apparatus according to claim 10, characterized in that the control means determines the counting frequency based on the inverse ratio of the light emission period of the flashing light source, the shutter speed, and a predetermined sum of the count values ​​of the flashing light source and the steady light source within the exposure period.

13. The aforementioned image sensor is a rolling shutter type image sensor, The imaging apparatus according to claim 1, characterized in that the control means controls the exposure period such that the exposure period of each line of the image sensor includes at least a portion of the light emission period of the light source.

14. A control method for an imaging device having an image sensor having a light receiving unit that outputs a pulse signal in response to the incidence of photons, and a counting unit that counts the pulse signal, An acquisition process to acquire the flashing frequency and emission period of a light source, A control method for an imaging device, characterized by comprising a control step of controlling the exposure period of an image in the image sensor and the counting frequency used for counting in the counting unit, based on the acquired blinking frequency and emission period of the light source.

Images