Imaging system, control method, program, and storage medium

Abstract

To improve visibility of pixel blocks for which an exposure parameter can be set.SOLUTION: An imaging system includes: an imaging unit 201 including an imaging surface on which pixel blocks each including a plurality of pixels are two-dimensionally arranged, exposure parameters being separately settable for the pixel blocks; a determination unit 401 configured to determine a first exposure parameter and a second exposure parameter so that a difference between the first exposure parameter and the second exposure parameter is greater than or equal to a predetermined value; an exposure control unit 402 configured to set the first exposure parameter and the second exposure parameter, each determined by the determination unit 401, alternately to control imaging by the imaging unit; and a display unit 104 configured to display an image captured by the imaging unit.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to an imaging system, a control method, a program, and a storage medium. [Background technology]

[0002] In surveillance cameras and the like, it is desirable to capture images with an expanded dynamic range in environments with large differences in brightness or different lighting conditions.

[0003] Patent Document 1 discloses a technique for expanding the dynamic range by controlling the exposure time for each of a plurality of pixel blocks on the imaging surface. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] WO2017 / 018188 Summary of the Invention [Problem to be solved by the invention]

[0005] The problem to be solved by the present invention is to improve the visibility of pixel blocks for which exposure parameters such as exposure time can be set. [Means for solving the problem]

[0006] In order to solve the above problem, an imaging system according to one aspect of the present invention includes an imaging unit in which pixel blocks each consisting of a plurality of pixels are arranged two-dimensionally on an imaging surface, the imaging unit being capable of setting exposure parameters individually for the pixel blocks; a determination unit that determines a difference between a first exposure parameter and a second exposure parameter so that the difference is equal to or greater than a predetermined value; and a setting unit that applies the first exposure parameter and the second exposure parameter determined by the determination unit to the pixel blocks. In the two-dimensional The imaging device includes an exposure control unit that alternately sets and controls imaging, and a display unit that displays the image captured by the imaging unit. [Effects of the Invention]

[0007] According to the present invention, the visibility of pixel blocks for which exposure parameters such as exposure time can be set is improved. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of an imaging system according to a first embodiment. [Figure 2] FIG. 1 is a diagram showing an example of the configuration of an imaging apparatus according to a first embodiment. [Figure 3] FIG. 2 is a diagram showing an example of the configuration of a client device according to the first embodiment. [Figure 4] FIG. 1 is a diagram showing an example of the functional configuration of an imaging system according to a first embodiment. [Figure 5] 2A and 2B are diagrams showing an example of an imaging surface and a captured image in the imaging device according to the first embodiment. [Figure 6] FIG. 2 is a diagram showing an example of the operation of the imaging system according to the first embodiment. [Figure 7] FIG. 2 is a diagram schematically showing an example of a pixel block on an imaging surface of the imaging device according to the first embodiment. [Figure 8] FIG. 2 is a view showing an example of an image captured by the imaging system according to the first embodiment. [Figure 9] FIG. 2 is a diagram schematically showing an example of a pixel block on an imaging surface of the imaging device according to the first embodiment. [Figure 10] 3A and 3B are views showing an image captured by the imaging system according to the first embodiment and a designated area. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. The embodiments described below are examples of means for realizing the present invention, and should be appropriately modified or changed depending on the configuration of the device to which the present invention is applied and various conditions. The present invention is not limited to the following embodiments. Furthermore, a configuration may be made by appropriately combining parts of each embodiment described below.

[0010] <Embodiment 1> (System Configuration) 1 is a diagram showing an example of hardware constituting an imaging system according to this embodiment. The imaging system 100 includes an imaging device 101, a network 102, a client device 103, a display unit 104, and an input unit 105.

[0011] The imaging device 101 can communicate with the client device 103 via a network 102. The imaging device 101 captures an image of a subject to generate an image, and transmits the captured image to the client device 103 via the network 102. A display unit 104 and an input unit 105 are connected to the client device 103. Images sent from the imaging device 101 are output to and displayed on the display unit 104 via the client device 103. The input unit 105 is a keyboard, a mouse, or the like, and is used to operate the client device 103. Operations on the client device 103 include setting imaging conditions for the imaging device 101 via the network 102 and operating PTZ (Pan Tilt Zoom).

[0012] In this embodiment, the client device 103, the display unit 104, and the input unit 105 are separate entities, but the client device 103, the display unit 104, and the input unit 105 may be integrated into one unit, as in a notebook PC with a touch panel display. Also, the connection does not have to be via the network 102, and the surveillance camera 101 and the client device 103 may be directly connected. Furthermore, the imaging device 101, the client device 103, the display unit 104, and the input unit 105 may all be integrated into one unit, as in a consumer camera with a touch panel display.

[0013] (Device configuration) 2 is a diagram showing an example of the device configuration of an imaging device according to this embodiment. The imaging device 101 has an imaging unit 201, a network I / F 202, a CPU (Central Processing Unit) 203, a RAM (Random Access Memory) 204, and a ROM (Read Only Memory) 205. The imaging optical system 200 is detachably provided on the surveillance camera 101.

[0014] The imaging optical system 200 is a lens that focuses light from a subject onto an imaging surface of an imaging element 201a (described later), and is composed of, for example, a zoom lens, a focus lens, and a blur correction lens.

[0015] In this embodiment, the image capturing device 101 and the image capturing optical system 200 are provided as separate, detachable bodies, but the image capturing device 101 and the image capturing optical system 200 may be integrated into one body, as in a lens-integrated camera.

[0016] The imaging unit 201 captures an image of a subject using the imaging optical system 200 and generates an image. The imaging unit 201 has an image sensor 201a, an amplifier 201b, and an A / D converter 201c.

[0017] The image sensor 201a photoelectrically converts light from a subject collected by the imaging optical system 200 and outputs an image signal. Pixel blocks each consisting of a plurality of pixels (e.g., 128×128 pixels) are arranged two-dimensionally on the imaging surface of the image sensor 201a (image sensor 201). Exposure parameters can be set individually for each pixel block. Here, the exposure parameters are parameters related to exposure, and include at least one of exposure time, analog gain (described later), and exposure value. The image sensor 201a is, for example, a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor.

[0018] The amplifier 201b amplifies and outputs the electrical signal output from the image sensor 201a. An amplifier 201b is provided for each pixel, and the signal amplification factor (analog gain) can be set and changed for each pixel block of the image sensor 201a.

[0019] The A / D converter 201c converts the analog image signal output from the amplifier 201b into a digital signal.

[0020] The network I / F 202 is an interface for transmitting image data, which is a digital signal output from the A / D conversion unit 201c, to the client device 103 via the network 102. The image data transmitted thereby may be image data that has been encoded by image processing into a predetermined file format such as Motion JPEG, H264, or H265. The image data may be stored in an internal storage device such as a RAM 204 or ROM 205, which will be described later, or in a removable storage medium such as an SD card.

[0021] The CPU 203 is a central processing unit that controls the image capture device 101. The RAM 204 temporarily stores computer programs executed by the CPU 203. The RAM 204 provides a work area used when the CPU 203 executes processing. The RAM 205 also functions as a frame memory and a buffer memory.

[0022] The ROM 205 stores a program for the CPU 203 to control the image capturing apparatus 101 and the like.

[0023] 3 is a diagram showing an example of the device configuration of a client device according to this embodiment. The client device 103 is an information processing device having a CPU 301, a RAM 302, a ROM 303, an input I / F 304, an output I / F 305, and a network I / F 406.

[0024] The CPU 301 is a central processing unit that controls the client device 103 .

[0025] The RAM 302 temporarily stores programs and the like used by the CPU 301 to control the client device 103. The RAM 302 provides a work area used by the CPU 301 when executing processing. The RAM 302 also functions as a frame memory and a buffer memory.

[0026] The ROM 303 stores a program for the CPU 301 to control the client device 103 and the like.

[0027] The input I / F 304 is an interface for connecting the input unit 105 and the client device 103 , and accepts operations for the client device 103 input by the user via the input unit 105 .

[0028] The output I / F 306 is an interface for connecting to the display unit 104 and outputs the image output from the imaging device 101 .

[0029] The network I / F 306 is an interface that connects to the surveillance camera 101 via the network 102 , and is used to input operation information for the surveillance camera 101 and to receive images output from the surveillance camera 101 .

[0030] (Functional configuration) Fig. 4 is a diagram showing an example of the functional configuration of the imaging system according to this embodiment. The imaging system 100 has an imaging unit 201, a determination unit 401, an exposure control unit 402, and a display unit 104. Each unit shown in Fig. 4 has a function that can be achieved by the CPU 203 (or CPU 301) reading and executing a program stored in the RAM 204 (or 302) or the ROM 205 (or ROM 303).

[0031] In the imaging unit 201, pixel blocks each consisting of a plurality of pixels are arranged two-dimensionally on the imaging surface, and exposure parameters can be set individually for each pixel block.

[0032] The determination unit 401 determines the difference between the first exposure parameter and the second exposure parameter so that it is equal to or greater than a predetermined value. The first exposure parameter and the second exposure parameter are parameters including at least one of exposure time, gain, and exposure value. Therefore, the difference between the first exposure parameter and the second exposure parameter is the difference in exposure time, gain, and exposure value, for example. The predetermined value is a threshold, and is a different threshold for each of the exposure time, gain, and exposure value. In other words, at least one of the threshold for exposure time, gain, and exposure value is the predetermined value. The predetermined value is set by a user or a designer.

[0033] The exposure control unit 402 performs control to alternately set the first exposure parameter and the second exposure parameter determined by the determination unit 401 to a plurality of pixel blocks and capture an image.

[0034] The display unit 104 displays an image captured by the image capturing unit 201 under the control of the exposure control unit 402 .

[0035] (Operation description) The operation of the imaging system according to this embodiment will be described below with reference to FIGS.

[0036] FIG. 5 shows an example of the imaging surface of the image sensor 201a and a captured image. FIG. 5(a) is a schematic diagram of the imaging surface 501, and FIG. 5(b) is a diagram showing an example of an image captured by the imaging device 101. As shown in FIG. 5(a), pixel blocks 502 each consisting of a plurality of pixels are arranged two-dimensionally on the imaging surface 501. FIG. 5(b) shows an image displayed on the display unit 104, illustrating the state before the operation of the imaging system according to this embodiment begins. The position of each pixel block is expressed in a two-dimensional coordinate system, with the block in the ith column and jth row from the bottom left being represented as (i, j). Therefore, as shown in FIG. 5(a), the coordinates of the bottom left pixel block are represented as (1, 1), the pixel block adjacent to the right is represented as (2, 1), and the pixel block adjacent above is represented as (1, 2).

[0037] Fig. 6 is a flowchart showing an example of the operation of the imaging system according to this embodiment. The operation of the flowchart shown in Fig. 6 starts when the CPU 203 (or CPU 301) reads and executes a program stored in the RAM 204 (or 302) or the ROM 205 (or ROM 303). Therefore, each operation of the imaging system 100 may be performed by the imaging device 101 or the client device 103.

[0038] In step S601, an area for displaying pixel block boundaries is designated. The area for displaying pixel block boundaries is designated by a designation unit (not shown) when an area is input from the input unit 105. If no area is input to the input unit 105, the area for displaying pixel blocks of the entire screen is determined. In this embodiment, a case will be described in which pixel block boundaries are displayed across the entire image, but it is also possible to display pixel block boundaries of an area designated by the user via the input unit 105.

[0039] In step S602, the determination unit 401 determines the difference between the first exposure time (first exposure parameter) and the second exposure time (second exposure parameter) so that the difference is equal to or greater than a predetermined value. FIG. 7 shows an example of a screen for setting the difference between the first exposure time and the second exposure time. In FIGS. 7(a) and 7(b), a predetermined value (threshold value) can be set by dragging a knob 702 on a slider 701 with a finger, a mouse, or the like. The determination unit 401 determines the difference between the first exposure time and the second exposure time so that the difference is equal to or greater than the set predetermined value. In this embodiment, the difference between the first exposure time and the second exposure time is determined to be a predetermined value. Note that the first exposure time and the second exposure time may have any value as long as the difference between them is equal to or greater than the predetermined value. For example, if the difference in exposure time is set to 1 / 60, the first exposure time may be 1 / 30 and the second exposure time may be 1 / 60, or the first exposure time may be 1 / 60 and the second exposure time may be 1 / 30.

[0040] In step S603, the exposure control unit 402 controls the image capture by alternately setting the first exposure parameter and the second exposure parameter to pixel blocks on the imaging surface. Figure 7(a) shows an example where the predetermined value is set small, and Figure 7(b) shows an example where the predetermined value is set large.

[0041] In step S604, the display unit 104 displays the image captured by the imaging unit 201 under the control of step S603. FIG. 8 shows an example of an image displayed by the display unit 104. FIG. 8(a) shows the image before the operation starts, and FIG. 8(b) shows the image after the operation of step S604 is completed. In FIG. 8(a), it is difficult to visually recognize the boundaries of pixel blocks for which exposure parameters can be set. On the other hand, in FIG. 8(b), the visibility of the boundaries of pixel blocks is improved. This is effective when a user wants to assign an appropriate pixel block to a specific subject (e.g., a target subject). If the boundaries of pixel blocks can be visually recognized, it becomes possible to change the size of the pixel blocks or adjust the imaging position so that the target subject does not straddle the boundaries of the pixel blocks or so that the contours of the subject fit within the pixel blocks.

[0042] As described above, the imaging system according to this embodiment can improve the visibility of pixel blocks for which exposure parameters such as exposure time can be set.

[0043] A modification of this embodiment will be described below. In this embodiment, the case where the difference between the first and second exposure parameters is determined to be equal to or greater than a certain value has been described. However, the determination unit 401 may also determine the difference between the first and second target luminance values ​​to be equal to or greater than a predetermined value. Here, the target luminance is the target value of the luminance output from the pixel block. FIG. 9 shows an example of a screen for setting the luminance difference as a threshold. As shown in FIG. 9(a), as the knob 702 is turned up, the difference between the first and second target luminance values ​​increases, making the boundaries of the pixel blocks more clearly visible. On the other hand, as shown in FIG. 9(b), turning down the knob can improve the visibility of the pixel blocks without reducing the visibility of the image. The exposure control unit 402 controls the exposure parameters for each pixel block so that the luminance of the pixel block alternates between the first target luminance and the second target luminance.

[0044] Fig. 10 shows an example of a case where pixel blocks are visible only in a partial area of ​​an image in the imaging system of this embodiment. As shown in Fig. 10, the user can specify an area in which pixel blocks are to be displayed by dragging the input unit 105. In step S601, if a specified area 1001 is specified, an indication unit (not shown) indicates the specified area 1001 as an area in which pixel blocks are to be displayed. The exposure control unit 402 performs control to alternately set a first exposure parameter and a second exposure parameter to the pixel blocks corresponding to the area specified by the indication unit and capture the image. This makes it possible to make only the area in which the user has visually recognized pixel blocks visible.

[0045] <Other embodiments> The present invention can be realized by a process of reading and executing a program that realizes one or more functions of the above-described first embodiment. This program is supplied to a system or device via a network or a storage medium, and is read and executed by one or more processors in the computer of the system or device. The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. [Explanation of symbols]

[0046] 100 Imaging System 104 Display section 201 Imaging unit 401 Decision Section 402 Exposure control unit

Claims

1. an imaging unit in which pixel blocks each consisting of a plurality of pixels are arranged two-dimensionally on an imaging surface, and exposure parameters can be set individually for each pixel block; a determination unit that determines a difference between the first exposure parameter and the second exposure parameter to be equal to or greater than a predetermined value; an exposure control unit that performs control to capture an image by alternately setting the first exposure parameter and the second exposure parameter determined by the determination unit to the pixel block in the two-dimensional pattern; a display unit that displays an image captured by the imaging unit; An imaging system having:

2. 2. The imaging system according to claim 1, wherein the exposure parameters include at least one of an exposure time, a gain, and an exposure value.

3. the determination unit determines a difference between a first target luminance and a second target luminance to be equal to or greater than a predetermined value; the exposure control unit performs control to individually set the exposure parameters for each of the pixel blocks arranged in the two-dimensional shape and capture an image so that the first target luminance and the second target luminance are alternately output from the pixel blocks in the two-dimensional shape.

3. The imaging system according to claim 1, wherein the imaging system includes: a first optical fiber;

4. 4. The imaging system according to claim 1, further comprising an input unit for inputting the predetermined value.

5. further comprising an indicator for indicating a predetermined area from the image displayed on the display unit; the exposure control unit performs control to capture an image by alternately setting the first exposure parameter and the second exposure parameter in the two-dimensional pattern to pixel blocks corresponding to the area designated by the designation unit.

5. The imaging system according to claim 1, wherein the imaging system includes: a first lens;

6. 6. The imaging system according to claim 1, wherein the determination unit determines the first exposure parameter and the second exposure parameter.

7. 4. The imaging system according to claim 3, wherein the determination unit determines the first target luminance and the second target luminance.

8. an imaging step of individually setting exposure parameters for pixel blocks each made up of a plurality of pixels arranged two-dimensionally on an imaging surface and capturing an image; a determining step of determining a difference between the first exposure parameter and the second exposure parameter to be equal to or greater than a predetermined value; an exposure control step of performing control to capture an image by alternately setting the first exposure parameter and the second exposure parameter determined in the determination step to the pixel block in the two-dimensional pattern; a display step of displaying the image captured in the exposure control step; A control method for an imaging system having the above features.

9. 9. The method for controlling an image capturing system according to claim 8, wherein the exposure parameters include at least one of an exposure time, a gain, and an exposure value.

10. In the determining step, a difference between the first target luminance and the second target luminance is determined to be equal to or greater than a predetermined value; the exposure control step performs control to capture an image by individually setting the exposure parameters for each of the pixel blocks arranged in the two-dimensional shape so that the first target luminance and the second target luminance are alternately output from the pixel blocks in the two-dimensional shape.

10. The method for controlling an imaging system according to claim 8 or 9.

11. 11. The method for controlling an imaging system according to claim 8, further comprising an input step of inputting the predetermined value.

12. The method further includes a designation step of designating a predetermined area from the image displayed in the display step, In the exposure control step, the first exposure parameter and the second exposure parameter are alternately set in the two-dimensional pattern to pixel blocks corresponding to the area designated in the designation step, and control is performed to capture an image.

12. The method for controlling the imaging system according to claim 8.

13. 13. The method of controlling an imaging system according to claim 8, wherein the determining step determines the first exposure parameter and the second exposure parameter.

14. 11. The method for controlling an imaging system according to claim 10, wherein the determining step determines the first target luminance and the second target luminance.

15. A program for causing a computer to execute the method for controlling an imaging system according to any one of claims 8 to 14.

16. A computer-readable storage medium storing the program according to claim 15.

Images