Imaging control device, imaging control method and program

Abstract

To reduce discomfort of an image while making it possible to set an exposure time of each area obtained by dividing an image.SOLUTION: An imaging control device according to one embodiment comprises: first instruction means which instructs imaging so as to obtain a first image captured on the basis of, an exposure time set to each divided area obtained by dividing an image; calculation means which on the basis of the exposure time of each divided area, calculates an exposure time to apply uniformly to a plurality of divided areas included in the image; and second instruction means which instructs imaging so as to obtain a second image captured on the basis of, the exposure time calculated by the calculation means.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an imaging control device, an imaging control method, and a program. [Background technology]

[0002] Patent Document 1 discloses a technology that has an exposure time control means that controls the exposure time for each pixel or each pixel group of an imaging means based on a set value, and sets different exposure time setting values ​​based on photometry values ​​for photometry and focus detection. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-161329 A Summary of the Invention [Problem to be solved by the invention]

[0004] The problem to be solved by the present invention is to reduce the sense of incongruity in an image while enabling the setting of an exposure time for each region obtained by dividing an image. [Means for solving the problem]

[0005] An imaging control device according to one aspect of the present invention includes: An imaging control device for controlling an imaging means having an imaging element capable of setting an exposure time for each of a plurality of divided regions obtained by dividing an image, the imaging control device controlling the imaging means Set for each division area was done Exposure time Using Image 1 To get A first instruction means for instructing imaging and based on an exposure time for each of the divided regions, The above A calculation means for calculating an exposure time to be uniformly applied to a plurality of divided regions; With respect to the imaging means, The exposure time calculated by the calculation means is uniformly applied to all of the multiple divided regions. Second image To get A second instruction means for instructing imaging; and the calculation means calculates an exposure time to be uniformly applied to all of the plurality of divided regions based on a distribution of exposure times for the plurality of divided regions set when the first image is acquired. It is characterized by: Effect of the Invention

[0006] According to one aspect of the present invention, it is possible to reduce the sense of incongruity in an image while enabling the setting of an exposure time for each region obtained by dividing an image. [Brief description of the drawings]

[0007] [Figure 1] 1 is a block diagram showing an example of the arrangement of an imaging apparatus according to a first embodiment. [Diagram 2] 4 is a flowchart showing an imaging control method according to the first embodiment. [Diagram 3] 3A and 3B are views showing examples of a first image and a second image according to the first embodiment. [Figure 4] 5A to 5C are diagrams showing examples of exposure time settings for each divided area according to the first embodiment. [Diagram 5] 6A to 6C are diagrams showing an example of calculation of an exposure time of a second image according to the first embodiment. [Figure 6] FIG. 2 is a block diagram showing an example of the hardware configuration of a control processing unit according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, the embodiments of the present invention will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the present invention, and not all of the combinations of features described in the embodiments are necessarily essential to the solution of the present invention. The configurations of the embodiments may be appropriately modified or changed depending on the specifications of the device to which the present invention is applied and various conditions (such as conditions of use and environment of use). The technical scope of the present invention is determined by the claims, and is not limited by the individual embodiments below.

[0009] First Embodiment FIG. 1 is a block diagram showing an example of the configuration of an imaging device according to the first embodiment. Of the functional blocks shown in FIG. 1, the functions realized by software are stored in a memory such as a ROM (Read Only Memory) in order to provide the functions of the functional blocks. Then, the programs are read into a RAM (Random Access Memory) and executed by a CPU (Central Processing Unit) to realize the functions. For the functions realized by hardware, for example, a specific compiler may be used to automatically generate a dedicated circuit on an FPGA from a program for realizing the functions of the functional blocks. FPGA stands for Field Programmable Gate Array. Also, a gate array circuit may be formed in the same manner as an FPGA to realize the functions as hardware. Also, the functions may be realized by an ASIC (Application Specific Integrated Circuit). Note that the configuration of the functional blocks shown in FIG. 1 is an example, and a plurality of functional blocks may constitute one functional block, or any of the functional blocks may be divided into blocks performing a plurality of functions.

[0010] 1, the imaging device 10 includes an imaging section 11, an image processing section 12, an imaging control section 13, a storage section 14, an operation section 15, a display section 16, and a communication section 17. The imaging section 11 includes an imaging optical system 110, an aperture 111, and a solid-state imaging element 112. The imaging control section 13 includes a first imaging instruction section 130, an exposure time calculation section 131, and a first imaging instruction section 132. The operation section 15 includes a region-of-interest setting section 150.

[0011] The imaging unit 11 captures an image based on light from a subject. The imaging optical system 110 forms an image of the light from the subject on the solid-state imaging element 112. Although only one lens is shown in FIG. 1, the imaging optical system 110 may include multiple lenses, and may include, for example, a zoom lens, a focus lens, and a blur correction lens. The aperture 111 adjusts the amount of light incident on the solid-state imaging element 112 through the imaging optical system 110. The solid-state imaging element 112 converts the light from the subject into an electrical signal for each pixel and outputs the signal. The solid-state imaging element 112 can set an exposure time for each divided area into which an image is divided. Note that the divided area can include one or more pixels. The solid-state imaging element 112 is, for example, a semiconductor element such as a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor, and a peripheral circuit.

[0012] The image processing unit 12 performs A / D conversion on the image signal sent from the solid-state imaging element 112 to convert it into digital data, and then performs signal processing including demosaicing processing, white balance processing, gamma processing, etc. to generate a digital image.

[0013] The imaging control unit 13 drives the imaging optical system 110 for zooming and focusing, controls the aperture diameter of the diaphragm 111, and controls the gain and exposure time for each region of the solid-state imaging element 112. At this time, the imaging control unit 13 can calculate an exposure time to be uniformly applied to a plurality of divided regions included in the image based on the exposure time for each divided region into which the image is divided, and perform imaging. The exposure time for each divided region into which the image is divided may be the exposure time for each divided region for all divided regions included in the image, or may be the exposure time for each divided region for a divided region including a region of interest set in the image. The plurality of divided regions is, for example, the entire image. For example, the imaging control unit 13 can calculate an exposure time to be applied to the entire image based on the distribution of the exposure time for each divided region into which the image is divided. At this time, the imaging control unit 13 may calculate an exposure time to be applied to the entire image based on the median value of the exposure time for each divided region into which the image is divided. In addition, when the number of data is even, the median may be the value of either one of the two data close to the center, or the average value of the two data close to the center. The imaging control unit 13 may use the mode of the exposure time for each divided region instead of the median of the exposure time for each divided region.

[0014] The first imaging instruction unit 130 instructs the imaging unit 11 to capture an image so as to obtain a first image captured based on an exposure time set for each divided region of the image. The exposure time calculation unit 131 calculates an exposure time to be uniformly applied to the multiple divided regions included in the image based on the exposure time for each divided region. The second imaging instruction unit 132 instructs the imaging unit 11 to capture an image so as to obtain a second image captured based on the exposure time calculated by the exposure time calculation unit 131.

[0015] Here, the first image can be used as, for example, a surveillance image. The second image can be used as, for example, an evidence image, an ornamental image, or a thumbnail image. In this case, by using the first image as a surveillance image, the dynamic range of the surveillance image can be improved, and the surveillance performance of illegal dumping at night can be improved. In addition, by using the second image as an evidence image, a natural image without any sense of incongruity can be submitted as evidence material, and the admissibility of evidence can be ensured.

[0016] The storage unit 14 stores the first image and the second image generated by the image processing unit 12. The storage unit 14 may include a volatile memory such as a RAM, or may include a non-volatile memory such as a NAND flash memory.

[0017] The operation unit 15 inputs instructions 19 from a user. The operation unit 15 is, for example, a button, a mode dial, or a touch panel attached to the display unit 16. The operation unit 15 also provides the user with an operating environment for setting an attention area. The attention area setting unit 150 sets the attention area of ​​the image based on the instruction 19 from the user.

[0018] The display unit 16 displays the first image and the second image generated by the image processing unit 12. It also displays an attention area set based on an instruction 19 from a user. The display unit 16 is, for example, a display configured with a CRT (Cathode Ray Tube), liquid crystal, organic EL (Electro Luminescence), or the like. The display unit 16 may be a touch screen or a touch panel.

[0019] The communication unit 17 distributes the first image and the second image generated by the image processing unit 12. At this time, the communication unit 17 converts the captured image from the image processing unit 12 in accordance with a network protocol and distributes it to the network 18.

[0020] The network 18 is, for example, a local area network (LAN), and includes a router or the like that satisfies a communication standard such as Ethernet (registered trademark) etc. The imaging device 10 and a client device (not shown) are connected to the network 18 by a LAN cable or the like.

[0021] FIG. 2 is a flowchart showing the imaging control method according to the first embodiment. Each step in FIG. 2 is realized by the imaging control unit 13 reading and executing a program stored in the storage unit 14 of the imaging device 10 in FIG. 1. At least a part of the flowchart shown in FIG. 2 may be realized by hardware. When realized by hardware, for example, a specific compiler may be used to automatically generate a dedicated circuit on an FPGA from a program for realizing each step. Also, a gate array circuit may be formed in the same manner as an FPGA to realize the hardware. Also, it may be realized by an ASIC. In this case, each block in the flowchart shown in Fig. 2 can be regarded as a hardware block. Note that multiple blocks may be integrated into one hardware block, and one block may be configured as multiple hardware blocks.

[0022] 2, the imaging control unit 13 causes the solid-state imaging element 112 to capture a first image in which the exposure time is set so that the amount of exposure is optimized for each divided region, and acquires the first image captured by the solid-state imaging element 112 (S21). At this time, the imaging control unit 13 can set exposure time information for each divided region.

[0023] Next, the imaging control unit 13 determines whether or not an attention area has been set by the attention area setting unit 150 (S22). If an attention area has been set, the imaging control unit 13 calculates a median value of the exposure time of the attention area from the exposure time information of each divided area of ​​the first image set in S21 (S24). On the other hand, if an attention area has not been set, the imaging control unit 13 calculates a median value from the exposure time information of each divided area of ​​the first image set in S21 (S23).

[0024] Next, the imaging control unit 13 uniformly sets the exposure time calculated in S23 or S24 for each divided area of ​​the solid-state imaging element 112. Then, the imaging control unit 13 acquires a second image captured by the solid-state imaging element 112 (S25).

[0025] Figure 3(a) is a diagram showing an example of a first image in the first embodiment, Figure 3(b) is a diagram showing an example of a second image when a focus area RA is set in the first image, and Figure 3(c) is a diagram showing an example of a second image when a focus area RA is not set in the first image.

[0026] In Fig. 3(a), the first image G1 is divided into, for example, vertical x horizontal = 3 x 4 = 12 divided areas A1 to A3, B1 to B3, C1 to C3, and D1 to D3. In this case, the imaging control unit 13 in Fig. 1 can set the exposure time individually for each of the divided areas A1 to A3, B1 to B3, C1 to C3, and D1 to D3. Also, an attention area RA can be set in the first image G1. In the example of Fig. 3(a), an example is shown in which the attention area RA is included in the divided areas B2, C2, and D2.

[0027] Here, by setting the exposure time individually for each of the divided areas A1 to A3, B1 to B3, C1 to C3, and D1 to D3, a first image G1 in which gradation loss is suppressed can be obtained even in an environment with a wide dynamic range of brightness where areas of high and low illuminance are mixed. However, if the exposure time is set individually for each of the divided regions A1-A3, B1-B3, C1-C3, and D1-D3 so that the exposure amount is optimized, block-shaped brightness differences (hereinafter referred to as block differences) occur at the boundaries between the regions with different exposure times. For this reason, block differences BD occur in the first image G1, which may create a sense of incongruity compared to when the entire image is captured with a uniform exposure time.

[0028] When an attention area RA is set in the first image G1, the imaging control unit 13 calculates an exposure time to be applied to the entire image based on the median of the exposure times for the divided areas B2, C2, and D2 including the attention area RA. Then, the imaging control unit 13 applies the exposure time to be applied to the entire image uniformly to each of the divided areas A1 to A3, B1 to B3, C1 to C3, and D1 to D3, and causes the solid-state imaging element 112 to capture the image. As a result, as shown in Fig. 3(b), the block step BD in Fig. 3(a) is eliminated, and a second image G2A in which the exposure amount of the attention area RA is optimized is obtained.

[0029] When the attention area RA is not set in the first image G1, the imaging control unit 13 calculates an exposure time to be applied to the entire image based on the median of the exposure times for each of the divided areas A1-A3, B1-B3, C1-C3, and D1-D3 of the first image G1. Then, the imaging control unit 13 uniformly applies the exposure time to be applied to the entire image to each of the divided areas A1-A3, B1-B3, C1-C3, and D1-D3 and causes the solid-state imaging element 112 to capture the image. As a result, as shown in FIG. 3(c), a second image G2B in which the block step BD in FIG. 3(a) is eliminated is obtained.

[0030] Here, the imaging control unit 13 can use the median of the exposure times for each divided area of ​​the first image as the exposure time of the second image. This allows the exposure amount of the entire image to be optimized compared to when the average value of the exposure times of the divided areas of the first image is used as the exposure time of the second image. For example, when only one divided area among the multiple divided areas is darker than the other divided areas, if the average value of the exposure times of the divided areas of the first image is used as the exposure time of the second image, the brightness of the entire image is attracted to the brightness of the dark divided area, and the brightness of the entire image becomes dark. In contrast, when the median value of the exposure times of the divided areas of the first image is used as the exposure time of the second image, it is possible to prevent the brightness of the entire image from becoming dark even when only one divided area among the multiple divided areas is darker than the other divided areas.

[0031] FIG. 4 is a diagram showing an example of exposure time settings for each divided area according to the first embodiment. FIG. 4(a) shows an example of exposure time for each divided area A1-A3 of the first image G1 in FIG. 3(a). FIG. 4(b) shows an example of exposure time for each divided area B1-B3 of the first image G1 in FIG. 3(a). FIG. 4(c) shows an example of exposure time for each divided area C1-C3 of the first image G1 in FIG. 3(a). FIG. 4(d) shows an example of exposure time for each divided area D1-D3 of the first image G1 in FIG. 3(a).

[0032] 4(a) to 4(d), when acquiring the first image, the exposure time is set individually for each of the divided regions A1 to A3, B1 to B3, C1 to C3, and D1 to D3 so that the exposure amount is optimized. For example, as shown in FIG. 4(a), the exposure time for each of the divided regions A1 and A3 is set to 1 / 30 seconds, and the exposure time for the divided region A2 is set to 1 / 300 seconds.

[0033] Fig. 5 is a diagram showing an example of calculation of the exposure time of the second image according to the first embodiment. Fig. 5(a) shows an example of the median value when the attention area RA is set in the first image G1 in Fig. 3(a), and Fig. 5(b) shows an example of the median value when the attention area RA is not set in the first image G1 in Fig. 3(a).

[0034] In Fig. 5(a), when the attention area RA is set in the first image G1 in Fig. 3(a), the imaging control unit 13 calculates a median value from the exposure time for each of the divided areas B2, C2, and D2 including the attention area RA. In the example of Fig. 5(a), the median value of the exposure time for each of the divided areas B2, C2, and D2 is 1 / 500 seconds, which is the exposure time for the divided area D2. Here, when the attention area RA is set in the first image G1, by calculating the median value from the exposure time for each of the divided areas B2, C2, and D2 including the attention area RA, the exposure amount of the attention area RA can be optimized compared to the case where the median value is calculated from the entire image.

[0035] In Fig. 5(b), when the attention area RA is not set in the first image G1 in Fig. 3(a), the imaging control unit 13 calculates a median value from the exposure times of the divided areas A1-A3, B1-B3, C1-C3, and D1-D3 included in the first image G1. In the example of Fig. 5(b), the median value of the exposure times of the divided areas A1-A3, B1-B3, C1-C3, and D1-D3 is 1 / 50 seconds, which is the exposure time of the divided area B3.

[0036] As described above, according to the first embodiment, the imaging device acquires a first image in which an exposure time is set for each divided region, thereby enabling imaging with appropriate exposure even in an environment in which high and low illuminance regions coexist. In addition, the imaging device acquires a second image in which the sense of incongruity is reduced by setting the exposure time for the entire image based on the median value of the exposure time calculated from the exposure time information of the first image.

[0037] <Other embodiments> FIG. 6 is a block diagram showing an example of a hardware configuration of a control processing unit applied to the imaging apparatus of FIG. 6, the control processing unit 30 can realize functions realized by software on the imaging device 10 among the functions of the imaging device 10 in Fig. 1. At this time, the control processing unit 30 can realize the functions of the image processing unit 12 and the imaging control unit 13 in Fig. 1. In addition, the control processing unit 30 can cause the imaging unit 11, the image processing unit 12, the imaging control unit 13, the storage unit 14, the operation unit 15, the display unit 16, and the communication unit 17 in Fig. 1 to operate in cooperation with each other.

[0038] The control processing unit 30 includes a processor 31, a communication control unit 32, a communication interface 33, a main memory unit 34, an auxiliary memory unit 35, and an input / output interface 37. The processor 31, the communication control unit 32, the communication interface 33, the main memory unit 34, the auxiliary memory unit 35, and the input / output interface 37 are connected to each other via an internal bus 36. The main memory unit 34 and the auxiliary memory unit 35 are accessible from the processor 31.

[0039] In addition, an input device 40, a display device 41, and an image sensor 42 are provided outside the control processing unit 30. The input device 40, the display device 41, and the image sensor 42 are connected to the internal bus 36 via an input / output interface 37. The input device 40 is, for example, a keyboard, a mouse, a touch panel, a card reader, a voice input device, etc. The display device 41 is, for example, a liquid crystal monitor, an organic EL display, or a micro LED display. The image sensor 42 is, for example, a CCD sensor or a CMOS sensor.

[0040] The processor 31 controls the overall operation of the control processing unit 30. The processor 31 may be a CPU or a GPU (Graphics Processing Unit). The processor 31 may be a single-core processor or a multi-core processor. The processor 31 may include a hardware circuit (for example, an FPGA or ASIC) such as an accelerator that speeds up part of the processing.

[0041] The main memory unit 34 can be composed of, for example, a semiconductor memory such as a static random access memory (SRAM) or a dynamic random access memory (DRAM). The main memory unit 34 can store a program being executed by the processor 31 and can provide a work area for the processor 31 to execute the program.

[0042] The auxiliary storage unit 35 is a non-volatile storage device, such as a ROM, a hard disk device, or an SSD (Solid State Drive). The auxiliary storage unit 35 can hold executable files of various programs and data used to execute the programs.

[0043] The communication control unit 32 is hardware having a function of controlling communication with the outside. The communication control unit 32 is connected to a network 39 via a communication interface 33. The network 39 may be the Internet, a WAN, a LAN such as WiFi or Ethernet (registered trademark), or a mixture of the Internet, a WAN, and a LAN.

[0044] The input / output interface 37 converts data input from the input device 40 and the image sensor 42 into a data format that can be processed by the processor 31, and converts data output from the processor 31 into a data format that can be processed by the display device 41.

[0045] The processor 31 can realize the processing of FIG. 2 by reading out a program stored in the auxiliary storage unit 35 into the main storage unit 34 and executing it. Note that execution of the program for realizing the processing of Fig. 2 may be shared among multiple processors or computers. Alternatively, processor 31 may instruct a cloud computer or the like to execute all or part of the program for realizing the processing of Fig. 2 via network 39, and receive the execution results.

[0046] The present invention may be implemented by supplying a program for implementing one or more of the functions of the above-described embodiments to a system or device via a network or a storage medium. One or more of the functions of the above-described embodiments may also be implemented by a process in which one or more processors in a computer of the system or device read and execute the program. Alternatively, one or more of the functions may also be implemented by a circuit (e.g., an FPGA or an ASIC) that implements the functions.

[0047] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. For example, in the above-mentioned embodiment, an example of acquiring a first image and a second image has been shown, but the imaging device 10 may also acquire images other than the first image and the second image, such as an image in which only a region of interest is cut out. Alternatively, a correction means may be provided to generate a third image corrected so as to reduce the luminance difference at the boundary between adjacent regions in the first image, and either or both of the second image and the third image may be acquired together with the first image.

[0048] The present invention can be applied to both still image capture and video capture. In particular, in video capture, the image capture control unit 13 may cause the image capture unit 11 to capture only the first image at a frame rate equal to or higher than a set value, and may cause the image capture unit 11 to capture the first and second images at a frame rate lower than the set value. For example, when the image capture is performed at the maximum frame rate at which the solid-state image capture element 112 can operate so as not to exceed the maximum operating capacity of the solid-state image capture element 112, the image capture control unit 13 may cause only the first image to be captured. Also, when the image capture is performed at a frame rate lower than the maximum frame rate at which the solid-state image capture element 112 can operate, the image capture control unit 13 may cause both the first and second images to be captured. This allows both the first and second images to be captured when there is a margin for the load on the processor 31 in FIG. 6, and prevents the processing of the processor 31 from being delayed. [Explanation of symbols]

[0049] REFERENCE SIGNS LIST 10 imaging device, 11 imaging section, 12 image processing section, 13 imaging control section, 14 storage section, 15 operation section, 16 display section, 17 communication section, 110 imaging optical system, 111 aperture, 112 solid-state imaging element

Claims

1. An imaging control device for controlling an imaging means having an imaging element capable of setting an exposure time for each of a plurality of divided regions obtained by dividing an image, a first instruction means for instructing the imaging means to acquire a first image using an exposure time set for each of the plurality of divided regions; a calculation means for calculating an exposure time to be uniformly applied to the plurality of divided regions based on an exposure time for each of the divided regions; a second instruction means for instructing the imaging means to uniformly apply the exposure time calculated by the calculation means to all of the plurality of divided regions to obtain a second image, The imaging control device is characterized in that the calculation means calculates an exposure time to be uniformly applied to all of the multiple divided areas based on the distribution of exposure times for each of the multiple divided areas set when acquiring the first image.

2. 2. The imaging control device according to claim 1, wherein the calculation means calculates an exposure time to be uniformly applied to all of the multiple divided regions included in the image based on a median value in a distribution of exposure times for each of the multiple divided regions.

3. 3. The imaging control device according to claim 1, wherein the first image is used as a monitoring image.

4. 4. The imaging control device according to claim 1, wherein the second image is used as a thumbnail image.

5. 5. The imaging control device according to claim 1, further comprising a storage unit for storing the first image and the second image.

6. The imaging control device according to claim 1 , further comprising a communication unit that distributes the first image and the second image.

7. When a setting value of a frame rate in imaging by the imaging means is equal to or higher than a predetermined setting value, the imaging means is caused to capture only the first image, 7. The imaging control device according to claim 1, wherein, when a set value of a frame rate for imaging by the imaging means is smaller than a predetermined set value, the imaging means is caused to capture the first image and the second image.

8. An imaging control method for controlling an imaging means having an imaging element capable of setting an exposure time for each of a plurality of divided regions obtained by dividing an image, comprising: instructing the imaging means to capture a first image using an exposure time set for each of the plurality of divided regions; calculating an exposure time to be uniformly applied to the plurality of divided regions based on an exposure time for each of the divided regions; instructing the imaging means to capture a second image by uniformly applying the calculated exposure time to all of the plurality of divided regions; An imaging control method, characterized in that an exposure time to be uniformly applied to all of the multiple divided areas is calculated based on the distribution of exposure times for each of the multiple divided areas set when acquiring the first image.

9. A program for causing a computer to operate as the imaging control device according to any one of claims 1 to 7.

Images