Imaging control device, imaging control method, and program

Abstract

To enable proper exposure control.SOLUTION: An image control device (103) has a detection means (406) for detecting a specific subject area in an image sent from an imaging apparatus (101), an acquisition means (404) for obtaining an exposure setting value of the imaging apparatus (101), determination means (407) for determining an exposure amount for the subject area on the basis of the exposure setting value of the imaging apparatus (101) and the information on the subject area, and control means (407, 402) for controlling exposure of the imaging apparatus (101) on the basis of an exposure amount determined by the determination means. The control means (407, 402) performs exposure control of the imaging apparatus (101) according to at least a first exposure correction method and a second exposure correction method.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a technique for controlling an imaging device. [Background technology]

[0002] In recent years, a technology has been proposed for imaging devices such as surveillance cameras, digital cameras, and video cameras that automatically detects a specific area such as a subject from a captured image and adjusts shooting settings such as exposure and image quality based on information related to the detected area. Also, a technology is known in which an evaluation device separate from the imaging device determines shooting conditions suitable for various applications based on images obtained from the imaging device and sets the imaging device with the imaging conditions, thereby making it possible to arbitrarily change the images output by the imaging device.

[0003] Patent Document 1 discloses a technique for performing degamma processing on the evaluation device side, which corresponds to the gamma processing performed on the imaging device side, to obtain an image signal before gamma processing on the imaging device, and to determine the exposure state of the imaging device based on the image signal. Patent Document 2 discloses a technique for performing degamma processing to obtain an appropriate exposure determination value that is not dependent on processing between devices in order to fill in the difference between devices when the processing device and the evaluation device are different. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 4-165876 [Patent Document 2] JP 2007-102284 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, with the techniques described in Patent Documents 1 and 2, for example, there are cases where it is not possible to perform appropriate exposure control on the imaging device so as to obtain an image in which the subject in question has a proper exposure.

[0006] Therefore, an object of the present invention is to enable appropriate exposure control. [Means for solving the problem]

[0007] The imaging control device of the present invention comprises a detection means for detecting a specific subject area in an image sent from an imaging device, an acquisition means for acquiring an exposure setting value of the imaging device, a determination means for determining an exposure amount of the subject area based on the exposure setting value of the imaging device and information on the subject area, and a control means for controlling the exposure of the imaging device based on the exposure amount determined by the determination means, wherein the control means When the exposure amount determined by the determining means is greater than a predetermined amount, a coarse adjustment is performed to adjust the exposure of the imaging device at a first speed until the exposure amount reaches the predetermined amount. a first exposure compensation method; After the predetermined amount is reached, the exposure of the imaging device is adjusted at a second speed slower than the first speed. The present invention is characterized in that exposure control of the imaging device is performed by a second exposure correction method. Effect of the Invention

[0008] According to the present invention, appropriate exposure control becomes possible. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of an imaging control system. [Diagram 2] FIG. 1 is a diagram illustrating an example of the internal configuration of a surveillance camera. [Diagram 3] FIG. 2 illustrates an example of the internal configuration of a client device. [Figure 4] FIG. 2 illustrates an example of a functional configuration of a client device. [Diagram 5] FIG. 2 is a diagram used to explain processing between a camera and a client device. [Figure 6] FIG. 6 is a diagram used to explain the luminance state in the example of FIG. 5. [Figure 7] 4 is a flowchart of an exposure control process according to the first embodiment. [Figure 8] 11A and 11B are diagrams used to explain a process of changing exposure control. [Figure 9]11A and 11B are diagrams used to explain processing when exposure control is performed according to an embodiment. [Figure 10] 10 is a flowchart of an exposure control process according to a second embodiment. [Figure 11] FIG. 2 is a diagram showing the characteristics of gamma processing and degamma processing. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. First, common matters in explaining the present embodiment will be explained, and then a detailed explanation will be given. Note that the following embodiment does not limit the present invention, and not all of the combinations of features explained in the present embodiment are necessarily essential for the solution of the present invention. The configuration of the embodiment may be appropriately modified or changed depending on the specifications of the device to which the present invention is applied and various conditions (conditions of use, environment of use, etc.). In addition, the embodiment may be configured by appropriately combining parts of each embodiment described later. In each of the following embodiments, the same configurations are explained with the same symbols.

[0011] One or more of the functional blocks described below may be realized by hardware such as an ASIC or a programmable logic array (PLA), or may be realized by a programmable processor such as a CPU or an MPU executing software. They may also be realized by a combination of software and hardware. Therefore, even when different functional blocks are described as operating subjects in the following description, they may be realized by the same hardware as the main body. Note that ASIC is an abbreviation for Application Specific Integrated Circuit. CPU is an abbreviation for Central Processing Unit. MPU is an abbreviation for Micro-Processing Unit.

[0012] FIG. 1 is a diagram showing an example of the configuration of an imaging control system 100 according to this embodiment. The imaging control system 100 includes a surveillance camera 101, a network 102, a client device 103, an input device 104, and a display device 105. The surveillance camera 101 is an imaging device for acquiring moving images, and is capable of imaging a subject and processing the images. The surveillance camera 101 and the client device 103 are connected to each other via the network 102 in a state in which they can communicate with each other. The client device 103 is connected to the input device 104 and the display device 105 in a state in which they can communicate with each other. The client device 103 is a device that processes various information, and therefore may be called an information processing device. In addition, the client device 103 is a device that controls the imaging of the surveillance camera 101, and therefore may be called an imaging control device.

[0013] The input device 104 is composed of a mouse, a keyboard, and the like, and is operated by the user of the client device 103 . The display device 105 is a device equipped with a monitor or the like for displaying an image received from the client device 103. The display device 105 can also function as a UI such as a touch panel. In this case, the display device 105 can also function as an input device for inputting instructions, information, data, and the like to the client device 103. UI is an abbreviation for User Interface.

[0014] In FIG. 1, the client device 103, the input device 104, and the display device 105 are depicted as independent devices, but the present embodiment is not limited to such a configuration. For example, the client device 103 and the display device 105 may be integrated, or the input device 104 and the display device 105 may be integrated. Also, the client device 103, the input device 104, and the display device 105 may be integrated. When the client device 103 and the display device 105 are integrated, the integrated device may be in the form of, for example, a personal computer, a tablet terminal, or a smartphone.

[0015] FIG. 2 is a block diagram showing an example of the internal configuration of the surveillance camera 101. The surveillance camera 101 has an imaging optical system 201 and an imaging element 202 as components for capturing an image. The surveillance camera 101 further has a camera CPU 203, ROM 204, RAM 205, an imaging system control unit 206, a control unit 207, an A / D conversion unit 208, an image processing unit 209, an encoder unit 210, and a network I / F 211. The camera CPU 203 to the network I / F 211 of the surveillance camera 101 are interconnected by a system bus 212. Note that CPU is an abbreviation for Central Processing Unit. ROM is an abbreviation for Read Only Memory. RAM is an abbreviation for Random Access Memory. A / D is an abbreviation for Analog / Digital. I / F is an abbreviation for Interface.

[0016] The imaging optical system 201 is a group of optical members that collects light from a subject, and is composed of a zoom lens, a focus lens, a blur correction lens, an aperture, a shutter, etc. The optical image of the subject etc. collected by the imaging optical system 201 is formed on the imaging surface of the image sensor 202. The imaging element 202 is a charge-accumulation type solid-state imaging element such as a CMOS or CCD that converts the light image collected by the imaging optical system 201 into a current value (signal value), and is an imaging unit that acquires color information by combining with a color filter or the like. CMOS is an abbreviation for Complementary Metal Oxide Semiconductor. CCD is an abbreviation for Charge-Coupled Device. The imaging element 202 is connected to an A / D conversion unit 208.

[0017] The A / D conversion unit 208 converts the amount of light of the subject detected by the image sensor 202 into a digital signal (image data). The A / D conversion unit 208 transmits the digital signal to the image processing unit 209. The image processing unit 209 performs image processing on the image data of the digital signal received from the imaging element 202. The image processing unit 209 is connected to an encoder unit 210. The encoder unit 210 converts the image data processed by the image processing unit 209 into a file format such as Motion JPEG, H264, H265, etc. The encoder unit 210 is connected to a network I / F 211.

[0018] The camera CPU 203 is a control unit that performs overall control of the operation of the security camera 101. The camera CPU 203 reads instructions stored in the ROM 204 or RAM 205, and executes processing according to the results. The imaging system control unit 206 controls each unit of the surveillance camera 101 based on instructions from the camera CPU 203. For example, the imaging system control unit 206 controls the imaging optical system 201 in terms of focus control, shutter control, aperture adjustment, and the like.

[0019] The network I / F 211 is an interface used for communication with external devices such as the client device 103 via the network 102 , and is controlled by the control unit 207 . The control unit 207 controls communication with the client device 103, and performs control such as transmitting control commands (control signals) from the client device 103 to each unit of the security camera 101 to the camera CPU 203.

[0020] The network 102 is an IP network that connects the surveillance camera 101 and the client device 103. The network 102 is composed of a plurality of routers, switches, cables, etc. that comply with a communication standard such as Ethernet (registered trademark). In this embodiment, the network 102 may be any network that can perform communication between the surveillance camera 101 and the client device 103, and the communication standard, scale, configuration, etc. are not important. For example, the network 102 may be composed of the Internet, a wired LAN (Local Area Network), a wireless LAN (Wireless LAN), a WAN (Wide Area Network), etc.

[0021] FIG. 3 is a block diagram showing an example of the internal configuration of the client device 103. As shown in FIG. The client device 103 includes a client CPU 301, a main memory device 302, an auxiliary memory device 303, an input I / F 304, an output I / F 305, and a network I / F 306. The elements of the client device 103 are connected to each other via a system bus 307 so as to be able to communicate with each other.

[0022] The client CPU 301 is a central processing unit that performs overall control of the operation of the client device 103. Note that the client CPU 301 may also perform overall control of the security camera 101 via the network . The main memory device 302 is a memory device such as a RAM that functions as a temporary storage location for data of the client CPU 301 .

[0023] The auxiliary storage device 303 is a storage device such as an HDD, a ROM, or an SSD that stores various programs, various setting data, and the like. Note that HDD is an abbreviation for Hard Disk Drive. SSD is an abbreviation for Solid State Drive. The program according to this embodiment is stored in the auxiliary storage device 303. In this embodiment, the client CPU 301 executes processing based on the program read from the auxiliary storage device 303 and deployed in the main storage device 302, thereby realizing the functions and processing of the client device 103 shown in FIG. 4. Details of this will be described later. In addition, the auxiliary storage device 303 may be pre-stored with, for example, a pattern for pattern matching (a pattern corresponding to a facial feature or a human body feature) that is used when the client device 103 performs face detection or human body detection based on image data. Note that these pattern matching patterns may be formed by executing a program and stored in the main storage device 302.

[0024] The input I / F 304 is an interface that is used when the client device 103 receives an input (signal) from the input device 104 or the like. The output I / F 305 is an interface that is used when outputting information (signals) from the client device 103 to the display device 105 or the like. The network I / F 306 is an interface used for communication with external devices such as the surveillance camera 101 via the network 102 .

[0025] Fig. 4 is a functional block diagram showing each function executed by the client device 103. In other words, each functional unit (functional block) shown in Fig. 4 is a functional unit that can be executed by the client CPU 301, and each of these functional units is synonymous with the client CPU 301. 4, client CPU 301 of client device 103 includes the functional units of input signal acquisition unit 401, communication control unit 402, input image acquisition unit 403, camera information acquisition unit 404, and detection method setting unit 405. Client CPU 301 also includes the functional units of subject detection unit 406, exposure determination unit 407, and display control unit 408. Note that in client device 103, the functional units from input signal acquisition unit 401 to display control unit 408 shown in FIG. 4 may be configured by hardware (or software) separate from client CPU 301.

[0026] The input signal acquisition unit 401 receives an input from the user via the input device 104 . The communication control unit 402 executes control for receiving images transmitted from the surveillance camera 101 (images captured by the surveillance camera 101) via the network 102. The communication control unit 402 also executes control for transmitting control commands from the client device 103 to the surveillance camera 101 via the network 102.

[0027] The input image acquisition unit 403 acquires an image received from the surveillance camera 101 via the communication control unit 402 as an input image (an image to which the subject detection process is applied) that is a target of the subject detection process. Details of the detection process will be described later. The camera information acquisition unit 404 acquires camera information when the surveillance camera 101 captures an image of a subject or the like via the communication control unit 402. The camera information is various camera setting information and image processing information when capturing an image of a subject or the like and acquiring the image. Specifically, the camera information includes, for example, exposure parameters (exposure-related setting values) such as aperture, shutter speed, and gain, and information on image processing related to brightness such as gamma correction, edge emphasis, and white balance.

[0028] The detection method setting unit 405 sets a predetermined (appropriate) detection method from among various detection methods including detection of a face area (face detection) and detection of a human body area (human body detection) for the input image acquired by the input image acquisition unit 403. In this embodiment, the detection method setting unit 405 sets (selects) the detection method of face detection or the detection method of human body detection.

[0029] The subject detection unit 406 detects a specific subject area in an input image captured by the surveillance camera 101 and acquired by the input image acquisition unit 403. For example, when face detection is set in the detection method setting unit 405, the subject detection unit 406 detects a face area from the input image with priority. Also, when human body detection is set in the detection method setting unit 405, the subject detection unit 406 detects a human body area from the input image with priority.

[0030] Note that this embodiment is not limited to such settings. For example, a detection method for detecting a characteristic area of ​​a part of a person, such as the upper body, head, and some organs such as the eyes, nose, and mouth of the face, may be set (selected). In addition, in this embodiment, the specific subject to be detected is a person, but a configuration capable of detecting a characteristic area related to a specific subject other than a person may be used. For example, a configuration capable of detecting a specific subject preset in the client device 103, such as an animal's face or a car, may be used.

[0031] The exposure determination unit 407 has a function of determining the exposure amount of the subject area based on the exposure setting value of the surveillance camera 101 acquired by the camera information acquisition unit 404 and the image information of the subject area detected by the subject detection unit 406. Furthermore, the exposure determination unit 407 also has a function of controlling the exposure of the surveillance camera 101 based on the determined exposure amount. The exposure control of the surveillance camera 101 in the exposure determination unit 407 is performed by sending an exposure control value based on the determined exposure amount to the surveillance camera 101 via the communication control unit 402. That is, the exposure determination unit 407 calculates an exposure correction amount indicating the amount of change in exposure required to make the subject area properly exposed based on the exposure setting value of the surveillance camera 101 and the image information of the subject area, and sends an exposure control value according to the exposure correction amount to the communication control unit 402. Then, the communication control unit 402 transmits a control command according to the exposure control value (exposure correction amount) to the surveillance camera 101 via the network I / F 306. As a result, in the surveillance camera 101 that has received the control command, the control unit 207 and the imaging system control unit 206 execute exposure control.

[0032] In this embodiment, the exposure determination unit 407 performs exposure control using at least a first exposure correction method and a second exposure correction method. Although details will be described later, the exposure determination unit 407 performs exposure control using a first exposure correction method, which is a coarse adjustment method that changes the exposure of the surveillance camera at once up to a predetermined amount, and thereafter performs exposure control using a second exposure correction method that finely adjusts the exposure of the surveillance camera. In particular, when the calculated exposure correction amount is greater than the predetermined amount (when the change amount of exposure is large), the exposure determination unit 407 performs a coarse adjustment method that changes the exposure at once up to the predetermined amount, and thereafter performs fine adjustment. On the other hand, when the calculated exposure correction amount is equal to or less than the predetermined amount, the exposure determination unit 407 changes the exposure at once to the exposure correction amount during the coarse adjustment period. Detailed processing in the exposure determination unit 407 will be described later using the flowchart of FIG. 7 and the like.

[0033] The display control unit 408 , in accordance with an instruction from the client CPU 301 , outputs to the display device 105 the captured image in which the exposure compensation based on the exposure compensation amount determined by the exposure determination unit 407 is reflected.

[0034] In the above configuration, when the client device 103 controls the exposure of the surveillance camera 101, it is necessary to perform, for example, degamma processing as described above to correctly determine the brightness of the image output by the image sensor 202 of the surveillance camera 101. However, since the surveillance camera 101 and the client device 103 have different methods for determining the exposure state, it is necessary to take into consideration the difference in the determination methods.

[0035] In the following description, for example, it is assumed that the surveillance camera uses the entire image as a photometric area and determines the amount of exposure based on the luminance obtained from the entire image. On the other hand, it is assumed that the client device uses a subject area, such as a person detected from an image, as a photometric area and performs exposure determination based on luminance information obtained from the subject area. That is, the surveillance camera determines the amount of exposure using the entire image as an evaluation area for exposure setting, whereas the client device performs exposure determination using the subject area in the image as an evaluation area for exposure setting. In this embodiment, a scene is assumed that aims to enable shooting with an exposure that provides appropriate brightness for the subject area when the evaluation areas for exposure state determination are different between the surveillance camera 101 and the client device 103.

[0036] Fig. 5 is a diagram for explaining the process exchange between a surveillance camera and a client device, showing a specific example of a case where the exposure control of the camera is not performed well. Fig. 6(a) and Fig. 6(b) are diagrams used to explain the state of the brightness of an image in each state of Fig. 5. Fig. 6(a) is a histogram showing the relationship between the brightness value and the number of pixels in an image, and Fig. 6(b) is a diagram showing the average brightness value in an image.

[0037] Image 501 shown in Fig. 5 shows an image when a surveillance camera first captures a subject, and shows an example of an image captured in a backlit scene. In a backlit scene, the area of ​​the subject, that is, the person, is generally dark and the surroundings are bright. Even in such a case, in order to perform more accurate person recognition, it is desirable to set the exposure state so that the brightness of the area of ​​the person is appropriate.

[0038] Graph 601 in Fig. 6 shows the distribution of luminance in a brightness state like that of image 501 in Fig. 5. In the case of image 501, since it is backlit, the image has a luminance distribution dominated by dark and bright areas, as shown in graph 601. Graph 602 in Fig. 6 shows this converted into the amount of brightness of the entire image, which is the photometry area (evaluation area for exposure determination) of the surveillance camera.

[0039] Also, image 502 in Fig. 5 shows an example of an image in which de-gamma correction is performed on the client device side to reproduce the brightness state captured by the actual image sensor so as not to be affected by the shooting conditions of the surveillance camera. At this time, the client device focuses on the person as a photometry area (evaluation area for exposure determination) and outputs brightness, and since the person is dark as shown in image 502, it tries to increase the exposure to an appropriate brightness. That is, as shown by arrow 603, the client device tries to adjust the current brightness of the person, shown by the dotted line in the figure, to an appropriate brightness, shown by the solid line in the figure. The process shown by arrow 603 is assumed to be a process of adjusting the exposure level so that the brightness is doubled from the current brightness before processing, as shown by graph 604.

[0040] If the exposure control of the surveillance camera is performed based on instructions from such a client device, overcorrection occurs, causing the human area to be blown out, as shown in image 503 in FIG. 5. For example, when controlling the exposure to double the brightness, the surveillance camera adjusts the brightness using the entire captured image as the evaluation area, so that the high-brightness areas reach a saturated state where they cannot be brighter any more. That is, the surveillance camera adjusts the exposure so that the brightness of the low-brightness areas shown by the dotted line is more than doubled, as shown by the arrow 605 in FIG. 6, and the overall brightness is doubled, as shown by the solid line. Graph 606 in FIG. 6 shows the brightness before and after the exposure adjustment process to more than double the brightness of such low-brightness areas and double the overall brightness.

[0041] As mentioned above, when the evaluation areas are different between the surveillance camera and the client device, for example, even if the brightness of a person is adjusted appropriately, overcorrection may occur. In other words, when the evaluation areas are different between the surveillance camera and the client device, there is a high possibility that appropriate exposure control cannot be performed.

[0042] First Embodiment In the first embodiment, a configuration and processing that enable appropriate exposure control to be performed even if the targets (evaluation areas) of exposure evaluation are different between the surveillance camera side and the client device side as described above will be described. Fig. 7 is a flowchart showing a flow from subject detection processing to exposure control processing executed by the client CPU 301 of the client device 103 according to the first embodiment. In the imaging control system 100 of Fig. 1, it is assumed that the powers of the surveillance camera 101, the client device 103, the input device 104, and the display device 105 are turned on, and a connection (communication) between the surveillance camera 101 and the client device 103 is established. It is also assumed that in this state, the surveillance camera 101 captures an image of a subject or the like at a predetermined update period, the surveillance camera 101 transmits image data to the client device 103, and the display device 105 displays the image. It is also assumed that the client CPU 301 starts the processing of the flowchart of Fig. 7 in response to input of a captured image of a subject or the like from the surveillance camera 101 to the client device 103 via the network 102.

[0043] First, in step S701, subject detection unit 406 executes a process of detecting a subject from an image sent from surveillance camera 101. In this embodiment, an example is given of detecting a human body or a face as the subject, and therefore, prior to the subject detection process, detection method setting unit 405 sets a face or human body detection method for subject detection unit 406. Then, subject detection unit 406 performs face detection process or human body detection process on the input image according to the setting by detection method setting unit 405. Patterns corresponding to facial features and human body features, respectively, are stored in advance in auxiliary storage device 303 of client device 103, and subject detection unit 406 detects a face region or a human body region by pattern matching based on the patterns.

[0044] In addition, when detecting a face region, it is generally possible to detect the face with high accuracy and clearly distinguish the face region of the subject from the non-face region. However, if the face direction, face size, face brightness, etc. are not suitable conditions for face detection, the face region may not be detected accurately. In contrast, when detecting a human body, it is possible to detect an area where a person exists regardless of the face direction, face size, face brightness, etc. In addition, the human body detection in this embodiment does not necessarily need to detect the entire body, and may detect the upper body, bust up, or head region including the face.

[0045] Furthermore, when a pattern matching method is adopted as a method for detecting a subject, a pattern (classifier) ​​created using statistical learning may be used as a pattern used in pattern matching. Alternatively, subject detection may be performed using a method other than pattern matching. For example, subject detection may be performed using a luminance gradient in a local region of an image. In other words, the method for detecting a subject is not limited to a specific detection method, and various methods such as detection based on machine learning and detection based on distance information may be adopted.

[0046] Next, in step S702, the subject detection unit 406 determines whether a subject (face area or human body area) has been detected in the image by the subject detection process executed in step S701. If at least one subject has been detected, the process of the client CPU 301 proceeds to step S703, whereas if no subject has been detected, this process ends.

[0047] In step S703, the exposure determination unit 407 measures the brightness (luminance) of the subject area. Next, in step S704, the exposure determination unit 407 acquires the current exposure value acquired by the camera information acquisition unit 404 from the surveillance camera 101 (the current exposure setting value of the surveillance camera). Next, in step S705, the exposure determination unit 407 calculates an exposure correction amount that represents the amount of change in exposure required to make the subject area properly exposed, based on the brightness information (luminance information) of the subject areas and the current exposure setting value of the surveillance camera.

[0048] Next, in step S706, the exposure determination unit 407 judges whether the exposure correction amount (amount of change in exposure) is equal to or less than a predetermined amount. If the exposure correction amount is equal to or less than the predetermined amount in step S706, the process of the client CPU 301 proceeds to step S707, whereas if it exceeds the predetermined amount, the process proceeds to step S708. That is, in step S706, a process is performed to check the extent of the exposure correction amount representing the amount of change in exposure. This is because, as described above, when the evaluation area (photometric area) is different between the monitoring camera and the client device, the exposure correction amount calculated on the client device side is likely to be a correction amount that is significantly different from the actual situation. Therefore, when the exposure correction amount is greater than the predetermined amount (when the amount of change in exposure is large), the exposure is not changed all at once by the calculated exposure correction amount, but a rough adjustment is performed to change the exposure all at once to a certain extent, and thereafter, fine adjustment is performed. In this way, it becomes easier to adjust the brightness, and the adverse effect of luminance hunting occurring when only rough adjustment is performed can be suppressed.

[0049] FIG. 8 is a diagram used to explain an example of such exposure adjustment. In the example of FIG. 8, a predetermined upper limit of specifications is used as a predetermined amount used in determining the exposure compensation amount. When the exposure compensation amount calculated in step S705 is equal to or less than the upper limit of specifications, the exposure determination unit 407 follows a transition of small compensation amount shown by a solid line 801 in the figure, and changes the exposure of the surveillance camera in one go up to the exposure compensation amount in the coarse adjustment period. On the other hand, when the exposure compensation amount calculated in step S705 exceeds the upper limit of specifications, the exposure determination unit 407 follows a transition of large compensation amount shown by a solid line 802 in the figure, and changes the exposure in one go up to the upper limit of specifications. Then, the exposure determination unit 407 performs exposure control such that the exposure level is gradually changed after the upper limit of specifications is reached until the appropriate exposure is obtained in the subject area.

[0050] The predetermined amount used for comparison with the exposure compensation amount may be changed as appropriate according to the degree of agreement between the evaluation area in the surveillance camera and the evaluation area in the client device. In addition, when exposure control as shown by the solid line 801 in Fig. 8 is performed, the upper limit of the specification for suddenly changing the exposure may be a value different from the predetermined amount used for comparison with the exposure compensation amount.

[0051] If it is determined in step S706 that the exposure compensation amount is equal to or less than the predetermined amount and the process proceeds to step S707, the exposure determination unit 407 sends an instruction to the surveillance camera 101 via the communication control unit 402 to control the exposure of the surveillance camera 101 in accordance with the exposure compensation amount calculated as described above. As a result, the surveillance camera 101 performs exposure control based on the instruction corresponding to the exposure compensation amount, and as a result, it becomes possible to capture an image in which the subject has appropriate brightness.

[0052] On the other hand, if it is determined in step S706 that the exposure compensation amount exceeds the predetermined amount and the process proceeds to step S708, the exposure determination unit 407 sends an instruction to the surveillance camera 101 via the communication control unit 402 to control the exposure of the surveillance camera 101 as shown by the large compensation amount in Figure 8.

[0053] Next, in step S709, the exposure determination unit 407 measures the brightness of the subject area of ​​the image obtained after the exposure control as described above, and further determines in step S710 whether the exposure of the subject area is a desired appropriate value. If it is determined in step S710 that the exposure is an appropriate value, the client CPU 301 ends the process of the flowchart in FIG. 7. On the other hand, if it is determined in step S710 that the exposure is not an appropriate value, the exposure determination unit 407 further adjusts the exposure of the surveillance camera via the communication control unit 402 in step S711, and then returns to the process of step S709. Then, the exposure determination unit 407 measures the brightness of the subject area again in step S709, and then performs the determination process of step S710. Note that, in the exposure adjustment in step S711, it is desirable to change the exposure while finely adjusting it, rather than changing it greatly, as described above. Furthermore, when the exposure determination unit 407 returns to the process of step S709 to remeasure the brightness of the subject area, the exposure determination unit 407 performs the measurement after a certain interval, taking into consideration the time until the exposure adjustment in the surveillance camera 101 is reflected.

[0054] By carrying out the above-mentioned processing, it is possible to obtain an image in which the subject area is properly exposed. FIG. 9 is a diagram for explaining the process exchange between the surveillance camera and the client device in the first embodiment described above, and shows a specific example of how camera control is preferably performed. An image 901 shown in FIG. 9, like the image 501 in FIG. 5, shows an image in the initial state when the surveillance camera captures an image of a subject, and shows an example of an image captured in a backlit scene.

[0055] The client device performs the above-mentioned exposure evaluation process based on image 902, which is similar to image 502 in Fig. 5. At this time, the client device issues an instruction to increase the exposure because the area of ​​the person in image 902 is dark, but in this case, the amount of exposure compensation will be large, so as described above, a coarse adjustment is performed once, and control is performed by suppressing the amount of exposure compensation once at a certain level. Image 903 shows an example of an image captured by a surveillance camera in which exposure control has been performed by the client device in this manner.

[0056] The client device then detects the brightness of the subject again, performs fine adjustments to gradually change the brightness as shown in image 904, and controls the exposure of the surveillance camera so that the subject has appropriate brightness. Image 905 shows an example of an image obtained by performing such exposure control on the surveillance camera.

[0057] As described above, in the first embodiment, by changing the brightness in stages, it is possible to obtain an image with appropriate brightness even if the evaluation areas of the surveillance camera and the client device are different.

[0058] <Second embodiment> In the second embodiment, a process in which degamma processing is added to the first embodiment will be described. As described above, degamma processing is a process used to obtain stable brightness independent of the settings of the surveillance camera, such as exposure. In the second embodiment, by adding degamma processing, it is possible to realize more suitable processing than the example of the first embodiment.

[0059] Fig. 10 is a flowchart showing the flow of processing executed by the client CPU 301 in the second embodiment. As in the above, the processing of the flowchart of Fig. 10 is also assumed to be performed when the power of the surveillance camera 101, the client device 103, the input device 104, and the display device 105 are turned on and a connection (communication) between the surveillance camera 101 and the client device 103 is established. In this state, it is assumed that the transmission of image data from the surveillance camera 101 to the client device 103 and the display of the image on the display device 105 are repeated. Then, it is assumed that the processing of the flowchart of Fig. 10 is started by the client CPU 301 in response to the input of a captured image of a subject from the surveillance camera 101 to the client device 103 via the network 102.

[0060] First, in step S901, the subject detection unit 406 executes a process of detecting a subject from an image sent from the surveillance camera 101, similar to step S701 in FIG. Next, in step S902, the subject detection unit 406 determines whether or not a subject (a face in this example) has been detected. If a subject has not been detected in step S902, the client CPU 301 ends the process of the flowchart in Fig. 10, whereas if a subject has been detected, the process proceeds to step S903.

[0061] In step S903, the exposure determination unit 407 measures the brightness of the subject area (face area). Next, in step S904, the exposure determination unit 407 performs degamma processing by referring to a brightness conversion table (degamma table). Fig. 11 is a diagram for explaining an example of de-gamma processing. Characteristics 1101 in Fig. 11 indicate image characteristics in the image sensor 202 of the surveillance camera 101, and characteristics 1102 indicate input / output characteristics (gamma characteristics in this embodiment) in the luminance conversion performed in the surveillance camera 101. In other words, the surveillance camera 101 outputs an image that has been subjected to gamma processing (1102) within the camera in response to the image characteristics (1101) input from the image sensor 202.

[0062] In this embodiment, it is assumed that the input / output characteristic information of the brightness conversion performed by the surveillance camera, that is, the information indicating the gamma characteristic, is acquired by the camera information acquisition unit 404 and prepared as a gamma table (brightness conversion table), for example. The gamma characteristic information may be acquired in a state of being held as metadata of the input image, or may be acquired from a plurality of different patterns of gamma characteristic information corresponding to the type of the connectable surveillance camera 101 that are pre-stored. The plurality of different patterns of gamma characteristic information may be stored in, for example, the auxiliary storage device 303, or may be formed by executing a program and stored in the main storage device 302. For example, when a plurality of different patterns of gamma characteristic information are pre-stored, the camera information acquisition unit 404 performs an identification information acquisition process to acquire an ID, a serial number, an individual identification number, etc., indicating the type of the surveillance camera 101. Then, the camera information acquisition unit 404 selects the applicable gamma characteristic information from the plurality of pre-stored gamma characteristic information based on at least one of the identification information.

[0063] The exposure determination unit 407 acquires gamma characteristic information from the above-mentioned gamma table, and performs de-gamma processing on the input image from the surveillance camera so as to return it to the state before gamma processing was applied within the camera, that is, the state of the image captured by the image sensor 202.

[0064] Next, in step S905, the exposure determination unit 407 performs a process of calculating an exposure correction amount based on the luminance value after the degamma process. Furthermore, in step S 906 , the exposure determination unit 407 acquires the current exposure setting value of the surveillance camera acquired by the camera information acquisition unit 404 from the surveillance camera 101 . Then, in step S907, the exposure determination unit 407 calculates the amount of exposure correction required to make the subject area appropriately exposed, in the same manner as in the first embodiment described above.

[0065] Next, in step S908, the exposure determination unit 407 judges whether the exposure correction amount calculated in step S907 is equal to or less than a predetermined amount. That is, in step S908, a process is performed to check the amount of exposure correction (amount of change in exposure). As described above, when the evaluation areas are different between the monitoring camera and the client device, the exposure correction amount calculated on the client device side is likely to be a correction amount that is significantly different from the actual amount, so the judgment process in step S908 is performed. In the second embodiment, as in the first embodiment, when the exposure correction amount is greater than the predetermined amount, a rough adjustment is performed to change the exposure at once to a certain extent, and thereafter, fine adjustment is performed. This makes it easier to adjust the brightness, and also suppresses the adverse effect of brightness hunting that occurs when only rough adjustment is performed. If the exposure correction amount is equal to or less than the predetermined amount in step S908, the process of the client CPU 301 proceeds to step S910, and if it exceeds the predetermined amount, the process proceeds to step S909.

[0066] When the process proceeds to step S909, the exposure determination unit 407 sends an instruction to the surveillance camera 101 to control the surveillance camera 101 according to the amount of exposure compensation calculated in step S907 via the communication control unit 402. The surveillance camera 101 performs exposure control based on the instruction corresponding to the amount of exposure compensation, thereby making it possible to obtain appropriate brightness for the subject.

[0067] On the other hand, if it is determined in step S908 that the exposure compensation amount exceeds the predetermined amount and the process proceeds to step S910, the exposure determination unit 407 sends, via the communication control unit 402, an instruction to change the exposure of the surveillance camera 101 as shown by the large compensation amount in Figure 8 of the first embodiment.

[0068] Next, in step S911, the exposure determination unit 407 measures the brightness of the subject area (face area), and further determines whether the exposure is an appropriate value in step S912. If it is determined in step S912 that the exposure is an appropriate value, the client CPU 301 ends the process of the flowchart in FIG. 10. On the other hand, if it is determined in step S912 that the exposure is not an appropriate value, the exposure determination unit 407 further adjusts the exposure of the surveillance camera via the communication control unit 402 in step S913, and then returns the process to step S911. Then, the exposure determination unit 407 measures the brightness of the subject area again in step S911, and then performs the determination process of step S912. In the second embodiment as well, it is desirable to change the exposure control here by finely adjusting it, rather than changing it greatly, as described above. Also in the second embodiment, when the exposure determination unit 407 returns the process to step S911 and remeasures the brightness of the subject area, it does so after a certain interval.

[0069] By carrying out the above-mentioned processing, an image with appropriate brightness can be obtained in the second embodiment. Note that, in the second embodiment, the image of the processing exchange between the surveillance camera and the client device is the same as that explained using FIG. As described above, according to the second embodiment, it is possible to obtain stable brightness independent of the exposure setting conditions of the surveillance camera, and the surveillance camera 101 can capture appropriate images.

[0070] In the technology of Patent Document 1 described above, the brightness of the image sensor can be obtained regardless of the shooting conditions by performing degamma processing, but no method is disclosed for controlling the exposure of the image pickup device based on the result. In addition, the technology of Patent Document 2 describes processing when there is no information on the degamma curve, but does not describe camera control after degamma processing. Therefore, it is assumed that the technologies of Patent Documents 1 and 2 cannot correctly control the camera to obtain an appropriate brightness for the target subject. In contrast, according to the first and second embodiments, exposure control is realized that can obtain an image with appropriate brightness.

[0071] The present invention can also be realized by supplying a program for implementing one or more of the functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions. The above-mentioned embodiments are merely examples of the implementation of the present invention, and the technical scope of the present invention should not be interpreted as being limited by these. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features. [Explanation of symbols]

[0072] 100: Imaging control system, 101: Surveillance camera, 103: Client device, 403: Input image acquisition unit, 404: Camera information acquisition unit, 407: Exposure determination unit

Claims

1. A detection means for detecting a specific subject area within an image sent from an imaging device; an acquisition means for acquiring an exposure setting value of the imaging device; a determining means for determining an exposure amount of the subject area based on an exposure setting value of the imaging device and information on the subject area; a control unit that controls exposure of the imaging device based on the exposure amount determined by the determination unit, an exposure compensation method for compensating for an exposure of the imaging device by adjusting the exposure of the imaging device at a first speed until the exposure amount determined by the determination method is greater than a predetermined amount, and a second exposure compensation method for adjusting the exposure of the imaging device at a second speed slower than the first speed after the exposure amount determined by the determination method is greater than a predetermined amount.

2. the acquiring means acquires the exposure setting value that is set based on the luminance measured by the imaging device in an evaluation area of ​​the imaging device; 2. The imaging control device according to claim 1, wherein the determining means uses, as the information on the subject region, information on luminance measured using the subject region as an evaluation region.

3. 2. The imaging control device according to claim 1, wherein, when the exposure amount determined by the determination means is equal to or less than a predetermined amount, the control means performs exposure control of the imaging device using the first exposure correction method until the exposure amount is reached.

4. 4. The imaging control device according to claim 3, wherein the predetermined amount that the control means compares with the exposure amount is an amount that is changed depending on the degree of coincidence between the evaluation area of ​​the imaging device and the evaluation area of ​​the subject area.

5. A conversion means for converting input / output characteristics of luminance, 5. The imaging control device according to claim 1, wherein the information on the subject area used by the determination means when determining the exposure amount is information on the subject area within an image in which the input / output characteristics have been converted by the conversion means for the image sent from the imaging device.

6. 6. The imaging control device according to claim 1, wherein when the determination unit remeasures a change in exposure due to exposure control of the imaging device after the exposure control has been performed, the determination unit performs the remeasurement after a certain time interval.

7. An imaging control method executed by an imaging control device, A detection step of detecting a specific subject area in an image sent from an imaging device; acquiring an exposure setting value of the imaging device; a determining step of determining an exposure amount of the subject area based on an exposure setting value of the imaging device and information on the subject area; a control step of controlling exposure of the imaging device based on the exposure amount determined by the determination step, an exposure control method for controlling an exposure of the imaging device, the exposure control method comprising: a first exposure compensation method for performing coarse adjustment of the exposure of the imaging device at a first speed until the exposure amount determined in the determination method becomes a predetermined amount when the exposure amount determined in the determination method is greater than a predetermined amount; and a second exposure compensation method for adjusting the exposure of the imaging device at a second speed slower than the first speed after the exposure amount becomes the predetermined amount.

8. A program for causing a computer to function as each of the means of the imaging control device according to any one of claims 1 to 6.

Images