Imaging control device, imaging control method, and program

Abstract

To enable proper exposure control.SOLUTION: An imaging control device (103) includes detection means (406) for detecting a specific subject area in an image captured by an imaging apparatus (101), determination means (403 or 404) for determining whether input / output characteristic is valid, conversion means (407) for converting predetermined input / output characteristics for the image, decision-making means (407) for deciding an exposure control value on the basis of the luminance value of a specific subject area of the image for which the conversion has been made, and control means (402) for controlling exposure of the imaging apparatus (101) on the basis of the exposure control value. A method of controlling exposure by control means (407) is changed according to the result of determining whether the input / output characteristic is valid.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a technique for controlling an imaging device. [Background technology]

[0002] In recent years, in imaging devices such as surveillance cameras, digital cameras, and video cameras, a technology has been proposed that automatically detects a specific area of ​​a subject from a captured image and adjusts shooting settings such as exposure and image quality appropriately based on information related to the detected area. Patent Document 1 discloses a technology that uses pre-set gamma information when gamma information cannot be obtained when performing inverse gamma processing on a captured image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2007-102284 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technique of Patent Document 1 is unable to perform appropriate exposure control when the pre-set gamma information is incompatible.

[0005] Therefore, an object of the present invention is to enable appropriate exposure control. [Means for solving the problem]

[0006] The imaging control device of the present invention comprises: An acquisition means for acquiring an image captured by an imaging device; A detection means for detecting a particular subject region within an image; a determination means for determining whether the input / output characteristics applied to the image are valid depending on whether the input / output characteristics can be acquired; and when the determination means determines that the input / output characteristics are valid, Convert input / output characteristics and obtain an inverse input / output characteristic corresponding to the input / output characteristic. A conversion means, and the conversion means converts the The inverse input / output characteristica determining unit that determines an exposure control value based on a luminance value of the specific object region of an image, and a control unit that controls exposure of the imaging device based on the exposure control value, When it is determined that the input / output characteristics are not valid, The control means The time until exposure control by the input / output characteristic is started is set to be slower than when the input / output characteristic is determined to be valid. The present invention is characterized by the above. Effect of the Invention

[0007] According to the present invention, appropriate exposure control is possible. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of an imaging control system. [Diagram 2] FIG. 1 is a diagram illustrating an example of the internal configuration of a surveillance camera. [Diagram 3] FIG. 2 illustrates an example of the internal configuration of a client device. [Figure 4] FIG. 2 illustrates an example of a functional configuration of a client device. [Diagram 5] 4 is a flowchart of an exposure control process according to the first embodiment. [Figure 6] 1A and 1B are diagrams illustrating an example of detection of a face region and a human body region. [Figure 7] 4 is a diagram illustrating characteristics of gamma processing and inverse gamma processing according to the first embodiment. FIG. [Figure 8] 4A and 4B are diagrams showing exposure value changes in two exposure control methods. [Figure 9] 10 is a flowchart of an exposure control process according to a second embodiment. [Figure 10] FIG. 13 is an image diagram of a specific chart. [Figure 11] 13 is a conceptual diagram of an input image when exposure is controlled within a predetermined range. [Figure 12] 13 is a diagram illustrating characteristics of gamma processing and inverse gamma processing according to the second embodiment. FIG. [Figure 13] 13A to 13C are diagrams illustrating an example of tone correction processing according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, embodiments of the present invention will be described with reference to the drawings. The configurations described in the following embodiments are merely examples, and the present invention is not limited to the configurations shown in the drawings, etc. In the following embodiments, the same configurations are described with the same reference numerals.

[0010] First Embodiment FIG. 1 is a block diagram showing an example of the configuration of an imaging control system 100 according to the first embodiment. The imaging control system 100 includes a surveillance camera 101, a network 102, a client device 103, an input device 104, and a display device 105. The surveillance camera 101 is an imaging device for acquiring moving images, and is capable of imaging a subject and processing the images. The surveillance camera 101 and the client device 103 are connected to each other via the network 102 in a state in which they can communicate with each other. The client device 103 is connected to the input device 104 and the display device 105 in a state in which they can communicate with each other. The client device 103 is a device that processes various information, and therefore may be called an information processing device. In addition, the client device 103 is a device that controls the imaging of the surveillance camera 101, and therefore may be called an imaging control device.

[0011] The input device 104 is composed of a mouse, a keyboard, and the like, and is operated by the user of the client device 103 . The display device 105 is a device equipped with a monitor or the like for displaying an image received from the client device 103. The display device 105 can also function as a UI such as a touch panel. In this case, the display device 105 can also function as an input device for inputting instructions, information, data, and the like to the client device 103. The UI is an abbreviation for User Interface.

[0012] In FIG. 1, the client device 103, the input device 104, and the display device 105 are depicted as independent devices, but the present embodiment is not limited to such a configuration. For example, the client device 103 and the display device 105 may be integrated, or the input device 104 and the display device 105 may be integrated. Also, the client device 103, the input device 104, and the display device 105 may be integrated. When the client device 103 and the display device 105 are integrated, the integrated device may be in the form of, for example, a personal computer, a tablet terminal, or a smartphone.

[0013] FIG. 2 is a block diagram showing an example of the internal configuration of the surveillance camera 101. The surveillance camera 101 has an imaging optical system 201 and an imaging element 202 as components for capturing an image. The surveillance camera 101 further has a camera CPU 203, ROM 204, RAM 205, an imaging system control unit 206, a control unit 207, an A / D conversion unit 208, an image processing unit 209, an encoder unit 210, and a network I / F 211. The camera CPU 203 to the network I / F 211 of the surveillance camera 101 are interconnected by a system bus 212. Note that CPU is an abbreviation for Central Processing Unit. ROM is an abbreviation for Read Only Memory. RAM is an abbreviation for Random Access Memory. A / D is an abbreviation for Analog / Digital. I / F is an abbreviation for Interface.

[0014] The imaging optical system 201 is a group of optical members that collects light from a subject, and is composed of a zoom lens, a focus lens, a blur correction lens, an aperture, a shutter, etc. The optical image of the subject etc. collected by the imaging optical system 201 is formed on the imaging surface of the image sensor 202. The imaging element 202 is a charge-accumulation type solid-state imaging element such as a CMOS or CCD that converts the light image collected by the imaging optical system 201 into a current value (signal value), and is an imaging unit that acquires color information by combining with a color filter or the like. CMOS is an abbreviation for Complementary Metal Oxide Semiconductor. CCD is an abbreviation for Charge-Coupled Device. The imaging element 202 is connected to an A / D conversion unit 208.

[0015] The A / D conversion unit 208 converts the amount of light of the subject detected by the image sensor 202 into a digital signal (image data). The A / D conversion unit 208 transmits the digital signal to the image processing unit 209. The image processing unit 209 performs image processing on the image data of the digital signal received from the imaging element 202. The image processing unit 209 is connected to an encoder unit 210. The encoder unit 210 converts the image data processed by the image processing unit 209 into a file format such as Motion JPEG, H264, H265, etc. The encoder unit 210 is connected to a network I / F 211.

[0016] The camera CPU 203 is a control unit that performs overall control of the operation of the security camera 101. The camera CPU 203 reads instructions stored in the ROM 204 or RAM 205, and executes processing according to the results. The imaging system control unit 206 controls each unit of the surveillance camera 101 based on instructions from the camera CPU 203. For example, the imaging system control unit 206 controls the imaging optical system 201 in terms of focus control, shutter control, aperture adjustment, and the like.

[0017] The network I / F 211 is an interface used for communication with external devices such as the client device 103 via the network 102 , and is controlled by the control unit 207 . The control unit 207 controls communication with the client device 103, and performs control such as transmitting control commands (control signals) from the client device 103 to each unit of the security camera 101 to the camera CPU 203.

[0018] The network 102 is an IP network that connects the surveillance camera 101 and the client device 103. The network 102 is composed of a plurality of routers, switches, cables, etc. that comply with a communication standard such as Ethernet (registered trademark). In this embodiment, the network 102 may be any network that can perform communication between the surveillance camera 101 and the client device 103, and the communication standard, scale, configuration, etc. are not important. For example, the network 102 may be composed of the Internet, a wired LAN (Local Area Network), a wireless LAN (Wireless LAN), a WAN (Wide Area Network), etc.

[0019] FIG. 3 is a block diagram showing an example of the internal configuration of the client device 103. As shown in FIG. The client device 103 includes a client CPU 301, a main memory device 302, an auxiliary memory device 303, an input I / F 304, an output I / F 305, and a network I / F 306. The elements of the client device 103 are connected to each other via a system bus 307 so as to be able to communicate with each other.

[0020] The client CPU 301 is a central processing unit that performs overall control of the operation of the client device 103. Note that the client CPU 301 may also perform overall control of the security camera 101 via the network . The main memory device 302 is a memory device such as a RAM that functions as a temporary storage location for data of the client CPU 301 .

[0021] The auxiliary storage device 303 is a storage device such as an HDD, a ROM, or an SSD that stores various programs, various setting data, and the like. Note that HDD is an abbreviation for Hard Disk Drive. SSD is an abbreviation for Solid State Drive. The program according to this embodiment is stored in the auxiliary storage device 303. In this embodiment, the client CPU 301 executes processing based on the program read from the auxiliary storage device 303 and deployed in the main storage device 302, thereby realizing the functions and processing of the client device 103 shown in FIG. 4. Details of this will be described later. In addition, the auxiliary storage device 303 may be pre-stored with, for example, a pattern for pattern matching (a pattern corresponding to a facial feature or a human body feature) that is used when the client device 103 performs face detection or human body detection based on image data. Note that these pattern matching patterns may be formed by executing a program and stored in the main storage device 302.

[0022] The input I / F 304 is an interface that is used when the client device 103 receives an input (signal) from the input device 104 or the like. The output I / F 305 is an interface that is used when outputting information (signals) from the client device 103 to the display device 105 or the like. The network I / F 306 is an interface used for communication with external devices such as the surveillance camera 101 via the network 102 .

[0023] Fig. 4 is a functional block diagram showing each function executed by the client device 103. In other words, each functional unit (functional block) shown in Fig. 4 is a functional unit that can be executed by the client CPU 301, and each of these functional units is synonymous with the client CPU 301.

[0024] 4, client CPU 301 of client device 103 includes functional units, namely, an input signal acquisition unit 401, a communication control unit 402, an input image acquisition unit 403, a camera information acquisition unit 404, and a detection method setting unit 405. Client CPU 301 also includes functional units, namely, an object detection unit 406, an exposure determination unit 407, and a display control unit 408. Note that, in client device 103, each of the functional units, namely, input signal acquisition unit 401 to display control unit 408 shown in FIG. 4, may be configured by hardware (or software) separate from client CPU 301.

[0025] The input signal acquisition unit 401 receives an input from the user via the input device 104 . The communication control unit 402 executes control for receiving images transmitted from the surveillance camera 101 (images captured by the surveillance camera 101) via the network 102. The communication control unit 402 also executes control for transmitting control commands from the client device 103 to the surveillance camera 101 via the network 102.

[0026] The input image acquisition unit 403 acquires an image received from the surveillance camera 101 via the communication control unit 402 as an input image (an image to which the subject detection process is applied) that is a target of the subject detection process. Details of the detection process will be described later. The camera information acquisition unit 404 acquires camera information when the surveillance camera 101 captures an image of a subject or the like via the communication control unit 402. The camera information is various camera setting information and image processing information when capturing an image of a subject or the like and acquiring the image. Specifically, the camera information includes, for example, exposure parameters such as aperture, shutter speed, and gain (camera setting information related to exposure), and information related to image processing related to brightness such as gamma correction, edge emphasis, and white balance. Furthermore, in this embodiment, the input image acquisition unit 403 or the camera information acquisition unit 404 also performs a determination process to determine whether or not the input / output characteristics for the image are valid in the surveillance camera 101. The details of the input / output characteristic determination process will be described later.

[0027] The detection method setting unit 405 sets a predetermined (appropriate) detection method from among various detection methods including detection of a face area (face detection) and detection of a human body area (human body detection) for the input image acquired by the input image acquisition unit 403. In this embodiment, the detection method setting unit 405 sets (selects) the detection method of face detection or the detection method of human body detection.

[0028] The subject detection unit 406 detects a specific subject area in an input image captured by the surveillance camera 101 and acquired by the input image acquisition unit 403. For example, when face detection is set in the detection method setting unit 405, the subject detection unit 406 detects a face area from the input image with priority. Also, when human body detection is set in the detection method setting unit 405, the subject detection unit 406 detects a human body area from the input image with priority.

[0029] Note that this embodiment is not limited to such settings. For example, a detection method for detecting a characteristic area of ​​a part of a person, such as the upper body, head, and some organs such as the eyes, nose, and mouth of the face, may be set (selected). In addition, in this embodiment, the specific subject to be detected is a person, but a configuration capable of detecting a characteristic area related to a specific subject other than a person may be used. For example, a configuration capable of detecting a specific subject preset in the client device 103, such as an animal's face or a car, may be used.

[0030] The exposure determination unit 407 performs a conversion process for converting a predetermined input / output characteristic for an input image, and a determination process for determining an exposure control value based on a luminance value of a specific object region of the image with the input / output characteristic converted. Then, the exposure determination unit 407 performs exposure control of the surveillance camera 101 via the communication control unit 402 based on the determined exposure control value. That is, the exposure determination unit 407 determines an exposure level based on image information of the object region obtained from the object detection unit 406, and sends an exposure control value (exposure correction value) corresponding to the exposure level to the communication control unit 402. The communication control unit 402 transmits a control command corresponding to the exposure control value (exposure correction value) to the surveillance camera 101 via the network I / F 306. As a result, in the surveillance camera 101 that receives the control command, the control unit 207 and the imaging system control unit 206 execute exposure control. In addition, in this embodiment, the exposure determination unit 407 changes a method for controlling the exposure of the surveillance camera 101 depending on a determination result of whether the input / output characteristic is valid or not. The detailed processing in the exposure determination unit 407 will be described later with reference to the flowchart in FIG.

[0031] The display control unit 408 , in accordance with an instruction from the client CPU 301 , outputs to the display device 105 the captured image in which the exposure compensation based on the exposure compensation value determined by the exposure determination unit 407 is reflected.

[0032] FIG. 5 is a flowchart showing the flow of the object detection process and exposure control process according to this embodiment. The flowchart in FIG. 5 shows a detailed process flow related to the input image acquisition unit 403, the camera information acquisition unit 404, the detection method setting unit 405, the object detection unit 406, and the exposure determination unit 407. In the imaging control system 100 in FIG. 1, it is assumed that the powers of the surveillance camera 101, the client device 103, the input device 104, and the display device 105 are turned on, and a connection (communication) between the surveillance camera 101 and the client device 103 is established. In this state, it is assumed that the surveillance camera 101 captures an image of an object or the like at a predetermined update period, the surveillance camera 101 transmits image data to the client device 103, and the display device 105 displays the image. Then, it is assumed that the client CPU 301 starts the process of the flowchart in FIG. 5 in response to the input of a captured image of an object or the like from the surveillance camera 101 to the client device 103 via the network 102.

[0033] First, in step S501, detection method setting unit 405 sets a face or human body detection method for subject detection unit 406. Then, subject detection unit 406 performs face detection processing or human body detection processing on an input image according to the setting by detection method setting unit 405. Patterns corresponding to facial features and human body features, respectively, are stored in advance in auxiliary storage device 303 of client device 103, and subject detection unit 406 detects a face region or a human body region by pattern matching based on the patterns.

[0034] Next, in step S502, the subject detection unit 406 determines whether a face region or a human body region has been detected in the image by the face detection or human body detection process executed in step S501. If at least one face region or human body region has been detected, the process of the client CPU 301 proceeds to step S503, and if no face region or human body region has been detected, this process ends.

[0035] Here, when detecting a face region, it is generally possible to detect the face with high accuracy, and it is possible to clearly distinguish the face region of the subject from the non-face region. However, if the face direction, face size, face brightness, etc. are not suitable conditions for face detection, the face region may not be detected accurately. In contrast, when detecting a human body, it is possible to detect an area where a person exists regardless of the face direction, face size, face brightness, etc. Note that the human body detection in this embodiment does not necessarily need to detect the entire body, and may detect the upper body, bust up, or head region including the face. FIG. 6(a) is a diagram showing an example of a face region 601 detected from an image, and FIG. 6(b) is a diagram showing an example of a human body region 602 detected from an image.

[0036] Furthermore, when a pattern matching method is adopted as a method for detecting a subject, a pattern (classifier) ​​created using statistical learning may be used as a pattern used in pattern matching. Alternatively, subject detection may be performed using a method other than pattern matching. For example, subject detection may be performed using a luminance gradient in a local region of an image. In other words, the method for detecting a subject is not limited to a specific detection method, and various methods such as detection based on machine learning and detection based on distance information may be adopted.

[0037] When the process proceeds from step S502 to step S503, the input image acquisition unit 403 or the camera information acquisition unit 404 performs a process of determining whether or not predetermined input / output characteristics for the input image are valid. In this embodiment, the input image acquisition unit 403 or the camera information acquisition unit 404 determines whether or not predetermined gamma information for the input image can be acquired as the process of determining the input / output characteristics.

[0038] The gamma information may be acquired in a state where it is held as metadata of the input image, or may be acquired from a plurality of different patterns of gamma information corresponding to the types of connectable surveillance cameras 101 that are pre-stored. Note that these plurality of different patterns of gamma information may be stored in, for example, the auxiliary storage device 303, or may be formed by executing a program and stored in the main storage device 302. For example, when a plurality of different patterns of gamma information are pre-stored, the camera information acquisition unit 404 performs an identification information acquisition process to acquire an ID, a serial number, an individual identification number, etc., that indicate the type of the surveillance camera 101. Then, the camera information acquisition unit 404 selects applicable gamma information from the plurality of pre-stored gamma information based on at least one of the identification information.

[0039] 7 is an example of a graph showing the relationship between the input value and the output value of the gamma characteristic corresponding to the gamma information. If the gamma information for the input image cannot be acquired, that is, if the gamma characteristic is unknown, the process of the client CPU 301 proceeds to step S509, and if the gamma information can be acquired, the process proceeds to step S504. When the process proceeds to step S504, the exposure determination unit 407 performs a process of converting predetermined input / output characteristics for the input image. In the present embodiment, the exposure determination unit 407 performs input / output characteristic conversion processing such as performing inverse gamma processing corresponding to the gamma information acquired in step S503. At this time, the inverse gamma characteristic may be generated as a table or a gamma value. The relationship between the input value and the output value when the gamma value is γ can be expressed by the following formula (1).

[0040]

number

[0041] Here, I(x, y) in the formula represents the brightness value at the two-dimensional coordinate position (x, y) in the horizontal direction (x-axis direction) and vertical direction (y-axis direction) in the image, and Imax represents the maximum brightness value. The dashed dotted line 702 in FIG. 7 is a graph of the relationship between the input value and the output value in the inverse gamma characteristic corresponding to the gamma characteristic represented by the solid line 701.

[0042] Next, in step S505, the exposure determination unit 407 calculates the average luminance value of the face region or human body region (hereinafter referred to as the region of interest) detected in step S501. Specifically, the exposure determination unit 407 applies information related to the number of regions of interest detected in step S501, the positions of the regions of interest, and the sizes of the regions of interest to the following formula (2).

[0043]

number

[0044] In the formula, f represents the number of regions of interest, (v, h) represents the center coordinates of the region of interest, k represents the horizontal size of the region of interest, and l represents the detected size of the region of interest in the vertical direction.

[0045] Next, in step S506, the exposure determination unit 407 determines an exposure correction amount EVcorrection based on the average brightness value calculated in step S505. First, the exposure determination unit 407 calculates an exposure value addition amount EVadd using the average brightness value Iobject of the attention area calculated in step S505 and the target brightness value Iobjecttarget of the attention area as shown in formula (3). Here, the target brightness value Iobjecttarget is a target brightness value after inverse gamma processing. When a camera is connected to a system such as face recognition, an image after gamma processing is input to the system, so a target brightness value after gamma processing is first determined, and then inverse gamma processing is performed on this to set the target brightness value Iobjecttarget. The target brightness value may be arbitrarily set by the user, or may be set to a value that increases accuracy in comparison with the authentication accuracy during face recognition.

[0046]

number

[0047] Next, the exposure determination unit 407 determines the exposure correction amount EVcorrection as shown in equation (4).

[0048]

number

[0049] Here, EVcurrent in the formula is the current exposure value. Also, the parameter α is a coefficient that affects the degree of correction (speed) for the current exposure value EVcurrent. If the value of parameter α is set to a large value, the processing speed (or time) until the target value is reached will be faster, but if an erroneous judgment occurs in the detection result or if the subject detection is unstable, the brightness of the entire screen will fluctuate sharply. On the other hand, if the value of parameter α is set to a small value, the processing speed (or time) until the exposure reaches the target will be slower, but it will be more robust to erroneous detection and shooting conditions.

[0050] Next, in step S507, the camera information acquisition unit 404 acquires information on the current exposure setting value from the surveillance camera 101. In the case of this embodiment, the current exposure compensation value of the surveillance camera 101 is acquired. Next, in step S508, the exposure determination unit 407 communicates with the surveillance camera 101 via the communication control unit 402, and sets an exposure compensation value corresponding to the exposure compensation amount calculated in step S507 as an exposure control value in the surveillance camera 101. Thereafter, the client CPU 301 ends the processing of this flowchart.

[0051] On the other hand, if gamma information is not acquired in step S503 and the process proceeds to step S509 because the gamma information is unknown, the exposure determination unit 407 calculates the average luminance value of the attention areas detected in step S501 without performing inverse gamma processing. Specifically, the exposure determination unit 407 applies information related to the number of attention areas detected in step S501, the positions of the attention areas, and the sizes of the attention areas to the following formula (5). Explanation of the same elements in formula (5) as those in formula (2) will be omitted.

[0052]

number

[0053] Next, in step S510, the exposure determination unit 407 calculates a difference value between the average luminance value Iobject of the region of interest calculated in step S510 and the target luminance value Iobjecttarget of the region of interest, as shown in the following formula (6). Note that the target luminance value Iobjecttarget of the region of interest may be set arbitrarily by the user, for example, or may be set to a value that increases accuracy in light of the authentication accuracy during face authentication.

[0054]

number

[0055] Next, the exposure determination unit 407 determines the exposure correction amount EVcorrection by equation (7).

[0056]

number

[0057] The parameter β in the formula is a coefficient that affects the degree (speed) of correction when correcting the exposure to the underexposure side or the overexposure side. By setting the value of the parameter β to a large value, the processing speed (or time) until the target value is reached becomes faster, but when an erroneous judgment occurs in the detection result or when the detection of the object is unstable, the brightness of the entire screen changes sharply. On the other hand, when the value of the parameter β is set to a small value, the processing speed (or time) until the exposure reaches the target becomes slower, but it becomes robust against erroneous detection and shooting conditions. This parameter β is set as an exposure correction value on the over side when the difference ΔDiff exceeds the set threshold value Th. On the other hand, when the difference ΔDiff falls below the set threshold value -Th, it is set as an exposure correction value on the under side. By repeating this, the brightness of the object area is controlled to approach the target brightness value Iobject target.

[0058] In addition, the exposure control method when gamma information cannot be acquired and is unknown may be one that relatively delays the time until exposure control starts. This is for the purpose of preventing exposure from being controlled too sensitively to disturbances such as sudden changes in brightness. Fig. 8 is a graph showing an example of the change in exposure value over time until the target exposure value is reached due to the difference in the above-mentioned exposure control method, that is, an example of changing the exposure control method in this embodiment. The solid line 801 in Fig. 8 represents the change in exposure value when gamma information can be acquired, and the dashed line 802 represents the change in exposure value when gamma information cannot be acquired.

[0059] As described above, the client device 103 of the first embodiment changes the exposure control method based on the luminance value of the face area or human body area depending on the result of acquiring gamma information, that is, the result of judging whether the input / output characteristics are valid or not. As a result, it is possible to perform appropriate exposure control depending on the type of surveillance camera communicating with the client device 103. In other words, the client device 103 can perform appropriate exposure control regardless of whether gamma information of the connected surveillance camera can be acquired. Therefore, according to the first embodiment, it is possible to stably capture the face of a visitor or a visitor subject (person), for example.

[0060] <Second embodiment> Hereinafter, the second embodiment will be described with reference to Fig. 7 and Fig. 9 to Fig. 13. In the second embodiment, a method for generating gamma information when the gamma information of the camera cannot be acquired and the gamma characteristics are unknown will be described. Note that the same reference numerals are used for the same configurations and processes as those in the first embodiment, and detailed descriptions thereof will be omitted.

[0061] 9 is a flowchart showing the subject detection process and exposure control process executed by the client CPU 301 of the client device 103 according to the second embodiment. Note that the start timing of the process and the like are the same as in the first embodiment, and therefore a description thereof will be omitted. Also, the processes of steps S501 to S508 are the same as in the first embodiment, and therefore a description thereof will be omitted.

[0062] In the second embodiment, if gamma information cannot be acquired in step S503 and the gamma characteristics are unknown, the process of the client CPU 301 proceeds to step S601, and if gamma information can be acquired, the process proceeds to step S504. In step S601, the subject detection unit 406 detects a specific chart from the input image.

[0063] Here, the specific chart is a reflective chart 1001 as shown in Fig. 10, and has a plurality of patch sections 1002 with different shading (reflectance). It is preferable that the specific chart is created in advance taking into consideration a predetermined gamma characteristic (e.g., γ=2.2). Then, in step S601, if the specific chart is not detected, the process of the client CPU 301 proceeds to step S603, and if the specific chart is detected, the process proceeds to step S602.

[0064] In step S602, the input image acquisition unit 403 generates a gamma table to be used in the next step S504 from an input image including a specific chart. The input image acquisition unit 403 may acquire data on gamma characteristics of a specific chart prestored in the auxiliary storage device 303, for example. The input image acquisition unit 403 may also calculate the luminance value of each patch portion 1002 of a specific chart in the input image, and generate a gamma table based on the luminance value of each patch portion 1002. Here, it is assumed that gamma characteristics and gamma values ​​similar to those of the solid line 107 in FIG. 7 are obtained. In this case, in step S504 following step S602, an inverse gamma characteristic such as the dashed line 702 is obtained from the gamma characteristics and gamma values ​​of the solid line 701 in FIG. 7 described above.

[0065] On the other hand, if no specific chart is detected in step S601 and the process proceeds to step S603, the exposure determination unit 407 controls the exposure from over to under within the possible range. Note that the order of over and under exposure control does not matter at this time. If the exposure control range can be acquired by the camera information acquisition unit 404, the exposure determination unit 407 performs exposure control within the acquired exposure control range. Figure 11 shows an example of an image obtained when the exposure is controlled from over to under.

[0066] Next, in step S604, the input image acquisition unit 403 calculates the average luminance value of multiple images for each predetermined exposure correction value obtained in the process of controlling the exposure in step S603. The method of calculating the average luminance value corresponds to the above-mentioned formula (5) in which the attention area is set to the entire image, and therefore a description thereof will be omitted. The solid line 1201 in Fig. 12 is a graph of the average luminance value when the exposure is controlled to be over until the average luminance value of the image becomes maximum and under until it becomes minimum with respect to the reference exposure value.

[0067] Next, in step S605, the input image acquisition unit 403 generates an estimated gamma table or an estimated gamma value based on the average luminance value of the multiple images calculated in step S604. After that, the process of the client CPU 301 proceeds to step S504. In this case, in step S504, the exposure determination unit 407 performs an inverse conversion of the estimated gamma table or the estimated gamma value. The dashed and dotted line 1202 in FIG. 12 is a graph showing the characteristics when an inverse conversion is performed on the luminance characteristics shown by the solid line 1201.

[0068] Next, in step S606, the camera information acquisition unit 404 determines whether or not a predetermined image process related to the luminance of an image other than gamma processing is valid. For example, the predetermined image process may be an S-shaped tone correction process used to reduce fog or mist on an image, as shown by a solid line 1301 in FIG. 13. The predetermined image process may also be an image process having different luminance conversion characteristics, such as correction of dark areas or light areas of an image. If the image process related to the luminance of an image other than gamma processing is valid, the process of the client CPU 301 proceeds to step S607, whereas if it is invalid, the process ends after passing through each process from step S505 to step S508.

[0069] When it is determined in step S606 that the predetermined image processing is valid and the process proceeds to step S607, the exposure determination unit 407 converts the luminance conversion characteristics of the predetermined image processing into inverse characteristics. The characteristics obtained by inversely converting the luminance conversion characteristics represented by the solid line 1301 in FIG. 13 become the dashed and dotted line 1302.

[0070] Thereafter, the exposure determination unit 407 calculates the average luminance in step S505 using the conversion characteristics obtained by multiplying the inverse conversion characteristics and the inverse gamma characteristics in step S607, and then completes this process through the processes in steps S506 to S508.

[0071] In the second embodiment, even if gamma information cannot be obtained from the camera, new gamma information can be generated using a different method. As a result, according to the second embodiment, appropriate exposure control can be performed for the currently connected camera. Furthermore, according to the second embodiment, even if image processing that affects the brightness of an image, such as fog / haze correction or dark area correction other than gamma processing, is enabled, appropriate exposure control can be performed in a similar manner.

[0072] The present invention can also be realized by supplying a program for implementing one or more of the functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions. The above-mentioned embodiments are merely examples of the implementation of the present invention, and the technical scope of the present invention should not be interpreted as being limited by these. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features. [Explanation of symbols]

[0073] 100: Imaging control system, 101: Surveillance camera, 103: Client device, 403: Input image acquisition unit, 404: Camera information acquisition unit, 407: Exposure determination unit

Claims

1. An acquisition means for acquiring an image captured by an imaging device; a detection means for detecting a specific subject region within the image; a determination means for determining whether the input / output characteristics applied to the image are valid according to whether the input / output characteristics can be acquired; a conversion means for converting the input / output characteristics to obtain an inverse input / output characteristic corresponding to the input / output characteristics when the determination means determines that the input / output characteristics are valid; a determining means for determining an exposure control value based on the inverse input / output characteristic converted by the converting means and a luminance value of the specific object region of the image; a control unit that controls exposure of the imaging device based on the exposure control value, An imaging control device characterized in that, when the judgment means judges that the input / output characteristics are not valid, the time until exposure control by the control means is started is delayed compared to when the input / output characteristics are judged to be valid.

2. 2. The image capture control device according to claim 1, wherein the detection means detects a face area or a human body area as the specific subject area in the image.

3. The input / output characteristic is a gamma characteristic, and the inverse input / output characteristic is an inverse gamma characteristic, 3. The imaging control device according to claim 1, wherein the conversion means performs inverse gamma processing to convert the gamma characteristic into the inverse gamma characteristic and apply the inverse gamma characteristic to the image.

4. 4. The imaging control device according to claim 1, further comprising a storage unit for storing a plurality of different input / output characteristics in advance.

5. a selection unit for acquiring identification information of the imaging device and selecting an input / output characteristic suitable for the identification information from among the plurality of stored input / output characteristics, thereby acquiring the input / output characteristic; the determination means determines whether the input / output characteristics are valid depending on whether an input / output characteristic suitable for the identification information can be selected from the plurality of stored input / output characteristics; 5. The imaging control device according to claim 4, wherein the conversion means performs the conversion based on information on the selected input / output characteristics.

6. The determination means determines whether the input / output characteristics are valid depending on whether the input / output characteristics can be acquired from the imaging device, 5. The imaging control device according to claim 1, wherein the conversion means performs the conversion based on information about the input / output characteristics obtained from the imaging device.

7. An acquisition means for acquiring an image captured by an imaging device; a detection means for detecting a specific subject region within the image; a determination means for determining whether the input / output characteristics applied to the image are valid according to whether the input / output characteristics can be acquired; a generating means for generating an input / output characteristic when an input / output characteristic corresponding to the imaging device is not available; a conversion means for converting the input / output characteristics generated by the generation means to obtain an inverse input / output characteristic corresponding to the input / output characteristic; a determining means for determining an exposure control value based on the inverse input / output characteristic converted by the converting means and a luminance value of the specific object region of the image; a control means for controlling exposure of the imaging device based on the exposure control value; An imaging control device comprising:

8. 8. The imaging control device according to claim 7, wherein the generating means generates the information on the input / output characteristics based on luminance values ​​of the specific subject area of ​​a plurality of images.

9. 9. The imaging control device according to claim 8, wherein the generating means generates the input / output characteristics based on luminance values ​​of the subject regions of the plurality of images when the control means controls the exposure of the imaging device within a predetermined range.

10. An imaging control method for controlling an imaging device, comprising: an acquisition step of acquiring an image captured by the imaging device; a detection step of detecting a particular object region within the image; a determination step of determining whether the input / output characteristics applied to the image are valid depending on whether the input / output characteristics are obtainable; a conversion step of converting the input / output characteristics to obtain an inverse input / output characteristic corresponding to the input / output characteristics when the input / output characteristics are determined to be valid by the determination step; a determination step of determining an exposure control value based on the inverse input / output characteristics converted by the conversion step and a luminance value of the specific object region of the image; and a control step of controlling the exposure of the imaging device based on the exposure control value. An imaging control method, characterized in that, when the judgment step determines that the input / output characteristics are not valid, the time until exposure control by the control step is started is delayed compared to when the input / output characteristics are determined to be valid.

11. An imaging control method for controlling an imaging device, comprising: An acquisition step of acquiring an image captured by an imaging device; a detection step of detecting a particular object region within the image; a determination step of determining whether the input / output characteristics applied to the image are valid depending on whether the input / output characteristics are obtainable; a generating step of generating input / output characteristics corresponding to the imaging device when the input / output characteristics corresponding to the imaging device are not available; a conversion step of converting the input / output characteristics generated by the generation step to obtain inverse input / output characteristics corresponding to the input / output characteristics; a determination step of determining an exposure control value based on the inverse input / output characteristics converted by the conversion step and a luminance value of the specific object region of the image; a control step of controlling exposure of the imaging device based on the exposure control value; 13. An imaging control method comprising:

12. A program for causing a computer to function as each of the means of the imaging control device according to any one of claims 1 to 9.

Images