Camera device, camera method and storage medium

Abstract

The present invention provides an imaging device, an imaging method and a storage medium. The imaging device comprises: an imaging unit configured to set exposure conditions for each area on an imaging plane; a setting unit configured to set a privacy area in an image captured by the imaging unit; a decision unit configured to determine the exposure conditions of the privacy area set by the setting unit to be exposure conditions capable of protecting the privacy of a subject in the privacy area; and a control unit configured to perform control so that the imaging unit captures an image of the subject based on the exposure conditions determined by the decision unit.

Description

Technical Field

[0001] The invention relates to a camera device, a camera method and a storage medium. Background Art

[0002] In order to prevent infringement of portrait rights or copyrights, Japanese Patent No. 4512763 discusses a technology for performing image processing such as blurring processing on a predetermined internal area of ​​image data. Summary of the invention

[0003] A camera device includes: a camera unit, which is configured to set exposure conditions for various areas on a camera surface; a setting unit, which is configured to set a privacy area in an image captured by the camera unit; a decision unit, which is configured to decide the exposure conditions of the privacy area set by the setting unit to be exposure conditions that can protect the privacy of a subject in the privacy area; and a control unit, which is configured to perform control so that the camera unit captures an image of the subject based on the exposure conditions determined by the decision unit.

[0004] Further features of various embodiments will become apparent from the following description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Figure 1 A block diagram showing a schematic configuration example of an image pickup apparatus according to a first exemplary embodiment.

[0006] Figure 2 is a luminance distribution map used for calculating image pickup conditions according to the first exemplary embodiment.

[0007] Figure 3 A flowchart of a process for calculating imaging conditions according to the first exemplary embodiment is shown.

[0008] 4A to 4D An image acquired by the imaging device according to the first exemplary embodiment is shown.

[0009] Figure 5 A block diagram showing a schematic configuration example of an image pickup apparatus according to a second exemplary embodiment.

[0010] Figure 6 A flowchart of a process for calculating imaging conditions according to the third exemplary embodiment is shown.

[0011] Figure 7 A block diagram showing a configuration example of an image pickup system according to a fourth exemplary embodiment.

[0012] Figure 8A flowchart showing an example of mask setting processing is shown.

[0013] Fig. 9A and Fig. 9B Examples of captured images before and after masking processing is performed are shown.

[0014] Fig.10 A flowchart showing an example of mask type changing processing according to the fifth exemplary embodiment is shown.

[0015] Fig.11 A flowchart showing another example of the mask type changing process. DETAILED DESCRIPTION

[0016] Hereinafter, exemplary embodiments will be described with reference to the accompanying drawings. The exemplary embodiments described hereinafter are not intended to limit the embodiments, and, as a scheme of each embodiment, not all combinations of features described in the exemplary embodiments are required. The configurations described in the exemplary embodiments may be appropriately modified or changed according to the specifications of the system and device to which the present disclosure is applied and various conditions such as usage conditions and usage environment. The technical scope of the present disclosure is determined by the scope of the appended claims and should not be determined by the specific exemplary embodiments described below.

[0017] Will refer to Figures 1 to 3 and 4A to 4D The imaging device 100 according to the first exemplary embodiment is described. For example, the imaging device 100 is a monitoring camera installed on a wall or ceiling in a room. The monitoring camera can perform operations such as panning, tilting, and zooming.

[0018] Figure 1 The camera device 100 shown calculates a camera condition, and performs camera shooting under the calculated camera condition, under which the inside of a protection area (also called a privacy area) of the image data acquired by the camera cannot be recognized or is difficult to recognize. The inside of the protection area refers to an area within a predetermined area specified by a user. For example, the protection area is an area around a person's face, which can also be called a "privacy area". The camera condition according to the present exemplary embodiment is a combination of an exposure time and an analog gain (brightness analog gain). The exposure time is used to indicate the time for the camera unit (for example, the photodiode (PD) unit 107 described below) to shoot. The analog gain is used to amplify the analog brightness signal. The camera condition can also be called an "exposure condition". In the exemplary embodiment described below, the PD unit 107 is also referred to as a "PD unit 107". The camera condition according to the present exemplary embodiment is a camera condition that can protect the privacy of a subject (for example, a person).

[0019] The imaging device 100 includes an interface 101 , an imaging controller 102 , and an imaging sensor unit 103 .

[0020] The interface 101 acts as a receiving unit for receiving setting values ​​set by a user. The setting values ​​received by the interface 101 include thresholds and parameters, which are used as references when calculating the range of the protection area or the imaging condition. The interface 101 is, for example, a communication interface (I / F) unit or a touch panel. In the case where the interface 101 is a communication I / F unit, the user inputs (sends) the setting values ​​to the imaging device 100 from a remote location through wired or wireless communication. In the case where the interface 101 is a touch panel, the user inputs the setting values ​​to the imaging device 100 using the interface 101.

[0021] The camera controller 102 performs various types of image processing, such as white balance processing, noise reduction processing, and gamma correction processing, on the image data acquired by the PD unit 107. The camera controller 102 performs brightness gap correction on the image data to prevent a considerable brightness difference from occurring at the boundary between regions. The camera controller 102 also receives settings (setting values) from the user via the interface 101, and reflects the settings from the user on the image data.

[0022] The above-described image processing function does not always have to be installed on the imaging controller 102 , but all or part of the image processing function may be stacked and installed on the imaging sensor unit 103 .

[0023] The imaging sensor unit 103 includes an imaging condition decision unit 104, an analog gain control unit 105, an exposure time control unit 106, and a PD unit 107. The imaging sensor unit 103 is configured to control imaging conditions for each region on an imaging plane. The imaging sensor unit 103 acquires image data by performing imaging according to the imaging conditions.

[0024] The PD unit 107 receives incident light through an optical system such as a shutter and a lens. The PD unit 107 is composed of a color filter and an image sensor such as a complementary metal oxide semiconductor (CMOS). The PD unit 107 functions as an imaging unit for capturing an image of a subject.

[0025] The analog gain control unit 105 controls an analog gain value used as an imaging condition for each area.

[0026] The exposure time control unit 106 controls the exposure time used as the imaging condition for each area.

[0027] The imaging condition decision unit 104 receives image data from the PD unit 107 , receives user settings (setting values) from the imaging controller 102 , and decides imaging conditions for the next frame and subsequent frames.

[0028] Next, the imaging condition decision unit 104 that decides imaging conditions for acquiring an unrecognizable image or an image that is difficult to recognize will be described in detail.

[0029] The imaging condition determination unit 104 includes a brightness distribution generation unit 200 for generating a brightness distribution, an imaging condition calculation unit 201 for calculating imaging conditions based on the brightness distribution and user settings, and an imaging condition addition unit 202 for adding imaging conditions inside the protection area to imaging conditions outside the protection area. The imaging condition calculation unit 201 includes an imaging condition calculation unit 203 outside the protection area (hereinafter referred to as "the outside protection area calculation unit 203") and an imaging condition calculation unit 204 inside the protection area (hereinafter referred to as "the inside protection area calculation unit").

[0030] The brightness distribution generation unit 200 receives the image data acquired by the PD unit 107 and generates a brightness distribution according to the image data. According to the protection area set by the user, the brightness distribution generation unit 200 divides the generated brightness distribution into brightness distribution information inside the protection area and brightness distribution information outside the protection area. Then, the brightness distribution generation unit 200 sends the brightness distribution information inside the protection area to the protection area calculation unit 204, and sends the brightness distribution information outside the protection area to the protection area calculation unit 203. Then, each calculation unit in the protection area calculation unit 204 and the protection area calculation unit 203 calculates the imaging conditions of each area. The protection area calculation unit 204 and the protection area calculation unit 203 send the calculated imaging conditions to the imaging condition addition unit 202. The imaging condition addition unit 202 adds the received imaging conditions.

[0031] The protection area out-of-area calculation unit 203 receives the brightness distribution information of the area other than the set protection area from the brightness distribution generation unit 200, and determines whether imaging is performed under imaging conditions suitable for each area.

[0032] The range of the protection area is set by the interface 101 via the imaging controller 102. If it is determined that imaging is not performed under appropriate imaging conditions due to changes in brightness distribution (caused by, for example, changes in the brightness of the lighting in the room), the protection area out-calculation unit 203 calculates appropriate imaging conditions. The protection area out-calculation unit 203 may also have a function of calculating imaging conditions by detecting a dynamic area from acquired image data and predicting subsequent brightness changes.

[0033] The protected area calculation unit 204 receives the brightness distribution information inside the protected area (privacy area) from the brightness distribution generation unit 200. The protected area calculation unit 204 receives the setting values ​​of two thresholds from the interface 101 via the camera controller 102, and the two thresholds are used as references when calculating the camera conditions. The two thresholds include a threshold to be compared with the tail width of the brightness distribution curve (hereinafter referred to as the "tail width threshold") and a threshold to be compared with the peak value of the brightness distribution curve (hereinafter referred to as the "peak threshold"). The protected area calculation unit 204 also receives the setting values ​​of the parameters used to decide whether to perform Gaussian fitting from the interface 101 via the camera controller 102. The above setting values ​​are set to the protected area calculation unit 204. The peak threshold is a predetermined value related to the maximum brightness value.

[0034] The various functional units of the imaging sensor unit 103 (e.g., the analog gain control unit 105, the exposure time control unit 106, the brightness distribution generation unit 200, the imaging condition addition unit 202, the protection area outside calculation unit 203, and the protection area inside calculation unit 204) are installed on the imaging device 100 as dedicated software or dedicated hardware (such as an application-specific integrated circuit (ASIC) or a programmable logic array (PLA)). In the case where these functional units are installed as hardware, these functional units can be installed as dedicated hardware modules, and each hardware module is constructed by one or more of these functional units. In the case where the functional units are installed as software, a program for executing each functional unit is stored in a memory (not shown), and the program is appropriately read and executed by a processor (not shown).

[0035] Now refer to Figure 2 , Figure 3 and 4A to 4D A method for calculating imaging conditions executed by the protection area calculation unit 204 is described.

[0036] Figure 2 The brightness distribution 300 received by the calculation unit 204 from the brightness distribution generation unit 200 within the protection area is shown. The brightness distribution 300 is a bar graph before the brightness distribution curve is created by Gaussian fitting, which shows the brightness inside the protection area.

[0037] The protection region calculation unit 204 performs the function processing of Gaussian fitting 301 based on the parameter for deciding whether to perform Gaussian fitting on the brightness distribution 300. The protection region calculation unit 204 does not perform the function processing according to the parameter for deciding whether to perform Gaussian fitting. The function processing does not always have to be Gaussian fitting.

[0038] When performing the function processing of Gaussian fitting 301, the protected area calculation unit 204 obtains (extracts) information about the peak height 302 of the fitting function (brightness distribution curve). The peak height 302 represents the brightness value at the maximum pixel distribution of the image data. The protected area calculation unit 204 also obtains information about the tail width 304 and the peak position 305 of the fitting function that can be obtained at the position (height) 303 corresponding to 10% of the peak height 302. The protected area calculation unit 204 also obtains the maximum brightness value 306 and information about the tail 307 and the tail 308 of the fitting function. The tail 307 and the tail 308 represent the brightness at 0% peak height of the fitting function. The tail 307 is the tail at high brightness, and the tail 308 is the tail at low brightness. The protected area calculation unit 204 calculates the imaging conditions in which the image data in the protected area cannot be recognized or is difficult to recognize based on the above-mentioned multiple information and the tail width threshold and the peak threshold set by the user. For 8-bit image data, the maximum brightness value 306 is 255. In the exemplary embodiments described below, the tail 307 may be referred to as a "right tail" and the tail 308 may be referred to as a "left tail".

[0039] The imaging condition under which the image data in the protection area cannot be recognized or is difficult to recognize refers to the following imaging condition: the brightness value (pixel value) of the pixel of the image data in the protection area falls within a predetermined range. This imaging condition refers to, for example, the following imaging condition: the brightness values ​​of all pixels in the protection area fall near the minimum value (underexposure state) or near the maximum value (overexposure state).

[0040] Figure 3 2 is a flowchart showing a process for calculating an imaging condition under which image data in a protection area cannot be recognized or is difficult to recognize, and the process is executed by the protection area calculation unit 204.

[0041] In S400 , the PD unit 107 acquires image data by performing imaging.

[0042] In S401 , the luminance distribution generation unit 200 receives the image data acquired by the PD unit 107 from the PD unit 107 and generates a luminance distribution.

[0043] In S402, the protection area calculation unit 204 receives the brightness distribution information of the protection area from the brightness distribution generation unit 200. The protection area calculation unit 204 also receives the tail width threshold (set value) and the peak threshold (set value) for calculating the imaging condition from the imaging controller 102, and also receives the parameter (set value) for deciding whether to perform Gaussian fitting.

[0044] In S403, the protected area calculation unit 204 offsets the brightness distribution received from the brightness distribution generation unit 200 based on the imaging condition applied when the image is acquired by the PD unit 107. This offset is performed to prevent the following problem from occurring: the imaging condition calculated from the received brightness distribution information cannot be used as the imaging condition in which the image data in the protected area cannot be recognized or is difficult to recognize. For example, this problem occurs when the imaging condition inside the protected area is calculated relative to the image data acquired under the imaging condition in which the exposure time and the analog gain value inside the protected area are maximized. In this case, even if the imaging condition in which the exposure time and the analog gain value are increased is calculated, the calculation result cannot be used as the imaging condition. Therefore, by offsetting the brightness distribution based on the image capturing condition applied when the image is captured, the calculation result can be used as the imaging condition. In addition, since the offset of the brightness distribution in S403 is performed based on the imaging condition applied when the image is acquired by the PD unit 107, there is also a case where the brightness distribution is not offset in S403.

[0045] In S404, the protection area calculation unit 204 refers to the setting parameters for deciding whether to perform Gaussian fitting, which have been received through the interface 101 via the camera controller 102, and decides (determines) whether to perform Gaussian fitting. If Gaussian fitting is to be performed ("Yes" in S404), the process proceeds to S405. If Gaussian fitting is not to be performed ("No" in S404), the process proceeds to S406.

[0046] In S405 , the protection region calculation unit 204 performs Gaussian fitting processing on the received brightness distribution, and extracts information (eg, peak height 302 ) for calculating imaging conditions.

[0047] In S406, the protection region calculation unit 204 calculates a height 303 corresponding to 10% of the peak height 302 using the peak height 302 obtained by the Gaussian fitting process. Then, the protection region calculation unit 204 compares the width of the height 303 (the tail width 304) with the tail width threshold value set by the user via the interface 101. More specifically, the protection region calculation unit 204 determines whether the tail width 304 is greater than the tail width threshold value (i.e., the set value). The tail width threshold value may also be referred to as a predetermined range of pixel brightness values.

[0048] When the tail width 304 is equal to or less than the tail width threshold value (No in S406), the brightness value of the pixel of the image data in the protection area falls within the predetermined range (i.e., the predetermined brightness range). This means that the image data in the protection area cannot be recognized or is difficult to be recognized. In this case, the imaging condition is not changed, so that the processing proceeds to S410. For example, the above state corresponds to the following description Figure 4B The status shown in .

[0049] In contrast, when the tail width 304 of the fitting function is greater than the tail width threshold value ("Yes" in S406), the brightness value of the pixel of the image data in the protection area does not fall within the predetermined brightness range. This means that the image data in the protection area can be identified. In this case, the protection area calculation unit 204 advances the processing to S407 to S409 to calculate and set the imaging conditions under which the image data in the protection area cannot be identified or is difficult to identify.

[0050] In S407, the protection region calculation unit 204 compares the peak position 305 acquired by the Gaussian fitting process with the set peak threshold value (set value). More specifically, the protection region calculation unit 204 determines whether the peak position 305 is greater than the peak threshold value set by the user via the interface 101. If the determination result acquired in S407 is "Yes", the process proceeds to S408. If the determination result acquired in S407 is "No", the process proceeds to S409.

[0051] In S408, the protected area calculation unit 204 sets (updates) the imaging condition so that the brightness distribution is concentrated on the right tail 307 (high brightness) of the fitting function. Then, the protected area calculation unit 204 sends the set imaging condition to the imaging condition addition unit 202. In the case where the brightness value of the right tail 307 is greater than the maximum brightness value 306, the protected area calculation unit 204 updates the imaging condition by which the brightness distribution is concentrated on the maximum brightness value 306 or a value near it.

[0052] In S409, the protected area calculation unit 204 sets (updates) the imaging condition so that the brightness distribution is concentrated on the left tail 308 (low brightness) of the fitting function. Then, the protected area calculation unit 204 sends the set imaging condition to the imaging condition addition unit 202. In the case where the brightness value of the left tail 308 is less than 0, the protected area calculation unit 204 updates the imaging condition by which the brightness distribution is concentrated on a value at or near the brightness value of 0.

[0053] In S410, the imaging condition adding unit 202 adds the imaging conditions inside and outside the protection area calculated by the imaging condition calculating unit 201 to obtain the imaging conditions of the entire image (integration of imaging conditions). Regarding the initial imaging conditions, since the imaging data has not yet been input therein, the imaging conditions can be set by the user via the interface 101.

[0054] In S411, among the imaging conditions calculated by the imaging condition decision unit 104, the analog gain value and the exposure time are sent and applied to the analog gain control unit 105 and the exposure time control unit 106, respectively. For example, in a case where the image of the protection area (privacy area) is to be overexposed, when the PD unit 107 captures the subject image, the exposure time control unit 106 extends the exposure time. Alternatively, the analog gain control unit 105 reduces the gain when the PD unit 107 captures the subject image, so that the image of the protection area can be overexposed. In addition, when the PD unit 107 captures the subject image, the exposure time control unit 106 extends the exposure time and the analog gain control unit 105 reduces the gain, so that the image of the protection area can also be overexposed.

[0055] In the case where the image of the protection area is to be underexposed, for example, when the PD unit 107 captures the subject image, the exposure time control unit 106 shortens the exposure time. Alternatively, the analog gain control unit 105 increases the gain when the PD unit 107 captures the subject image, so that the image of the protection area can be underexposed. Furthermore, when the PD unit 107 captures the subject image, the exposure time control unit 106 shortens the exposure time and the analog gain control unit 105 increases the gain, so that the image of the protection area can also be underexposed.

[0056] In S412, the imaging controller 102 determines whether an instruction to end the process is sent from the user. When the determination result is "No" ("No" in S412), the process returns to S400. In the case where an instruction to end the process is sent ("Yes" in S412), the imaging sensor unit 103 ends the process. Figure 3 Processing shown.

[0057] 4A to 4D A diagram showing an example of an image acquired by the imaging device 100 when a protection area is set within the imaging area of ​​the PD unit 107 and imaging conditions are set for each area by the method according to the present exemplary embodiment. 4A to 4D In FIG. 1 , persons X, Y, and Z are captured as subject images.

[0058] Figure 4A An image 500 is shown that is acquired before or immediately after a protection region is set for the image.

[0059] The protection areas 501, 502, and 503 are specified (set) by the user. Figure 4A In the image 500, from the perspective of protecting the privacy of persons X, Y, and Z, protection areas 501, 502, and 503 are set to include the facial areas (facial contours) of persons X, Y, and Z. Figure 4AIn the illustrated state, facial images 505, 506, and 507 of subjects X, Y, and Z are respectively identifiable within protection areas 501, 502, and 503. In the present exemplary embodiment, each of subjects X, Y, and Z wears a hat, so the "face of the subject" and the "facial outline" include the hat.

[0060] By adopting the method according to the first exemplary embodiment, imaging conditions are calculated based on the image data of the protection areas 501, 502, and 503, and the calculated imaging conditions are applied to Figure 4A As a result, we can obtain Figure 4B Image 510 is shown. Figure 4B The images 511, 512, and 513 in FIG. 5 are images of the protection areas 501, 502, and 503. As a result of controlling the imaging conditions inside the protection areas 501, 502, and 503, the faces of the subjects X, Y, and Z in the protection areas 501, 502, and 503 are unrecognizable or difficult to recognize.

[0061] Thereafter, imaging is continuously performed by the imaging apparatus 100. In other words, it is assumed that monitoring of persons X, Y, and Z is continuously performed while privacy protection of image data acquired by the PD unit 107 is performed by the method according to the first exemplary embodiment.

[0062] If dispersion of brightness distribution within the protection areas 501, 502, and 503 is included in the tail width threshold (set value), image data within the protection areas 501, 502, and 503 cannot be recognized or is difficult to recognize, so that the imaging device 100 does not change the imaging conditions.

[0063] If the dispersion of the brightness distribution within the protection areas 501, 502, and 503 is not included in the tail width threshold, it is necessary to change (set again) the imaging conditions. Figure 4C The image 520 in FIG. 1 is an example of an image when the imaging conditions need to be changed. For example, when the lighting in the room becomes brighter Figure 4B In the case of brighter lighting, the facial images 521, 522, and 523 within the protection areas 501, 502, and 503 are recognizable, as shown in FIG. Figure 4C In this case, the imaging conditions in which the facial images 521, 522, and 523 within the protection areas 501, 502, and 503 cannot be recognized or are difficult to recognize are calculated by the method described in the first exemplary embodiment, thereby changing (updating) the imaging conditions accordingly.

[0064] Figure 4D The image 530 is updated by the first exemplary method according to Figure 4CThe imaging conditions of the image 520 in the image are obtained when the PD unit 107 captures the images of the objects X, Y and Z. By updating the imaging conditions of the protection areas 501, 502 and 503, such as Figure 4D As shown in images 531, 532 and 533, Figure 4C The facial images 521, 522 and 523 that are recognizable in the image are not recognizable or are difficult to be recognized in the protection areas 501 to 503.

[0065] According to the present exemplary embodiment, image data that cannot be recognized or is difficult to recognize in a predetermined area (protected area) can be acquired by the imaging sensor unit 103 that can acquire image data under different imaging conditions for each area. As a result, image data with high security can be acquired when performing imaging. According to the imaging device 100 of the present exemplary embodiment, since privacy protection processing has been performed on the acquired image data when acquiring the image data (i.e., when performing imaging), privacy can be protected with high security.

[0066] In the present exemplary embodiment, the determination in S407 is performed when the determination result obtained in S406 is "Yes", and the process proceeds to S408 or S409 based on the determination result obtained in S407. However, when the determination result obtained in S406 is "Yes", the process may proceed to S408 or S409 without performing the determination in S407.

[0067] Alternatively, although the protection area is set to include all faces of persons X, Y, and Z, the protection area may be set to include only the face of a specific person (i.e., person X, Y, or Z) or the faces of persons X and Y (or persons X and Z or persons Y and Z).

[0068] The imaging device 100 may be a device other than a surveillance camera. Examples of the device include a web camera, a digital camera, a camera module mounted on a smartphone, and a camera module mounted on a tablet terminal.

[0069] In the case where a graphical user interface (GUI) such as a touch panel is used as the interface 101 , the interface 101 may be separated from the imaging device 100 .

[0070] Now refer to Figure 5 An image pickup apparatus 100A according to a second exemplary embodiment will be described. The same reference numerals are applied to Figure 1 The constituent elements of the image pickup device 100 of the first exemplary embodiment shown are similar to the constituent elements.

[0071] The configuration of the imaging apparatus 100A is substantially the same as that of the imaging apparatus 100, except that the imaging apparatus 100A includes an in-protection area imaging condition setting unit 600 (hereinafter referred to as "in-protection area setting unit 600") instead of the in-protection area calculation unit 204. The imaging apparatus 100A includes only the out-of-protection area calculation unit 203 as an imaging condition calculation unit. The imaging apparatus 100A can also transmit information from the imaging controller 102 to the interface 101 as indicated by arrow T.

[0072] In the present exemplary embodiment, the interface 101 of the imaging device 100A receives not only the range of the protection area but also the imaging conditions inside the protection area from the user. The imaging condition decision unit 104 receives the acquired image data from the PD unit 107, and receives the setting value regarding the range of the protection area from the user via the interface 101. The protection area out-calculation unit 203 receives the protection area range from the interface 101 via the imaging controller 102, and receives the brightness distribution from the brightness distribution generation unit 200. The protection area in-set unit 600 receives the protection area range and the setting value of the imaging conditions inside the protection area from the interface 101 via the imaging controller 102.

[0073] A method for setting the imaging conditions inside and outside the protection area will now be described. The outside-protection-area calculation unit 203 calculates the imaging conditions outside the protection area based on the brightness distribution received from the brightness distribution generation unit 200. The user sets the imaging conditions inside the protection area in the inside-protection-area setting unit 600 via the interface 101. Thereafter, the imaging conditions inside and outside the protection area are added by the imaging condition addition unit 202.

[0074] The imaging controller 102 according to the present exemplary embodiment determines whether a facial image within a protection area is recognizable in image data acquired under the imaging conditions added by the imaging condition adding unit 202. In other words, the imaging controller 102 determines whether the privacy of the subject can be protected by the imaging conditions received by the interface 101. This determination is performed based on the brightness distribution of the image data.

[0075] In the case where the facial image within the protection area of ​​the image data acquired under the imaging condition added by the imaging condition adding unit 202 is recognizable, the imaging device 100A notifies the user that the facial image is recognizable. This notification is sent from the imaging controller 102 to the user via the interface 101 (indicated by arrow T). In other words, if it is determined that the imaging condition received through the interface 101 cannot protect the privacy of the subject, the imaging controller 102 outputs information indicating that privacy protection cannot be performed to the outside via the interface 101.

[0076] The camera 100A may include an output unit (e.g., an image display unit, a speaker, or a light emitting unit) that outputs an image, a character, a sound, or a light. Instead of (or including) sending a notification indicated by an arrow T to a user, the camera 100A may output an alarm image showing content similar to that of the notification from the output unit. When a GUI is used as the interface 101, the above-mentioned output unit may be included in the interface 101.

[0077] The imaging apparatus 100A may not have a function of determining whether a facial image within a protection area is recognizable in image data acquired under the imaging conditions added by the imaging condition adding unit 202. In this case, the notification indicated by the arrow T is not sent to the user.

[0078] Similar to the imaging device 100 according to the first exemplary embodiment, the imaging device 100A according to the present exemplary embodiment can acquire image data that is unrecognizable or difficult to recognize in a protected area by using the imaging sensor unit 103 capable of acquiring image data under different imaging conditions for each area, thereby acquiring image data with high security when performing imaging.

[0079] Now refer to Figure 1 and Figure 6 A third exemplary embodiment is described.

[0080] The configuration of the image pickup device according to the present exemplary embodiment is similar to the configuration of the image pickup device 100 described in the first exemplary embodiment, and therefore, with respect to the configuration of the image pickup device according to the present exemplary embodiment, reference will be made to the following description. Figure 1 In the present exemplary embodiment, the imaging condition decision unit 104 receives the setting of the imaging condition inside the protection area from the interface 101 via the imaging controller 102 .

[0081] Figure 6 FIG. 4 is a flowchart showing a process for deciding (calculating) imaging conditions inside a protection area according to the present exemplary embodiment. The same reference numerals (ie, S400, S401, and S412) are applied to the same processing as those in FIG. Figure 3 The processing is similar to the processing shown in the flowchart. Figure 6 The processing of S400 to S406 in Figure 3 Similar.

[0082] In the present exemplary embodiment, if the determination result in S406 is “Yes”, the process proceeds to S700 .

[0083] In S700, the protected area calculation unit 204 determines whether the imaging condition set via the imaging controller 102 through the interface 101 is set as the imaging condition inside the protected areas 501, 502, and 503. Specifically, the protected area calculation unit 204 determines whether the facial image inside the protected areas 501, 502, and 503 is unrecognizable or difficult to recognize when imaging is performed under the set imaging condition, and decides whether to update the imaging condition to the imaging condition set via the imaging controller 102 through the interface 101. If the image data inside the protected areas 501, 502, and 503 is unrecognizable or difficult to recognize ("Yes" in S700), the processing proceeds to S410 so that the imaging condition is updated to the set imaging condition. The processing from S410 to S412 is the same as that of S410. Figure 3 The processing shown is similar. The protection area calculation unit 204 according to the present exemplary embodiment determines the imaging condition in addition to calculating the imaging condition, and thus the protection area calculation unit 204 can be referred to as a "protection area imaging condition decision unit."

[0084] In contrast, if the image data within the protection areas 501, 502, and 503 is recognizable (No in S700), the protection area calculation unit 204 does not update the imaging condition to the set imaging condition, and the process proceeds to S407 to calculate an imaging condition that makes the image data within the protection areas 501, 502, and 503 unrecognizable or difficult to recognize. The processes of S407 to S412 are the same as those of Figure 3 The processing shown is similar.

[0085] According to the present exemplary embodiment, similar to the first exemplary embodiment, image data that cannot be recognized or is difficult to recognize in the protection area can be acquired by the imaging sensor unit 103 that can acquire image data under different imaging conditions for each area. It is thereby possible to acquire image data with high security when performing imaging.

[0086] <Camera System Structure>

[0087] Figure 7 A block diagram showing a configuration example of an image pickup system according to a fourth exemplary embodiment.

[0088] The camera system includes a camera 1 and a graphical user interface (GUI) 2. The camera 1 captures an image of an area set by a user. The GUI 2 is executed by an information processing device such as a personal computer to perform display of information and acceptance of instructions input by the user.

[0089] The imaging device 1 includes: an imaging lens 10, which is configured to converge incident light to form an image; a lens driving unit 11, which is configured to change the zoom ratio of the imaging lens 10; an image sensor 12, which is configured to convert the formed image into an analog image signal; and an area exposure control unit 13, which is configured to perform setting of exposure conditions of the image sensor 12.

[0090] The camera 1 further includes an analog front end (AFE) 14 and an image processing unit 15. The AFE 14 is configured to amplify and remove noise from an image signal output from the image sensor 12. The image processing unit 15 is configured to perform various types of image processing on image data output from the AFE 14.

[0091] The imaging device 1 further includes a mask generation unit 16 and a mask data storage unit 17. The mask generation unit 16 is configured to generate a mask based on setting information (coordinate position) of a privacy area set by the imaging control unit 21. The mask data storage unit 17 is configured to store setting information of the mask generated by the mask generation unit 16.

[0092] The GUI 2 includes an image display unit 20 configured to display an image output from the image processing unit 15, and an imaging control unit 21 configured to set a privacy area of ​​the imaging device 1 and change the imaging area (i.e., field of view). The user is allowed to set a privacy area in an image through the imaging control unit 21, perform mask processing on the privacy area, and check the result of the mask set thereto through the image display unit 20.

[0093] An imaging lens (imaging optical system) 10 of the imaging device 1 includes a focus lens and a zoom lens for converging incident light to form an image.

[0094] The lens driving unit 11 drives the zoom lens of the imaging lens 10 to change the zoom magnification.

[0095] The image sensor 12 performs photoelectric conversion on the subject image (optical image) acquired via the imaging lens 10 and outputs an image signal. The image sensor 12 sets exposure conditions such as exposure time and analog gain for each predetermined imaging area (i.e., each pixel or a plurality of pixels). Hereinafter, an area composed or constructed of a single pixel or a plurality of pixels while controlling the exposure conditions will be referred to as a uniform exposure area.

[0096] The regional exposure control unit 13 controls the exposure by setting the exposure conditions of each uniform exposure area of ​​the image sensor 12. For example, if the amount of light entering the uniform exposure area is increased by extending the exposure time of the uniform exposure area, the image will be brighter. In contrast, if the amount of light entering the uniform exposure area is reduced by shortening the exposure time, the image will be darker. Generally, when performing video recording, in order to improve the visibility of the subject, exposure conditions for applying optimal exposure are applied to each pixel area according to the brightness value. In the present exemplary embodiment, a hidden target area (e.g., a privacy area) is set as a uniform exposure area. Then, mask processing is performed in a manner that controls the exposure by excessively extending or shortening the exposure time, so that the image data in the hidden target area enters a so-called overexposed state or underexposed state.

[0097] The AFE 14 includes an analog circuit configured to amplify the image signal output from the image sensor 12 and remove amplification noise and reset noise included in the image signal. The AFE 14 also includes an analog-to-digital (A / D) conversion unit configured to convert an analog signal (image signal) into a digital signal after performing predetermined analog processing on the analog signal.

[0098] The image processing unit 15 performs various types of image processing on the digital signal (image data) transmitted from the AFE 14 and outputs the digital signal.

[0099] The mask generation unit 16 generates a mask based on the setting information (coordinate position) of the privacy area set by the imaging control unit 21 .

[0100] The mask data storage unit 17 stores setting information of the mask generated by the mask generation unit 16. The mask data storage unit 17 is configured by a nonvolatile storage device such as a flash read-only memory (ROM). If the imaging apparatus 1 includes the mask data storage unit 17, the setting information of the mask can be stored in the imaging apparatus 1. The user can request the lens driving unit 11 to drive the imaging lens 10 by performing an operation of changing the field of view via the imaging control unit 21, thereby being able to change the imaging area of ​​the imaging apparatus 1.

[0101] In the above exemplary embodiment, for the purpose of description, the field of view is changed by driving the imaging lens 10. However, the field of view may be changed by controlling a driving mechanism (not shown) for panning or tilting the imaging device 1. Although the area exposure control unit 13, the image processing unit 15, and the mask generation unit 16 have been described separately, these units may also be included in a single CPU.

[0102] <Mask Setting Processing>

[0103] Figure 8 is a flowchart of an example of mask setting processing. Fig. 9A and Fig. 9B is a diagram showing an example of an image captured by the imaging device 1 and displayed on the GUI 2 . Fig. 9A shows an example of an image before masking is performed, Fig. 9B An example of an image after performing mask processing is shown.

[0104] In the following, reference will be made to Figure 8 , Fig. 9A and Fig. 9B An example of a process for setting a privacy area is described.

[0105] exist Figure 8 After the processing in S1 is started, the user displays the captured image 30 ( Fig. 9A ) sets a privacy area (hereinafter referred to as "mask setting area 31") on which mask processing is to be performed. At this time, the user sets the position (coordinates) and size of the area on which mask processing is to be performed. In S2, the mask generation unit 16 receives the mask generation request from the camera control unit 21, and obtains the brightness value information of the mask setting area 31 from the image processing unit 15. In S3, the mask generation unit 16 further determines whether the acquired brightness value of the mask setting area 31 is equal to or greater than a specific value (predetermined threshold value).

[0106] If the brightness value is equal to or greater than a specific value ("Yes" in S3), the processing proceeds to S4. In S4, the mask generation unit 16 requests the regional exposure control unit 13 to set the pixels of the image sensor 12 corresponding to the mask setting area 31 to a uniform exposure area. In order to overexpose the image in the set uniform exposure area, the mask generation unit 16 sends a request for setting an exposure condition with a brightness brighter than a predetermined value or equal to a predetermined value, for example, a condition for extending the exposure time (long exposure). In other words, the mask generation unit 16 controls the exposure of the uniform exposure area to be brighter than a predetermined value or equal to a predetermined value. In contrast, if the brightness value is less than a specific value ("No" in S3), the processing proceeds to S5. In S5, in order to underexpose the mask setting area 31, the mask generation unit 16 sends a request for setting an exposure condition with a brightness darker than a predetermined value or equal to a predetermined value, for example, a condition for shortening the exposure time (short exposure) of the uniform exposure area. In other words, the mask generation unit 16 controls the exposure of the uniform exposure area to be darker than a predetermined value or equal to a predetermined value.

[0107] The exposure condition can be changed by changing the exposure gain (analog gain value) instead of changing the exposure time. Alternatively, the exposure condition can be changed by changing both the exposure time and the exposure gain.

[0108] By performing imaging according to the exposure conditions set as above, the image data in the mask setting area 31 is overexposed or underexposed, so that the image of the mask setting area 31 with the mask is as shown in FIG. Fig. 9B The image is displayed on the image display unit 20. In S6, the mask generation unit 16 stores the setting information (coordinate position) of the mask setting area 31 in the mask data storage unit 17, and ends the processing. Subsequent imaging is performed by using the setting information of the mask setting area 31 stored in the mask data storage unit 17.

[0109] Through the above-mentioned processing, when the image processing unit 15 receives the image data, the image data of the area set as the privacy area is already overexposed or underexposed, so that it is not necessary to perform the processing for superimposing the mask image. Therefore, it is possible to reduce the processing load of the image processing unit 15. In addition, since the image data of the mask setting area 31 provided to the image processing unit 15 is already overexposed or underexposed, the security of the image data can be improved.

[0110] Furthermore, in the above exemplary embodiment, overexposed image data or underexposed image data is acquired by determining the brightness value of the mask setting area 31. However, the present exemplary embodiment is not limited thereto, and the exposure of the uniform exposure area corresponding to the privacy area may be uniformly set to a long exposure or a short exposure.

[0111] As described above, according to the present exemplary embodiment, when an image having a privacy area hidden in an imaging area is to be generated, the processing load can be reduced.

[0112] In the fourth exemplary embodiment described above, mask processing for hiding the privacy area is performed by changing the exposure condition of the uniform exposure area. In the fifth exemplary embodiment, mask processing is performed by a combination of methods such as a method of hiding the privacy area by changing the exposure condition of the uniform exposure area and a method of hiding the privacy area by superimposing a mask image. The camera system configuration and processing for changing the exposure condition of the uniform exposure area are similar to those described in the fourth exemplary embodiment. Therefore, hereinafter, only the mask processing performed on the privacy area will be described.

[0113] <Mask Setting Processing>

[0114] The user performs setting of the privacy area through the imaging control unit 21 of the GUI 2. The mask generation unit 16 performs mask processing on the image based on the setting information of the privacy area.

[0115] At this time, the mask generation unit 16 can select a hiding method for each privacy area from the following methods, namely, (1) a method for hiding the privacy area by superimposing a mask image on the image data by the image processing unit 15, and (2) a method for hiding the privacy area by controlling the exposure time of the uniform exposure area of ​​the image sensor 12 by the area exposure control unit 13.

[0116] In the former method (1), since the image processing unit 15 generates a mask image and superimposes it on a position corresponding to the privacy area, the processing load of the image processing unit 15 is increased. Therefore, the number of areas that can be set is limited. In contrast, in the latter method (2), the image data is input to the image processing unit 15 after the mask processing has been performed on the privacy area. Therefore, since the image processing unit 15 does not superimpose the mask image, the processing load is reduced. However, in the case of changing the imaging area of ​​the imaging device 1, the exposure time is changed after the privacy area is calculated again. As a result, there is a long time lag before the exposure time is changed. Therefore, compared with the former method (1), the tracking ability of the imaging device 1 to track movement (change of imaging area) is poor. In the present exemplary embodiment, its mask type is switched according to whether the imaging device 1 moves.

[0117] <Processing for Changing Mask Type>

[0118] Fig.10 is a flowchart illustrating an example of a process for changing (switching) a mask type.

[0119] Hereinafter, the processing will be described for the case where the mask type is switched according to whether the image pickup device 1 is moved.

[0120] In S11, similar to the above-described processing according to the fourth exemplary embodiment, the user sets the privacy area through the imaging control unit 21. The mask generation unit 16 sends a request for superimposing a mask image to the image processing unit 15 based on the setting information about the privacy area received via the imaging control unit 21. In response to the request, in S12, the image processing unit 15 generates an image superimposed with the mask image. In S13, the mask generation unit 16 stores the setting information about the set mask in the mask data storage unit 17. In S14, the mask generation unit 16 determines, for example, whether the imaging area has not changed or whether the imaging device 1 has not moved for a certain period of time.

[0121] In the case of changing the imaging area or moving the imaging device 1 (No in S14), the process proceeds to S15. In S15, the mask generation unit 16 acquires a change in the captured image, such as the amount of movement, and calculates the position of the changed privacy area based on the acquired amount of movement and the setting information about the mask stored in the mask data storage unit 17. Thereafter, the process returns to S12, and the mask generation unit 16 requests the image processing unit 15 to superimpose the mask image again on the area corresponding to the calculated privacy area. In S13, the mask generation unit 16 stores the setting information of the mask in the mask data storage unit 17. Herein, for the purpose of description, the mask image is set again according to the movement. However, by changing the position or shape of the mask image according to the amount of movement, the mask image can be constantly superimposed on the area corresponding to the privacy area. In addition, the change of the imaging area and the detection and acquisition of the movement or the amount of movement of the imaging device 1 can be performed based on, for example, a request for changing the field of view received from the imaging control unit 21 and information about the difference between the images acquired by the image processing unit 15. Alternatively, the above-described detection and acquisition may be performed based on output values ​​of various sensors (not shown) that can detect movement of the imaging device 1 .

[0122] In S14, if the imaging area has not changed or the imaging device 1 has not moved within a certain period of time ("Yes" in S14), the process proceeds to S16. In S16, the mask generation unit 16 sets the pixels of the image sensor 12 corresponding to the privacy area as a uniform exposure area, and changes the exposure time to overexpose or underexpose the image. Thereafter, in S17, the mask generation unit 16 stops the superimposition process of the mask image performed by the image processing unit 15. In S18, the mask generation unit 16 stores the setting information of the uniform exposure area in the mask data storage unit 17 and ends the process.

[0123] In addition, when the movement of the camera device 1, the change of the camera area, etc. is detected again, the mask generation unit 16 recalculates the privacy area based on the movement amount and the setting information of the uniform exposure area stored in the mask data storage unit 17, and switches the mask type through the image processing unit 15 to superimpose the mask image on the corresponding area.

[0124] Through the above-described processing, when the imaging area is not changed or the imaging device 1 is not moved, the processing load caused by the processing for superimposing the mask image on the privacy area can be reduced.

[0125] <Another Example of Mask Type Changing Process>

[0126] Fig.11 is a flowchart illustrating another example of a process for changing (switching) a mask type.

[0127] Hereinafter, the exemplary embodiment will be described with respect to processing for switching mask types performed when the number of mask images that can be superimposed by image processing unit 15 exceeds an upper limit value.

[0128] In S21, similar to Fig.10 In the process shown in FIG. 2 , the user sets a privacy area through the camera control unit 21. In S22, the mask generation unit 16 receives a request for setting a privacy area, and acquires information about the number of mask images superimposed on the image data from the mask data storage unit 17. In S23, the mask generation unit 16 determines whether the number of superimposed mask images exceeds a predetermined value related to the processing load. For example, the mask generation unit 16 determines whether the number of superimposed mask images exceeds the upper limit number (maximum number) of mask images that can be superimposed by the image processing unit 15.

[0129] If the number of superimposed mask images does not exceed the upper limit value ("Yes" in S23), the process proceeds to S24. In S24, the mask generation unit 16 requests the image processing unit 15 to superimpose the mask image on the area corresponding to the set privacy area. If the number of superimposed mask images exceeds the upper limit value ("No" in S23), the process proceeds to S25. In S25, the mask generation unit 16 sets the pixels of the image sensor 12 corresponding to the set privacy area as a uniform exposure area, and changes the exposure time to overexpose or underexpose the image data. In S26, the mask generation unit 16 stores the set mask information in the mask data storage unit 17 and ends the process.

[0130] In the above exemplary embodiment, the method of performing the mask processing is selected from the following methods: a method of hiding the privacy area by changing the exposure condition of the uniform exposure area, and a method of hiding the privacy area by superimposing the mask image according to the number of superimposed mask images. However, the exemplary embodiment is not limited thereto, and when the privacy area is set by the user, the user can select the type of mask processing via the GUI 2.

[0131] With such a configuration, the user is allowed to set a number of privacy areas greater than the number of mask images that the image processing unit 15 can superimpose.

[0132] As described above, according to the present exemplary embodiment, the processing load caused by the process for generating an image having a privacy area hidden in an imaging area can be reduced while suppressing an increase in the size of the device.

[0133] Some embodiments may also be implemented by processing performed in the following manner: a program for implementing one or more functions according to the above exemplary embodiments is supplied to a system or device via a network or storage medium, and one or more processors in the system or device read and execute the program. In addition, some embodiments may also be implemented using a circuit such as an application specific integrated circuit (ASIC) that implements one or more functions. The above program is a computer-readable program associated with the contents of the flowcharts shown in the accompanying drawings described in the exemplary embodiments.

[0134] Although the present disclosure has described exemplary embodiments, it is to be understood that some embodiments are not limited to the disclosed exemplary embodiments.The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.

Claims

1. A camera device, comprising: An imaging unit configured to set exposure conditions for each area on an imaging plane; a setting unit configured to set a privacy area in an image captured by the imaging unit; a decision unit configured to decide the exposure condition of the privacy area set by the setting unit to be an exposure condition capable of protecting the privacy of the subject in the privacy area; as well as A control unit is configured to perform control so that the imaging unit captures an image of the subject based on the exposure condition determined by the determination unit.

2. The camera device according to claim 1, further comprising: a mask unit configured to perform processing for superimposing a predetermined image on the privacy area set by the setting unit, The control unit controls the imaging unit to perform any one of the control performed by the control unit and the processing performed by the mask unit. 3 . The imaging device according to claim 1 , wherein, when a brightness value of the image in the privacy area is greater than or equal to a predetermined threshold, the determination unit determines that the exposure condition of the privacy area has a value greater than or equal to a predetermined value.

4. The imaging device according to claim 1, wherein: The decision unit decides that the exposure condition of the privacy area has a value less than or equal to the predetermined value, in a case where the brightness value of the image in the privacy area is less than or equal to a predetermined threshold value.

5. The imaging device according to claim 1, wherein: The determination unit determines an exposure condition of the privacy area based on a brightness distribution of the image data of the privacy area.

6. The imaging device according to claim 1, wherein: The determination unit determines an exposure condition of the privacy area based on a maximum brightness value or a minimum brightness value of pixels of the image data of the privacy area.

7. The imaging device according to claim 1, wherein: When the maximum luminance value of a pixel of the image data of the privacy area does not exceed a predetermined value, the decision unit decides an exposure condition such that the luminance value of the pixel is a minimum value or a value near the minimum value that the luminance value can take.

8. The imaging device according to claim 1, wherein: When the maximum luminance value of a pixel of the image data of the privacy area exceeds a predetermined value, the decision unit decides an exposure condition such that the luminance value of the pixel is a maximum value or a value near the maximum value that the luminance value can take.

9. The camera device according to claim 2, further comprising: a detection unit configured to detect a change in the image captured by the camera unit, The mask unit performs processing for superimposing a predetermined image on the privacy area set by the setting unit based on a change in the image captured by the imaging unit and detected by the detection unit.

10. The imaging device according to claim 2, wherein: The control unit controls the imaging unit to execute any one of the control executed by the control unit and the processing executed by the mask unit for each privacy area.

11. The camera device according to claim 1, further comprising: a receiving unit configured to receive an exposure condition from outside, When the receiving unit receives the exposure condition, the control unit performs control so that the imaging unit captures an image of the subject based on the received exposure condition.

12. The camera device according to claim 11, further comprising: a determination unit configured to determine whether the privacy of the subject can be protected by the exposure condition received by the receiving unit, Only when the determination unit determines that the privacy of the subject can be protected by the exposure condition received by the receiving unit, the control unit performs control so that the camera unit captures the image of the subject under the exposure condition received by the receiving unit.

13. The imaging device according to claim 1, wherein: The exposure condition includes at least one of an exposure time and an analog gain of the imaging unit.

14. A method for controlling a camera device, comprising: Capturing an image by setting exposure conditions for each area on the imaging surface; Set privacy zones in captured images; determining the exposure condition of the set privacy area to be an exposure condition capable of protecting the privacy of the subject in the privacy area; as well as Control is performed so that an image of the subject is captured based on the determined exposure conditions. 15 . The method for controlling an image pickup apparatus according to claim 14 , further comprising performing control so that any one of the control and processing for superimposing a predetermined image on a privacy area is performed. 16 . The method for controlling an imaging device according to claim 14 , further comprising determining that the exposure condition of the privacy area has a value greater than or equal to a predetermined value when a brightness value of the image of the privacy area is greater than or equal to a predetermined threshold. 17 . The method for controlling an imaging device according to claim 14 , further comprising determining that the exposure condition of the privacy area has a value less than or equal to a predetermined value when a brightness value of the image in the privacy area is less than or equal to a predetermined threshold. 18 . The method for controlling an imaging device according to claim 14 , further comprising determining an exposure condition of the privacy area based on a brightness distribution of image data of the privacy area. 19 . The method for controlling an imaging device according to claim 14 , further comprising determining an exposure condition of the privacy area based on a maximum brightness value or a minimum brightness value of pixels of image data of the privacy area.

20. A non-transitory computer-readable storage medium storing a program for causing a computer to execute a method comprising the steps of: Capturing an image by setting exposure conditions for each area on the imaging surface; Set privacy zones in captured images; determining the exposure condition of the set privacy area to be an exposure condition capable of protecting the privacy of the subject in the privacy area; and Control is performed so that an image of the subject is captured based on the determined exposure conditions.

Images