Control method, control device and electronic device system for image sensor

Abstract

The present technology relates to a control method, a control device, and an electronic device system for an image sensor capable of achieving reduced power consumption. The control method includes: operating the image sensor in a sleep state when the ambient illuminance is less than a threshold value; setting the operation mode of the image sensor directly from the sleep state to a low power consumption mode when the ambient illuminance is detected to be equal to or greater than the threshold value; and setting the operation mode of the image sensor from the low power consumption mode to a normal mode based on detecting a predetermined object in an image obtained by the image sensor in the low power consumption mode.

Description

[0001] This application is a divisional application of the Chinese national phase application whose international application date is March 9, 2018, international application number is PCT / JP2018 / 009149, and invention name is “Control device, control method, program and electronic device system”. The national phase entry date of the Chinese national phase application is June 13, 2019, the application number is 201880005053.9, and the invention name is “Control device, control method, program and electronic device system”. Technical Field

[0002] The present technology relates to a control apparatus, a control method, a program, and an electronic device system, and particularly to a control apparatus, a control method, a program, and an electronic device system capable of achieving, for example, low power consumption. Background Art

[0003] In a device that receives power supply from a battery and operates, such as a wearable device, etc. (hereinafter, also referred to as a mounted battery device), in order to make the battery last for a long time, when not in use, the operation mode is set to a low power consumption mode, and becomes a waiting state such as a sleep state, a standby state, etc. Then, when the user's use is detected, the operation mode is set (switched) to a normal mode, and normal operation is performed.

[0004] For example, with the portable terminal described in Patent Document 1, the illuminance sensor detects the surrounding illuminance, and when the illuminance is equal to or greater than a threshold value, the power of the camera module is turned on and an image is captured. Then, the face recognition process is performed using the image captured by the camera module. When the face is recognized, the operation mode is set to the normal mode. When the face is not recognized, the operation mode is set to the low power consumption mode (power saving mode).

[0005] Therefore, with the portable terminal described in Patent Document 1, in the case where a face is recognized, the normal mode is activated, and in the case where a face is not recognized, the low power consumption mode is activated. Therefore, when the user uses the portable terminal, the normal mode is activated, and when the user does not use the portable terminal, the low power consumption mode is activated. When the user does not use the portable terminal, power consumption can be suppressed.

[0006] Prior art literature

[0007] Patent Literature

[0008] Patent Document 1: Japanese Patent Application Publication No. 2014-027386 Summary of the invention

[0009] Problems to be solved by the present invention

[0010] With the portable terminal described in Patent Document 1, when the ambient illumination is equal to or greater than a threshold value, the power of the camera module is turned on and an image is captured. The image captured by the camera module is used for face recognition processing. Therefore, the image needs to have a certain degree of resolution, for example, an image with a resolution similar to that of an image in the case of using the portable terminal as a camera (hereinafter also referred to as an image with normal resolution).

[0011] Therefore, with the portable terminal described in Patent Document 1, whenever an illuminance equal to or greater than a threshold value is detected, the power of the camera module is turned on, and an image with a normal resolution is captured.

[0012] Furthermore, with the portable terminal described in Patent Document 1, a threshold value of illuminance is determined with reference to brightness required to perform face recognition processing.

[0013] Therefore, with the portable terminal described in Patent Document 1, the illumination equal to or greater than the threshold is frequently detected. The power of the camera module is turned on each time, and an image with normal resolution is captured. Therefore, the waiting time of the camera module is shortened, resulting in increased power consumption.

[0014] The present technology has been proposed in view of such circumstances, and can achieve low power consumption.

[0015] Solutions to the problem

[0016] According to the present technology, a control method for an image sensor includes: operating the image sensor in a sleep state when the ambient illumination is less than a threshold value; upon detecting that the ambient illumination is equal to or greater than the threshold value, directly setting the operation mode of the image sensor from the sleep state to a low power consumption mode; and setting the operation mode of the image sensor from the low power consumption mode to a normal mode based on detecting a predetermined object in an image obtained by the image sensor in the low power consumption mode.

[0017] According to the present technology, a control device for an image sensor includes: a first unit, which enables the image sensor to operate in a sleep state when the ambient illumination is less than a threshold value; a second unit, which sets the operation mode of the image sensor directly from the sleep state to a low power consumption mode when it is detected that the ambient illumination is equal to or greater than the threshold value; and a third unit, which sets the operation mode of the image sensor from the low power consumption mode to a normal mode based on detecting a predetermined object in an image obtained by the image sensor in the low power consumption mode.

[0018] According to a non-transitory computer-readable storage medium of the present technology, instructions are stored. When a processing circuit executes the instructions, a control method for an image sensor is executed, the method comprising: operating the image sensor in a sleep state when the ambient illumination is less than a threshold; setting the operation mode of the image sensor directly from the sleep state to a low power consumption mode when it is detected that the ambient illumination is equal to or greater than the threshold; and setting the operation mode of the image sensor from the low power consumption mode to a normal mode based on detecting a predetermined object in an image obtained by the image sensor in the low power consumption mode.

[0019] A control device or program according to the present technology is a control device for an image sensor, comprising: a detection unit, which detects the presence or absence of a preset predetermined object from an image with a low resolution output by the image sensor in a low power consumption mode, the image sensor including a low power consumption mode and a normal mode as operation modes, wherein an image with a low resolution is output in the low power consumption mode, and an image with a high resolution compared to the image with the low resolution is output in the normal mode; and a setting unit, which sets the operation mode of the image sensor to the normal mode when the presence of the predetermined object is detected, and causes the image sensor to output an image with a high resolution, or a program for causing a computer to be used as such a control device.

[0020] The control method according to the present technology is a control method for an image sensor, comprising: detecting the presence or absence of a preset predetermined object from an image with a low resolution output by the image sensor in a low power consumption mode, the image sensor including a low power consumption mode and a normal mode as operation modes, wherein an image with a low resolution is output in the normal mode, and an image with a high resolution compared to the image with the low resolution is output in the normal mode; and, in the case where the presence of the predetermined object is detected, setting the operation mode of the image sensor to the normal mode, and causing the image sensor to output an image with a high resolution.

[0021] An electronic device system according to the present technology is an electronic device system including: an image sensor, including a low power consumption mode and a normal mode as operation modes, outputting an image with a low resolution in the low power consumption mode, and outputting an image with a high resolution compared with the image with the low resolution in the normal mode; and a control unit, which controls the image sensor, and the control unit includes: a detection unit, which detects the presence or absence of a preset predetermined object from the image with a low resolution output by the image sensor in the low power consumption mode; and a setting unit, which sets the operation mode of the image sensor to the normal mode when the presence of the predetermined object is detected, and causes the image sensor to output an image with a high resolution.

[0022] According to the present technology, the presence or absence of a preset predetermined object is detected from an image with a low resolution output by an image sensor in a low power consumption mode, the image sensor including a low power consumption mode and a normal mode as operation modes, an image with a low resolution is output in the low power consumption mode, and an image with a high resolution compared to the image with the low resolution is output in the normal mode. Then, in the case where the presence of the predetermined object is detected, the operation mode of the image sensor is set to the normal mode, and the image sensor outputs an image with a high resolution.

[0023] Note that the control device may be an independent device, or may be an internal block constituting a single device.

[0024] Furthermore, the program can be provided by being transmitted via a transmission medium or by being recorded on a recording medium.

[0025] Effects of the Invention

[0026] According to the present technology, low power consumption can be achieved.

[0027] Note that the effects described herein are not necessarily limited but may also be any effects described in the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a block diagram showing a configuration example of an embodiment of an electronic device system to which the present technology is applied.

[0029] Figure 2 is a block diagram showing a configuration example of the camera system 11 .

[0030] Figure 3 is a flowchart illustrating an example of processing in a case where the electronic device system performs face authentication.

[0031] Figure 4 is a diagram further illustrating an example of processing in a case where the electronic device system performs face authentication.

[0032] Figure 5 is a diagram illustrating an example of processing in a case where an electronic device system performs iris authentication.

[0033] Figure 6 FIG. 1 is a diagram for explaining an example of processing in a case where the electronic device system performs palm print authentication.

[0034] Figure 7 is a diagram illustrating an example of processing in a case where the electronic device system is a surveillance camera of a pay parking lot and records license plates of parked cars.

[0035] Figure 8It is a diagram for explaining an example of processing in a case where the electronic device system is a surveillance camera of a house and records a suspicious person.

[0036] Fig. 9 is a block diagram showing a different configuration example of the camera system 11 .

[0037] Fig.10 is a flowchart illustrating an example of processing in the case where the original processing of the smartphone is executed in the case where the electronic device system is a smartphone and face authentication succeeds.

[0038] Fig.11 is a block diagram showing a configuration example of an embodiment of a computer to which the present technology is applied. DETAILED DESCRIPTION

[0039] <Embodiment of electronic device system to which the present technology has been applied>

[0040] Figure 1 is a block diagram showing a configuration example of an embodiment of an electronic device system to which the present technology is applied.

[0041] exist Figure 1 In the embodiment, the electronic device system includes a camera system 11 , an image processing unit 12 , and a battery 13 .

[0042] The camera system 11 captures an image and supplies a captured image obtained as a result of the capturing to the image processing unit 12 .

[0043] The image processing unit 12 performs image processing on the captured image from the camera system 11 .

[0044] The battery 13 supplies necessary power to the camera system 11 , the image processing unit 12 , and blocks constituting the electronic device system, which are not shown.

[0045] The electronic device system described above can be applied to, for example, portable terminals such as smartphones, electronic locks of buildings, surveillance cameras such as houses or paid parking lots, wearable devices such as wristband-type or glasses-type, and any device that captures images and performs image processing on images, especially a battery-mounted device that receives power supply from a battery and operates. However, the electronic device system can be applied to, for example, a device that receives power from a power outlet and operates, rather than a battery-mounted device.

[0046] For example, when the electronic device system is applied to a smartphone, the image processing unit 12 performs authentication processing using a captured image as image processing. Only when the authentication succeeds, the smartphone is unlocked and can enter a usable state that allows normal use.

[0047] In addition, for example, in the case where the electronic device system is applied to an electronic lock, the image processing unit 12 performs authentication processing using the captured image as image processing. Only when the authentication is successful, the electronic lock can be unlocked.

[0048] In addition, for example, in the case where the electronic device system is applied to a surveillance camera in a house or a toll parking lot, the image processing unit 12 can record a captured image of a suspicious person wandering around the house, or a so-called license plate (e.g., a car license plate number) of an illegally parked car as image processing.

[0049] Note that the image processing unit 12 may be provided separately from the camera system 11 , or may be included in the camera system 11 (eg, the control unit 22 described later).

[0050] <Configuration Example of Camera System 11>

[0051] Figure 2 It is shown Figure 1 1 is a block diagram of a configuration example of the camera system 11.

[0052] exist Figure 2 In the embodiment, the camera system includes an image sensor 21 and a control unit 22.

[0053] The image sensor 21 is, for example, a complementary metal oxide semiconductor (CMOS) image sensor and includes, as operation modes, a low power consumption mode for outputting a low resolution image having a low resolution and a normal mode for outputting a high resolution image having a higher resolution than the low resolution image.

[0054] The image sensor 21 sets an operation mode according to the control of the control unit 22, captures an image in the operation mode, and outputs a so-called raw image.

[0055] Here, in the case where the image sensor 21 is in the low power consumption mode, for example, output of a low-resolution image can be performed by floating diffusion (FD) addition, sparse readout of pixel values ​​(charges) of pixels, or the like.

[0056] In FD addition, with the image sensor 21, pixels constituting the image sensor 21 and not shown share FD in units of multiple pixels, and in the FD, each pixel value of the multiple pixels is added, for example, the pixels sharing the FD.

[0057] The image sensor 21 has an AD converter (ADC) (not shown) which analog-to-digital (AD) converts pixel values. However, in the case of performing FD addition or sparse reading of pixel values, the number of pixel values ​​to be read is small, resulting in a reduction in the number of ADCs that need to be operated. In addition, the speed of reading pixel values ​​or the operating speed of the ADC may be slow. Therefore, in the case where the image sensor 21 is in low power consumption mode, power consumption can be significantly suppressed compared to the case of a normal mode that outputs a high-resolution image.

[0058] In the low power consumption mode, the image sensor 21 outputs a low-resolution image having an extremely low resolution, for example, approximately 16×9 pixels in height and width, or the like, by the above-mentioned FD addition or sparse reading of pixels.

[0059] Meanwhile, in the normal mode, the image sensor 21 outputs, for example, a high-definition (HD) image using all pixels of the image sensor 21 , or a so-called 4k image, 8k image, etc., as a high-resolution image.

[0060] An original image, which is a low-resolution image or a high-resolution image output by the image sensor 21 , is supplied to the control unit 22 .

[0061] Note that as the image sensor 21 , in addition to an image sensor that receives (captures) visible light, an image sensor that receives light (electromagnetic waves) other than visible light, such as infrared light, etc. may be employed.

[0062] The control unit 22 controls the image sensor 21 to capture an image.

[0063] The control unit 22 includes a selection unit 31 , an object presence / absence detection unit 32 , a mode setting unit 33 , an exposure control unit 34 , and a development processing unit 35 .

[0064] The selection unit 31 selects one of the object presence / absence detection unit 32 and the development processing unit 35 as a destination to output the original image from the image sensor 21 according to the control of the mode setting unit 33, and outputs the original image to the selected image.

[0065] Here, in the case where the operation mode of the image sensor 21 is the normal mode and the image sensor 21 outputs a high-resolution image, the mode setting unit 33 controls the selection unit 31 to select the development processing unit 35 as the output destination of the high-resolution image. In addition, in the case where the operation mode of the image sensor 21 is the low power consumption mode and the image sensor 21 outputs a low-resolution image, the mode setting unit 33 controls the selection unit 31 to select the object presence / absence detection unit 32 as the destination of the output of the low-resolution image.

[0066] Therefore, in the case where the image sensor 21 outputs a high-resolution image, the high-resolution image is supplied to the development processing unit 35 via the selection unit 31. In addition, in the case where the image sensor 21 outputs a low-resolution image, the low-resolution image is supplied to the object presence / absence detection unit 32 via the selection unit 31. Note that power supply may be stopped with respect to a block that is not selected as a (direct or indirect) destination of the image output from the selection unit 31. In this case, low power consumption of the camera system 11 (and ultimately, the electronic device system) can be achieved.

[0067] The object presence / absence detection unit 32 detects the presence or absence of a preset predetermined object (hereinafter also referred to as a set object) from the low-resolution image provided by the selection unit 31. In other words, the object presence / absence detection unit 32 detects whether the set object is displayed in the low-resolution image provided by the selection unit 31.

[0068] The object presence / absence detecting unit 32 supplies detection information indicating a detection result regarding the presence or absence of a set object in the low-resolution image to the mode setting unit 33 .

[0069] Here, as the set object, not only individual items (individual objects) such as a specific person, a specific person's eyes, a specific car, etc. are adopted, but also general things belonging to a certain category such as any person, any person's eyes, any car, etc. are adopted. Therefore, even when the target of detecting the presence or absence of the set object is a low-resolution image, the presence or absence of the set object can be accurately detected.

[0070] The mode setting unit 33 sets the operation mode of the image sensor 21 according to the detection information from the object presence / absence detection unit 32 , and controls the selection unit 31 according to the operation mode.

[0071] In other words, the mode setting unit 33 sets the operation mode of the image sensor 21 to the low power consumption mode, for example, when the control unit 22 is activated, or when the detection information from the object presence / absence detection unit 32 does not indicate the presence of the set object (the set object is shown in a low-resolution image), etc.

[0072] Furthermore, the mode setting unit 33 sets the operation mode of the image sensor 21 to the normal mode, for example, in a case where the detection information from the object presence / absence detection unit 32 indicates that a set object is present.

[0073] As described above, the image sensor 21 operates with low power consumption and captures and outputs a low-resolution image in the low power consumption mode, and captures and outputs a high-resolution image in the normal mode.

[0074] Here, as mentioned above, Figure 2 In the embodiment, the image sensor 21 operates in the low power consumption mode in the normal state (normal time), and operates in the normal mode when the presence of the set object is detected. Therefore, it can be said that the image sensor 21 operates in the normal mode intermittently.

[0075] When the operation mode of the image sensor 21 is set to the low power consumption mode, the mode setting unit 33 controls the selection unit 31 to select the object presence / absence detection unit 32 as the destination for the output of the original image, which is a low-resolution image output from the image sensor 21, depending on the low power consumption mode.

[0076] In addition, when the operation mode of the image sensor 21 is set to the normal mode, the mode setting unit 33 controls the selection unit 31 to select the development processing unit 35 as the destination of the output of the original image, which is the high-resolution image output from the image sensor 21, depending on the normal mode.

[0077] The exposure control unit 34 controls the exposure of the image sensor 21 based on the low-resolution image output to the object presence / absence detection unit 32 through the selection unit 31. For example, in the case where the brightness of the low-resolution image is too low, the exposure control unit 34 controls the image sensor 21 to increase the exposure time. In addition, for example, in the case where the brightness of the low-resolution image is too high, the exposure control unit 34 controls the image sensor 21 to shorten the exposure time. Note that the exposure control unit 34 may control the exposure so as to set the object shown in the low-resolution image within an appropriate exposure range, for example, by a conventional automatic exposure technique, etc. In addition, the exposure control unit 34 may control the exposure so as to set each dynamic range of the object assumed to be within the exposure range, for example, by a conventional wide dynamic range technique.

[0078] The development processing unit 35 performs development processing of the original image, which is, for example, a high-resolution image supplied from the selection unit 31, to convert the original image of RGB (red, green, blue) into a YCbCr image, and outputs the image to the image processing unit 12 as a captured image.

[0079] <Processing of Electronic Device System>

[0080] Figure 3 Yes Description Figure 1 A flowchart of an example of a process in which an electronic device system performs, for example, facial authentication.

[0081] in other words, Figure 3 is a flowchart illustrating an example of processing in a case where the electronic device system is a smartphone and face authentication succeeds, the smartphone is unlocked and the original processing of the smartphone is performed.

[0082] In step S11 , the mode setting unit 33 sets the operation mode of the image sensor 21 to the low power consumption mode, and the process proceeds to step S12 .

[0083] In step S12 , the image sensor 21 sets the operation mode to the low power consumption mode, and starts capturing a low-resolution image according to the setting (control) of the mode setting unit 33 , and the process proceeds to step S13 .

[0084] The low-resolution image captured by the image sensor 21 in the low power consumption mode is supplied to the selection unit 31 .

[0085] In step S13, based on the fact that the operation mode of the image sensor 21 is the low power consumption mode, the mode setting unit 33 causes the selection unit 31 to select the object presence / absence detection unit 32 as the destination of the output of the low resolution image from the image sensor 21. Therefore, the low resolution image supplied from the image sensor 21 to the selection unit 31 is supplied to the object presence / absence detection unit 32.

[0086] Here, the low-resolution image supplied by the selection unit 31 to the object presence / absence detection unit 32 is also supplied to the exposure control unit 34. The exposure control unit 34 controls the exposure of the image sensor 21 based on the low-resolution image.

[0087] Thereafter, the process proceeds from step S13 to step S14. The object presence / absence detection unit 32 performs detection processing for detecting the presence or absence of a set object from the low-resolution image supplied from the selection unit 31, and supplies detection information indicating the detection result to the mode setting unit 33. The process proceeds to step S15.

[0088] Here, in the case of performing face authentication, as a setting object, a face (head) of a person (but not limited to an individual) who is a target of face authentication is adopted.

[0089] In step S15 , the mode setting unit 33 determines whether the set object exists (is shown) in the low-resolution image based on the detection information from the object presence / absence detection unit 32 .

[0090] In step S15 , in the case where it is determined that the set object does not exist (is shown) in the low-resolution image, the process returns to step S14 , and the detection process is repeated.

[0091] Furthermore, in step S15 , in the case where it is determined that the set object exists in the low-resolution image, that is, in the case where the low-resolution image shows the face that is the target of face authentication, the process proceeds to step S16 .

[0092] In step S16 , the mode setting unit 33 sets the operation mode of the image sensor 21 to the normal mode, and the process proceeds to step S17 .

[0093] In step S17 , the image sensor 21 sets the operation mode to the normal mode, and starts capturing a high-resolution image according to the setting of the mode setting unit 33 , and the process proceeds to step S18 .

[0094] The high-resolution image captured by the image sensor 21 in the normal mode is an image showing a face whose presence is detected in the low-resolution image in the detection process, and is supplied to the selection unit 31 .

[0095] In step S18, based on the fact that the operation mode of the image sensor 21 is the normal mode, the mode setting unit 33 causes the selection unit 31 to select the development processing unit 35 as the destination of the output of the high-resolution image from the image sensor 21. Therefore, the high-resolution image supplied from the image sensor 21 to the selection unit 31 is supplied to the development processing unit 35.

[0096] The development processing unit 35 performs development processing of the high-resolution image supplied from the selection unit 31 to convert the high-resolution image of RGB into a high-resolution image of YCbCr, and outputs the image to the image processing unit 12 as a captured image.

[0097] Thereafter, the process proceeds from step S18 to step S19. The image processing unit 12 performs face authentication (by personal authentication) using the captured image (the high-resolution image from the development processing unit 35), and the process proceeds to step S20.

[0098] In step S20 , the image processing unit 12 determines whether face authentication using the high-resolution image has succeeded.

[0099] In step S20, when it is determined that facial authentication has not succeeded, that is, when the face shown in the high-resolution image is not the face of the owner (specific individual) of the electronic device system which is a smartphone, the processing returns to step S11, and similar processing is repeated.

[0100] Furthermore, in step S20 , in the case where it is determined that facial authentication has succeeded, that is, in the case where the face shown in the high-resolution image is the face of the owner of the electronic device system which is a smartphone, the process proceeds to step S21 .

[0101] In step S21 , the electronic device system as the smartphone starts the original operation of the smartphone, and the process proceeds to step S22 .

[0102] In step S22 , the electronic device system as a smart phone determines whether a sleep condition for entering a sleep state is satisfied.

[0103] Here, as the sleep condition, for example, operating the power button of the electronic device system as a smartphone to enter the sleep state, not performing any operation on the electronic device system as a smartphone for a period of time or longer, etc. can be adopted.

[0104] In step S22, in the case where it is determined that the sleep condition is not satisfied, the process returns to step S22.

[0105] Furthermore, in step S22, in the case where it is determined that the sleep condition is satisfied, the process proceeds to step S23. The electronic device system as the smartphone becomes in the sleep state. The process returns to step S11. Similar processes are subsequently repeated.

[0106] As mentioned above, by Figure 3 According to the processing of the flowchart, while the setting object does not exist in the image (low-resolution image) output by the image sensor 21, the image sensor 21 operates in the low power consumption mode, and when the presence of the setting object is confirmed, it starts to operate in the normal mode.

[0107] Therefore, compared with a case where the image sensor 21 always operates in the normal mode regardless of the presence of a set object, the power consumption of the image sensor 21 can be significantly reduced while waiting for the image processing unit 12 to perform face authentication. Therefore, the duration of the battery 13 can be increased ( Figure 1 ).

[0108] According to simulations conducted by the present inventors, it was confirmed that the power consumption of the image sensor 21 can be reduced to about 1 / 40 (eg, about 1.5 mW) when performing face authentication in the waiting low power mode compared to the waiting normal mode.

[0109] In addition, according to Figure 3 According to the process of the flowchart, when it is confirmed that the set object exists in the low-resolution image captured in the low power consumption mode, a high-resolution image is captured in the normal mode, and facial authentication is performed using the generated high-resolution image. Therefore, facial authentication with high accuracy can be performed.

[0110] Figure 4 It is a further explanation Figure 1 A diagram of an example of processing in which an electronic device system performs facial authentication.

[0111] Figure 4 A shows the state of the viewing angle of the image captured by the image sensor 21. Figure 4In A, a face (head) (not limited to an individual) as a target of face authentication exists within the field of view of an image captured by the image sensor 21.

[0112] Figure 4 B shows the image captured by the image sensor 21 Figure 4 A diagram of a processing case of an electronic device system as a smartphone under a viewing angle state.

[0113] Using electronic device systems, Figure 4 The state within the angle of view of A is captured by the image sensor 21 in the low power consumption mode, and the low-resolution image obtained by the capturing is supplied to the object presence / absence detection unit 32 via the selection unit 31.

[0114] In the object presence / absence detection unit 32, detection processing is performed with the head (face) as the target of facial authentication as the set object to detect the presence or absence of the head as the set object from the low-resolution image captured in the low power consumption mode.

[0115] Here, as a detection process for detecting the presence or absence of a head as a set object from a low-resolution image, a simple process may be adopted, for example, a process of detecting an elliptical or rectangular object from near the center of the low-resolution image and detecting a shoulder structure toward the outer edge of the low-resolution image is performed. The detection process for the low-resolution image does not need to detect the head of a specific person. This is because it is sufficient if the presence of any person's head is detected.

[0116] By detecting an elliptical or rectangular object and detecting a shoulder structure toward the outer edge of the above-mentioned low-resolution image, the head can be detected with sufficient accuracy even from a low-resolution image of approximately 16×9 pixels to set (switch) the operating mode of the image sensor 21.

[0117] Note that the algorithm for the detection process for detecting the presence or absence of the set object from the low-resolution image is not particularly limited. For example, by the detection process for detecting the presence or absence of the head as the set object from the low-resolution image, the presence or absence of the head as the set object can be detected based on a learning algorithm for determining whether the head is displayed in the low-resolution image, etc., without performing the detection of the shoulder structure.

[0118] By the detection processing performed in the object presence / absence detection unit 32 , when the presence of the head as the set object is detected from the low-resolution image, the image sensor 21 changes to the normal mode and starts capturing a high-resolution image.

[0119] Then, in the image processing unit 12 , face authentication is performed using the high-resolution image captured by the image sensor 21 in the normal mode.

[0120] Figure 5 Yes Description Figure 1 A diagram of an example of processing in which an electronic device system performs iris authentication.

[0121] Figure 5 A shows the state of the viewing angle of the image captured by the image sensor 21. Figure 5 In A, the eye as the target of iris authentication exists within the image viewing angle captured by the image sensor 21.

[0122] Figure 5 B shows the image captured by the image sensor 21 Figure 5 A diagram of a processing case of an electronic device system as a smartphone under a viewing angle state.

[0123] Using the electronic device system, the image sensor 21 captures the image in a low power mode. Figure 5 The state within the A angle of view is determined, and the low-resolution image obtained by capturing is provided to the object presence / absence detection unit 32 via the selection unit 31.

[0124] In the subject presence / absence detection unit 32 , detection processing is performed with the eye as a target of iris authentication as a set subject to detect the presence or absence of the eye as a set subject from a low-resolution image captured in the low power consumption mode.

[0125] Here, as a detection process for detecting the presence or absence of eyes as a set object from a low-resolution image, a simple process can be adopted. The detection process does not need to detect the eyes of a specific person. This is because it is sufficient if the presence of any person's eyes is detected.

[0126] By the detection processing performed in the subject presence / absence detection unit 32 , when the presence of an eye as a set subject is detected from the low-resolution image, the image sensor 21 changes to the normal mode and starts capturing a high-resolution image.

[0127] Then, in the image processing unit 12 , iris authentication is performed using the high-resolution image captured by the image sensor 21 in the normal mode.

[0128] Figure 6 Yes Description Figure 1 FIG. 1 is a diagram of an example of a process in which an electronic device system performs palmprint authentication.

[0129] Figure 6 A shows the state of the viewing angle of the image captured by the image sensor 21. Figure 6 In A, a hand as a target of palm print authentication exists within the field of view of an image captured by the image sensor 21.

[0130] Figure 6 B shows the image captured by the image sensor 21 Figure 6 A diagram of a processing case of an electronic device system as a smartphone under a viewing angle state.

[0131] Using the electronic device system, the image sensor 21 is used to capture the image in a low power consumption mode. Figure 6 The state within the A angle of view, and the low-resolution image obtained by capturing is provided to the object presence / absence detection unit 32 via the selection unit 31.

[0132] In the object presence / absence detection unit 32 , detection processing is performed using the hand as a target of palm print authentication as a set object to detect the presence or absence of the hand as the set object from a low-resolution image captured in the low power consumption mode.

[0133] Here, as a detection process for detecting the presence or absence of a hand as a set object from a low-resolution image, a simple process can be adopted. The detection process does not need to detect the hand of a specific person. This is because it is sufficient if the presence of any person's hand is detected.

[0134] By the detection processing performed in the object presence / absence detection unit 32 , when the presence of a hand as a set object is detected from the low-resolution image, the image sensor 21 changes to the normal mode and starts capturing a high-resolution image.

[0135] Then, in the image processing unit 12 , palmprint authentication is performed using the high-resolution image captured by the image sensor 21 in the normal mode.

[0136] Figure 7 For example, in Figure 1 The electronic device system is a surveillance camera of a paid parking lot and records the license plates of parked cars to deal with an example of a process in the case of illegal parking.

[0137] Figure 7 A shows the state of the viewing angle of the image captured by the image sensor 21. Figure 7 In A, a car as a recording target exists within the field of view of the image captured by the image sensor 21.

[0138] Figure 7 B shows the image captured by the image sensor 21 Figure 7 FIG. 1 is a diagram of a processing case of an electronic device system as a monitoring camera under a state of a viewing angle of A. FIG.

[0139] Using the electronic device system, the image sensor 21 is used to capture the image in a low power consumption mode. Figure 7The state within the A angle of view is determined, and the low-resolution image obtained by capturing is provided to the object presence / absence detection unit 32 via the selection unit 31.

[0140] In the object presence / absence detection unit 32 , detection processing is performed using the car as a recording target as a set object to detect the presence or absence of the car as the set object from the low-resolution image captured in the low power consumption mode.

[0141] Here, as a detection process for detecting the presence or absence of a car as a setting object from a low-resolution image, a simple process can be adopted. The detection process does not need to detect a specific car. This is because it is sufficient if the presence of any car is detected.

[0142] By the detection processing performed in the object presence / absence detection unit 32 , when the presence of the automobile as the set object is detected from the low-resolution image, the image sensor 21 changes to the normal mode and starts capturing the high-resolution image.

[0143] Then, in the image processing unit 12 , recording (video recording) of a high-resolution image captured by the image sensor 21 in the normal mode is performed.

[0144] Note that in this case, regarding the recording of the high-resolution image, for example, until the car shown in the high-resolution image disappears (until the car is not shown in the high-resolution image), the high-resolution image may be video-recorded in a moving image or may be recorded in a single or multiple still images. Then, in a case where the car is not shown in the high-resolution image, etc., the operation mode of the image sensor 21 may be returned to the low power consumption mode.

[0145] Figure 8 Yes Description Figure 1 The electronic device system is a surveillance camera of a house and is a diagram of an example of a process in the event of a suspicious person being recorded.

[0146] Figure 8 A shows the state of the viewing angle of the image captured by the image sensor 21. Figure 8 In A, a person as a recording target exists within the field of view of the image captured by the image sensor 21.

[0147] Figure 8 B shows the image captured by the image sensor 21 Figure 8 FIG. 1 is a diagram of a processing case of an electronic device system as a monitoring camera under a state of a viewing angle of A. FIG.

[0148] Using the electronic device system, the image sensor 21 captures the image in a low power mode. Figure 8The state within the A angle of view is determined, and the low-resolution image obtained by capturing is provided to the object presence / absence detection unit 32 via the selection unit 31.

[0149] In the subject presence / absence detection unit 32 , detection processing is performed with a person as a recording subject as a set subject to detect the presence or absence of the person as the set subject from a low-resolution image captured in the low power consumption mode.

[0150] Here, as a detection process for detecting the presence or absence of a person as a set object from a low-resolution image, a simple process can be adopted. The detection process does not need to detect a specific person. This is because it is sufficient if the presence of any person is detected.

[0151] By the detection processing performed in the subject presence / absence detection unit 32 , when the presence of a person as a set subject is detected from the low-resolution image, the image sensor 21 changes to the normal mode and starts capturing a high-resolution image.

[0152] Then, in the image processing unit 12 , recording (video recording) of a high-resolution image captured by the image sensor 21 in the normal mode is performed.

[0153] Note that in this case, regarding the recording of the high-resolution image, for example, until a person shown in the high-resolution image disappears (until the person is not shown in the high-resolution image), the high-resolution image may be video-recorded in a moving image or may be recorded in a single or multiple still images. Then, in a case where the person is not shown in the high-resolution image, etc., the operation mode of the image sensor 21 may be returned to the low power consumption mode.

[0154] <Different Configuration Examples of Camera System 11>

[0155] Fig. 9 It is shown Figure 1 1 is a block diagram of different configuration examples of the camera system 11.

[0156] Note that the figure Figure 2 Corresponding parts are denoted by the same reference numerals and their descriptions are appropriately omitted below.

[0157] exist Fig. 9 In the embodiment, the camera system 11 includes an image sensor 21, a control unit 22, an illuminance sensor 51, and an operation control unit 52.

[0158] therefore, Fig. 9 Camera system 11 with Figure 2 The case is the same as that of in that the camera system 11 includes an image sensor 21 and a control unit 22 .

[0159] However, Fig. 9 Camera system 11 with Figure 2 The case of FIG. 1 is different in that the camera system 11 newly includes an illuminance sensor 51 and an operation control unit 52 .

[0160] The illuminance sensor 51 senses the surrounding illuminance of the camera system 11 (a range in which photographing is performed by the image sensor 21 ), and supplies the illuminance to the operation control unit 52 .

[0161] The operation control unit 52 controls the operations of the image sensor 21 and the control unit 22 according to the surrounding illuminance from the illuminance sensor 51 .

[0162] In other words, in a case where the ambient illuminance is not equal to or greater than the threshold value, that is, for example, in a case where the ambient illuminance is not an illuminance of a degree at which a low-resolution image can be captured (it is so bright that detection processing can be performed by the object presence / absence detection unit 32), the operation control unit 52 controls the image sensor 21 and the control unit 22 and stops the operation of the image sensor 21 and the control unit 22 to stop the operation (enter a state in which no processing is performed as much as possible, but processing can be started quickly).

[0163] In addition, in a case where the ambient illuminance is equal to or greater than a threshold value, that is, for example, in a case where the ambient illuminance is an illuminance of a degree at which a low-resolution image can be captured (it is bright enough that detection processing can be performed by the object presence / absence detection unit 32), the operation control unit 52 controls the image sensor 21 and the control unit 22, and starts the operation of the image sensor 21 and the control unit 22 to start the operation.

[0164] When the image sensor 21 and the control unit 22 start operating, similar to Figure 2 In the case of the object presence / absence detection unit 32, the mode setting unit 33 of the control unit 22 sets the operation mode of the image sensor 21 to the low power consumption mode, and the image sensor 21 starts capturing the low-resolution image in the low power consumption mode. In addition, in the control unit 22, the object presence / absence detection unit 32 starts the detection processing of the low-resolution image captured by the image sensor 21 in the low power consumption mode.

[0165] Fig.10 It is explained in the included Fig. 9 Flowchart of an example of processing in a case where the electronic device system of the camera system 11 is a smartphone and the smartphone is unlocked and original processing of the smartphone is performed when facial authentication has succeeded.

[0166] In step S41, the operation control unit 52 determines whether the ambient illuminance supplied from the illuminance sensor 51 is equal to or greater than a threshold value. In the case where it is determined that the ambient illuminance is not equal to or greater than the threshold value, the process returns to step S41. Similar processes are subsequently repeated.

[0167] During the process of step S41 , the operation control unit 52 stops the operations of the image sensor 21 and the control unit 22 .

[0168] Meanwhile, in step S41, when the operation control unit 52 determines that the surrounding illuminance supplied from the illuminance sensor 51 is equal to or greater than the threshold value, the operation control unit 52 starts the operation of the image sensor 21 and the control unit 22. The process proceeds to step S42.

[0169] In steps S42 to S54, the Figure 3 Steps S11 to S23 are processed similarly.

[0170] Then, in step S54, when the electronic device system as the smartphone enters the sleep state, the process returns to step S41. Similar processes are subsequently repeated.

[0171] As mentioned above, through Fig.10 In the processing after the flowchart of , when the ambient illuminance is not equal to or greater than the threshold value, the operation of the image sensor 21 and the control unit 22 is stopped. When the ambient illuminance becomes equal to or greater than the threshold value, the operation starts, and a similar operation is performed. Figure 3 handling of the situation.

[0172] The power consumption of the illuminance sensor 51 (for example, several μW) is sufficiently smaller than the power consumption of the image sensor 21 in the low power consumption mode (for example, about 1.5 mW). Fig.10 The processing after the flowchart is Figure 3 Compared with the case of FIG. 1 , the power consumption of the image sensor 21 can be further reduced.

[0173] <Description of Computer to Which This Technology Has Been Applied>

[0174] Next, the series of processing of the control unit 22 and the image processing unit 12 described above can be executed by hardware, and can also be executed by software. In the case of executing the series of processing by software, a program constituting the software is installed in a microcomputer or the like.

[0175] Fig.11 is a block diagram showing a configuration example of an embodiment of a computer in which a program that executes a series of processes is installed.

[0176] The program may be recorded in advance on the hard disk 105 or the ROM 103 which is a recording medium included in the computer.

[0177] Alternatively, the program may be stored (recorded) on a removable recording medium 111. Such a removable recording medium 111 may be provided as so-called package software. Here, examples of the removable recording medium 111 include a flexible disk, a compact disk read-only memory (CD-ROM), a magneto-optical (MO) disk, a digital versatile disk (DVD), a magnetic disk, a semiconductor memory, and the like.

[0178] Note that the program can be installed in the computer not only from the above-mentioned removable recording medium 111 but also downloaded to the computer and installed in the incorporated hard disk 105 via a communication network or a broadcasting network. In other words, for example, the program can be wirelessly transmitted to the computer from a download site via an artificial satellite for digital satellite broadcasting, or can be transmitted to the computer by wire via a network such as a local area network (LAN) or the Internet.

[0179] The computer includes a central processing unit (CPU) 102 . An input / output interface 110 is connected to the CPU 102 via a bus 101 .

[0180] When a user inputs a command through operation of input unit 107 or the like via input / output interface 110, CPU 102 accordingly executes a program stored in read only memory (ROM) 103. Alternatively, CPU 102 loads a program stored in hard disk 105 into random access memory (RAM) 104 and executes the program.

[0181] Thus, the CPU 102 executes the processing following the above-described flowchart or the processing executed by the configuration of the foregoing block diagram. Then, for example, the CPU 102 causes the processing result to be output from the output unit 106 via the input / output interface 110, sent from the communication unit 108, recorded by the hard disk 105, etc. as needed.

[0182] Note that the input unit 107 may include a keyboard, a mouse, a microphone, etc. Furthermore, the output unit 106 may include a liquid crystal display (LCD), a speaker, and the like.

[0183] Here, in this specification, the processing performed by the computer according to the program does not necessarily need to be performed in chronological order along the process described as a flowchart. In other words, the processing performed by the computer according to the program also includes processing performed in parallel or individually (for example, parallel processing or object processing).

[0184] In addition, the program may be processed by a single computer (processor), or may be processed in a distributed manner by a plurality of computers. In addition, the program may be transferred to a remote computer and executed by the remote computer.

[0185] In addition, in this specification, a system refers to a cluster of multiple constituent elements (devices, modules (components), etc.), and it is not important whether all the constituent elements are in the same housing. Therefore, multiple devices housed in different housings and connected via a network and a single device in which multiple modules are housed in a single housing are both the system.

[0186] Note that the embodiments of the present technology are not limited to the foregoing embodiments, but various changes can be made within the scope not departing from the gist of the present technology.

[0187] For example, the present technology may adopt a configuration of cloud computing in which one function is shared and jointly processed by a plurality of devices via a network.

[0188] Furthermore, each step described in the above flowchart may be performed by a single device, or may be shared and performed by a plurality of devices.

[0189] Furthermore, in the case where a single step includes a plurality of processes, the plurality of processes included in the single step may be executed by a single device, or may be divided and executed by a plurality of devices.

[0190] Furthermore, the effects described in this specification are merely illustrative and not restrictive, and other effects may be provided.

[0191] Note that the present technology may be configured as follows.

[0192] <1>

[0193] A control device for an image sensor, comprising:

[0194] a detection unit that detects the presence or absence of a preset predetermined object from an image with a low resolution output by an image sensor in a low power consumption mode, the image sensor including a low power consumption mode and a normal mode as operation modes, outputting an image with a low resolution in the low power consumption mode and outputting an image with a high resolution compared with the image with the low resolution in the normal mode; and

[0195] The setting unit sets the operation mode of the image sensor to a normal mode and enables the image sensor to output an image with a high resolution when the presence of the predetermined object is detected.

[0196] <2>

[0197] according to <1> control equipment,

[0198] Among them, authentication is performed using an image with high resolution.

[0199] <3>

[0200] according to <1> control equipment,

[0201] Here, recording of high-resolution images is performed.

[0202] <4>

[0203] according to <1> to <3> Any control device of

[0204] The image sensor starts operating when the ambient illumination is equal to or greater than a threshold.

[0205] <5>

[0206] A control method for an image sensor, comprising:

[0207] detecting the presence or absence of a preset predetermined object from an image with a low resolution outputted by an image sensor in a low power consumption mode, the image sensor including a low power consumption mode and a normal mode as operation modes, in which the image with the low resolution is outputted and in which the image with a high resolution compared with the image with the low resolution is outputted; and

[0208] In the case where the presence of a predetermined object is detected, the operation mode of the image sensor is set to a normal mode, and the image sensor is caused to output an image with a high resolution.

[0209] <6>

[0210] A program that causes a computer to function as one of the following:

[0211] a detection unit that detects the presence or absence of a preset predetermined object from an image with a low resolution output by an image sensor in a low power consumption mode, the image sensor including a low power consumption mode and a normal mode as operation modes, outputting an image with a low resolution in the low power consumption mode and outputting an image with a high resolution compared with the image with the low resolution in the normal mode; and

[0212] The setting unit sets the operation mode of the image sensor to a normal mode and enables the image sensor to output an image with a high resolution when the presence of the predetermined object is detected.

[0213] <7>

[0214] An electronic device system, comprising:

[0215] an image sensor including a low power consumption mode and a normal mode as operation modes, wherein an image having a low resolution is output in the low power consumption mode and an image having a high resolution compared to the image having the low resolution is output in the normal mode; and

[0216] A control unit that controls the image sensor,

[0217] The control unit includes:

[0218] a detection unit that detects the presence or absence of a preset predetermined object from an image with a low resolution output by the image sensor in a low power consumption mode, and

[0219] The setting unit sets the operation mode of the image sensor to a normal mode and enables the image sensor to output an image with a high resolution when the presence of the predetermined object is detected.

[0220] Explanation of symbols

[0221] 11 Camera system 12 Image processing unit 13 Battery 21 Image sensor

[0222] 22 control unit 31 selection unit 32 object presence / absence detection unit

[0223] 33 mode setting unit 34 exposure control unit 35 development processing unit

[0224] 51 illumination sensor 52 operation control unit 101 BUS

[0225] 102CPU 103ROM 104RAM 105hard disk 106output unit

[0226] 107 Input unit 108 Communication unit 109 Driver 110 Input / output interface

[0227] 111 Removable recording medium.

Claims

1. A control method for an image sensor, comprising: setting the operation mode of the image sensor from the low power consumption mode to a normal mode based on detecting a predetermined object in an image obtained by the image sensor in the low power consumption mode, wherein an image with a low resolution is output in the low power consumption mode of the image sensor, and an image with a high resolution is output in the normal mode of the image sensor, wherein authentication is performed using the image having the high resolution, and Among other things, the type of authentication depends on the type of a predetermined object detected in an image having a low resolution.

2. The control method according to claim 1, wherein: Based on detecting the predetermined object in the image with the low resolution, the image sensor is caused to output the image with the high resolution. The control method according to claim 1 , further comprising recording the image with high resolution. 4 . The control method according to claim 1 , further comprising detecting a characteristic shape of the predetermined object in the image with low resolution. 5 . The control method according to claim 1 , further comprising detecting the predetermined object in a predetermined area of ​​the image with the low resolution.

6. A control device for an image sensor, comprising: a third unit, based on detecting a predetermined object in an image obtained by the image sensor in the low power consumption mode, setting the operation mode of the image sensor from the low power consumption mode to the normal mode, wherein an image with a low resolution is output in the low power consumption mode of the image sensor and an image with a high resolution is output in the normal mode of the image sensor, The control device further includes a fifth unit that performs authentication using the image with high resolution, Among other things, the type of authentication depends on the type of a predetermined object detected in an image having a low resolution. 7 . The control device according to claim 6 , wherein the control device further comprises a fourth unit that causes the image sensor to output the image with the high resolution based on detection of the predetermined object in the image with the low resolution.

8. The control device according to claim 6, in, The control device further comprises a sixth unit which records the image with high resolution.

9. The control device according to claim 6, in, The control device further includes a seventh unit that detects a characteristic shape of the predetermined object in the image with the low resolution.

10. The control device according to claim 6, in, The control device further includes an eighth unit that detects the predetermined object in a predetermined area of ​​the image with the low resolution.

11. A non-transitory computer-readable storage medium storing instructions, when a processing circuit executes the instructions, a control method for an image sensor is executed, the method comprising: setting the operation mode of the image sensor from the low power consumption mode to a normal mode based on detecting a predetermined object in an image obtained by the image sensor in the low power consumption mode, wherein an image with a low resolution is output in the low power consumption mode of the image sensor, and an image with a high resolution is output in the normal mode of the image sensor, wherein authentication is performed using the image having the high resolution, and Among other things, the type of authentication depends on the type of a predetermined object detected in an image having a low resolution.

12. An electronic device system, comprising: Image sensor; A control unit controls the image sensor. The control unit comprises a processing circuit configured to: setting the operation mode of the image sensor from the low power consumption mode to a normal mode based on detecting a predetermined object in an image obtained by the image sensor in the low power consumption mode, wherein an image with a low resolution is output in the low power consumption mode of the image sensor, and an image with a high resolution is output in the normal mode of the image sensor, wherein authentication is performed using the image having the high resolution, and Among other things, the type of authentication depends on the type of a predetermined object detected in an image having a low resolution.

Images