Electronic device and control method thereof

Abstract

To provide an electronic device capable of distinguishing the type of nearby object with a simple configuration and a control method thereof The electronic device includes a first emitting means for emitting a first electromagnetic wave that is absorbed by water, and a first detecting means for detecting a reflected wave of the first electromagnetic wave. The electronic device determines whether an object that reflected the first electromagnetic wave contains water or does not contain water, depending on the intensity of the reflected wave.

Description

[Technical Field]

[0001] The present invention relates to an electronic device and a control method thereof, and more particularly to a technique for determining the type of a nearby object. [Background technology]

[0002] Patent Document 1 discloses a digital camera with a gaze detection function. When the digital camera cannot detect a pupil in an image of a nearby object, it determines that the nearby object is not an eye and stops displaying the electronic viewfinder (EVF). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-13057 Summary of the Invention [Problem to be solved by the invention]

[0004] The digital camera disclosed in Patent Document 1 can reduce unnecessary power consumption by stopping the EVF display when it determines that a nearby object is not an eye. However, in order to determine the type of nearby object, a complex configuration is required, including an optical system and an image sensor for capturing an image of the nearby object.

[0005] In one aspect, the present invention provides an electronic device capable of determining the type of a nearby object with a simple configuration, and a control method for the electronic device. [Means for solving the problem]

[0006] In one aspect, the present invention provides an electronic device comprising: a first emitting means for emitting a first electromagnetic wave that has the property of being absorbed by water; a first detecting means for detecting a reflected wave of the first electromagnetic wave; and a discriminating means for discriminating whether an object that reflected the first electromagnetic wave is an object that contains water or an object that does not contain water, depending on the intensity of the reflected wave. [Effects of the Invention]

[0007] According to the present invention, it is possible to provide an electronic device that can distinguish the type of a nearby object with a simple configuration, and a control method for the electronic device. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a cross-sectional view showing an example of the configuration of an imaging device as an example of an electronic device according to an embodiment; [Figure 2] FIG. 1 is a block diagram showing an example of the functional configuration of an imaging apparatus according to an embodiment; [Figure 3] FIG. 1 is a diagram for explaining the operation of detecting a nearby object and determining the type of the nearby object in an embodiment. [Figure 4] 1 is a flowchart illustrating an operation for detecting a nearby object and determining the type of the nearby object in an embodiment. [Figure 5] FIG. 10 is a cross-sectional view showing a configuration example of an HMD as an example of an electronic device according to a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] The present invention will be described in detail below based on exemplary embodiments with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the claimed invention. Furthermore, although multiple features are described in the embodiments, not all of them are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0010] In the following, the present invention will be described in terms of an embodiment using an imaging device as an example of an electronic device. The imaging device is not limited to devices primarily intended for capturing images, such as digital cameras and digital video cameras, but also includes devices with imaging capabilities. However, imaging capabilities are not essential for the present invention, and the present invention can be implemented in any electronic device, whether or not it has imaging capabilities. Examples of such electronic devices include computer devices (personal computers, tablets, media players, PDAs, etc.), smartphones, smartwatches, game consoles, ring-type devices, remote controls, earphones, headphones, and eyeglass-type devices. Examples of eyeglass-type devices include VR glasses, VR goggles, and HMDs. These are merely examples, and the present invention can be implemented in other electronic devices. However, the present invention is particularly suitable for implementation in electronic devices used in contact with or in close proximity to the human body, such as portable devices and wearable devices.

[0011] ●(First embodiment) <Configuration and Functions of the Imaging Device> 1 is a vertical cross-sectional view showing an example of the configuration of an imaging device 1 as an example of an electronic device according to the present invention. The imaging device 1 is a lens-interchangeable digital camera having a lens unit 1A and a camera body 1B. The lens unit 1A is detachably attached to the camera body 1B.

[0012] Lens unit 1A has an imaging optical system including a focus lens 101, a zoom lens 102, and an aperture 111. Note that for simplicity, FIG. 1 illustrates the imaging optical system as consisting of a single focus lens 101 and a single zoom lens 102, but in reality, the imaging optical system includes more lenses, such as other movable lenses. Furthermore, focus lens 101 and zoom lens 102 may also be composed of multiple lenses. Lens unit 1A forms an optical image of a subject on the imaging plane of image sensor 2.

[0013] The diaphragm 111 is used to control the amount of incident light. The opening size of the diaphragm 111 is controlled by the CPU 3 via a drive unit 112. The diaphragm 111 may be capable of functioning as a mechanical shutter.

[0014] The aperture driver 112 includes an actuator that drives the aperture 111, an aperture control circuit 212 (described later), and the like.

[0015] The lens unit 1A further includes a lens drive motor 113 , a lens drive member 114 , a photocoupler 115 , a pulse plate 116 , a mount contact 117 , and a focus adjustment circuit 118 .

[0016] Lens drive member 114 is connected to focus lens 101 and moves focus lens 101 in the optical axis direction in response to the rotation of lens drive motor 113. Pulse plate 116 is provided at the end of lens drive member 114. Patterns of different brightness are provided on the surface of pulse plate 116, and a pulse signal is output from photocoupler 115 in response to the rotation of pulse plate 116. Focus adjustment circuit 118 drives lens drive motor 113 until the number of pulses in the pulse signal obtained from photocoupler 115 reaches a value corresponding to the lens drive amount instructed by CPU 3.

[0017] 1 shows only the configuration for driving focus lens 101 for convenience, lens unit 1A also has a configuration for driving zoom lens 102 that is similar to the configuration for driving focus lens 101. Therefore, CPU 3 can also control the angle of view (focal length) of lens unit 1A by instructing lens unit 1A on the amount of drive of zoom lens 102. It is assumed that the drive directions of focus lens 101 and zoom lens 102 are specified, for example, by the sign of the amount of movement.

[0018] Mount contacts 117 are a group of electrical contacts provided on each of the camera body 1B and the lens unit 1A. The mount contacts 117 are arranged so as to be conductive when the lens unit 1A is attached to the camera body 1B. The mount contacts 117 are used for communication between the lens unit 1A and the camera body 1B (CPU 3) and for supplying power from the camera body 1B to the lens unit 1A.

[0019] The camera body 1B has an image sensor 2, a CPU 3, a memory unit 4, a display unit 5, an EVF 6, an eyepiece 7, an NIR (near infrared) light emitting unit 8, an NIR light receiving unit 9, a SWIR (short wave infrared) light emitting unit 10, a SWIR light receiving unit 11, and operation members 12 to 14.

[0020] The image sensor 2 converts the subject image that passes through the lens unit 1A into an electrical signal. The image sensor 2 may be, for example, a CCD or CMOS image sensor. The image sensor 2 has a pixel array in which multiple pixels are arranged two-dimensionally, and peripheral circuits for reading out signals from each pixel. The pixel array is provided with, for example, a primary-color Bayer color filter. Each pixel accumulates charge according to the amount of incident light through photoelectric conversion. A pixel signal group (analog image signal) representing the subject image formed on the imaging surface is obtained by reading out from each pixel a signal having a voltage according to the amount of charge accumulated during the exposure period. The image sensor 2 outputs a digital image signal (image data) by A / D converting the analog image signal. The CPU 3 stores the image data from the image sensor 2 in the memory unit 4. Note that the image sensor 2 does not have an A / D conversion function, and A / D conversion may be performed externally (for example, by an image processing circuit 204 described below) and the image data may be supplied to the CPU 3.

[0021] The image sensor 2 may have a configuration in which the photoelectric conversion area of ​​each pixel is divided so as to enable autofocus detection using an image plane phase difference detection method.

[0022] The CPU 3 (control means) executes a program to control the operations of the components of the imaging device 1 and realize the functions of the imaging device 1. The operation of the CPU 3 will be described in detail later.

[0023] The memory unit 4 temporarily stores images (including audio in the case of moving images) generated using the imaging element 2. The memory unit 4 has a capacity capable of storing at least a predetermined amount of still images and moving images.

[0024] The display unit 5 and EVF 6 have display devices such as liquid crystal displays, and display, for example, images for display stored in the memory unit 4, menu screens, etc. The EVF 6 displays video (live view images) that are continuously captured during shooting standby and shooting. The live view images may be displayed on the display unit 5. In this case, the display unit 5 also realizes the same functions as the EVF 6.

[0025] Since the display unit 5 is provided on the surface of the camera body 1B, the screen of the display unit 5 can be observed without bringing the eye close to the camera body 1. On the other hand, the EVF 6 is provided inside a peer-type finder that the camera body 1 has. The user can observe the screen of the EVF 6 through the eyepiece 7 by peering into the inside of the camera body 1 through the eyepiece (a viewing window with an eyepiece 7 provided). In this way, the eyepiece is a part that the user comes close to when using the imaging device 1.

[0026] Of the various input devices provided on camera body 1, operation members 12 to 14 indicate some of those provided on the rear surface of camera body 1. User operations on operation members 12 to 14 are detected by CPU 3. CPU 3 executes an operation according to the detected user operation.

[0027] The NIR light emitter 8 and the NIR light receiver 9 are used to detect the presence or absence of an object close to the imaging device 1 (more specifically, the eyepiece). The NIR light emitter 8 emits near-infrared light at regular intervals, and the NIR light receiver 9 outputs a signal or value corresponding to the amount of received near-infrared light or the incident intensity (hereinafter referred to as the received light amount) of the near-infrared light reflected by the object. The amount of received light by the NIR light receiver 9 increases as the object approaches, so changes in the distance to the object can be detected by changes in the received light amount. The presence or absence of a nearby object can also be determined based on whether the received light amount exceeds a threshold corresponding to the distance at which the object is considered to be close. The presence or absence of a nearby object can also be detected by other methods. For example, the time from when the NIR light emitter 8 emits light until the amount of received light by the NIR light receiver 9 exceeds the threshold varies depending on the distance of the object that reflected the near-infrared light. Therefore, the presence or absence of a nearby object can be determined based on whether the time from when the NIR light emitter 8 emits light until the amount of received light by the NIR light receiver 9 exceeds the threshold is equal to or less than the threshold corresponding to the distance at which the object is considered to be close.

[0028] In the following, the threshold used to determine whether an object is in proximity (whether a nearby object exists) will be referred to as the proximity threshold, regardless of whether the threshold indicates distance, the amount of received light or incident intensity, or time. Note that the proximity threshold conditions for determining that an object is in proximity may differ depending on whether the proximity threshold indicates distance, the amount of received light or incident intensity, or time.

[0029] The SWIR light emitter 10 and the SWIR light receiver 11 are used to determine the type of nearby object. Details of determining the type of nearby object using the SWIR light emitter 10 and the SWIR light receiver 11 will be described later. The SWIR light emitter 10 and the SWIR light receiver 11 may have the same configuration as the NIR light emitter 8 and the NIR light receiver 9, respectively, except for the wavelength.

[0030] Fig. 2 is a block diagram showing an example of the functional configuration of the imaging device 1. In Fig. 2, the same components as those in Fig. 1 are denoted by the same reference numerals.

[0031] The CPU 3 loads a program stored in the nonvolatile memory 211 into the system memory 210 and executes it to control the operation of each part of the imaging device 1 including the lens unit 1A, thereby realizing the functions of the imaging device 1.

[0032] The nonvolatile memory 211 is a rewritable nonvolatile memory, and stores programs executed by the CPU 3, various setting values ​​of the image capture device 1, GUI data, and the like. The system memory 210 is, for example, a RAM, and is used as the main memory of the CPU 3, a buffer for temporarily storing captured image data, and a work memory for temporarily storing data being processed by the image processing circuit 204. Furthermore, a part of the non-volatile memory 210 may be used as a video memory for storing image data for display.

[0033] The photometry circuit 201 generates a luminance evaluation value required for executing automatic exposure control (AE) based on luminance information of image data obtained from the imaging element 2. The photometry circuit 201 outputs the luminance evaluation value to the CPU 3.

[0034] The AF circuit 202 generates an evaluation value or a signal pair required for performing autofocus detection (AF) based on image data obtained from the image sensor 2. The AF circuit 202 outputs the evaluation value or signal pair to the CPU 3. The AF circuit 202 can generate at least one of an evaluation value used for contrast-based AF and a signal pair used for phase-difference detection AF. The AF circuit 202 may generate the signal pair used for phase-difference detection AF using a separate AF sensor instead of the image sensor 2.

[0035] The CPU 3 determines the amount of movement of the focus lens 101 that will bring the focus detection area into focus, using the evaluation value and signal pair generated by the AF circuit 202. The CPU 3 then transmits the determined amount of movement to the focus adjustment circuit 118, which adjusts the focus distance of the lens unit 1A.

[0036] Switch A, which turns on with the first stroke of the release button, and switch B, which turns on with the second stroke of the release button, are connected to signal input circuit 203. When switch A is in the ON state, the camera's photometry, distance measurement, line of sight detection, and other operations are initiated. When switch B is in the ON state, a photographing operation is initiated. The ON signals from switches A and B are input to signal input circuit 203 and sent to CPU 3.

[0037] The image processing circuit 204 applies image processing and signal processing to image data stored in the memory unit 4 or the system memory 210 to generate signals and image data according to the application, and acquire and / or generate various types of information. The image processing circuit 204 may be a dedicated hardware circuit such as an ASIC (Application Specific Integrated Circuit) designed to realize a specific function. Alternatively, the image processing circuit 204 may be configured such that a processor such as a DSP (Digital Signal Processor) or a GPU (Graphics Processing Unit) executes software to realize a specific function. The image processing circuit 204 outputs the acquired or generated information and data to the CPU 3.

[0038] The image processing applied by the image processing circuit 204 can include, for example, pre-processing, color interpolation processing, correction processing, detection processing, data processing, special effect processing, and the like. Pre-processing may include signal amplification, reference level adjustment, defective pixel correction, etc. Color interpolation, also known as demosaicing, is performed when the image sensor is equipped with a color filter, and is a process of interpolating the values ​​of color components that are not included in the individual pixel data that make up the image data. The correction processing can include white balance adjustment, tone correction, correction of image degradation caused by optical aberrations in the imaging optical system (image restoration), correction of the effects of vignetting in the imaging optical system, color correction, and the like. The detection process may include detection of characteristic regions (for example, face regions or human body regions) and their movements, person recognition processing, and the like. Data processing can include processes such as area extraction (trimming), compositing, scaling, encoding and decoding, header information generation (data file generation), etc. Data processing also includes the generation of image data for display or image data for recording. Special effect processing can include adding a blur effect, changing color tones, relighting, and the like. Note that these are examples of processing that can be applied by the image processing circuit 204, and do not limit the processing that can be applied by the image processing circuit 204. For example, the image processing circuit 204 may generate signals and evaluation values ​​generated by the AF circuit 202 and the photometry circuit 201.

[0039] The display unit drive circuit 205 controls the display of the display unit 5 . The EVF drive circuit 206 controls the display of the EVF 6 .

[0040] The nearby object detection circuit 207 controls the operations of the NIR control circuit 208 and the SWIR control circuit 209 in accordance with instructions from the CPU 3. The nearby object detection circuit 207 also determines the presence or absence of a nearby object based on the output of the NIR control circuit 208, and determines the type of nearby object based on the output of the SWIR control circuit 209. Details of the operation of the nearby object detection circuit 207 will be described later. A nearby object is an object whose distance from the image capture device 1 (more strictly, the NIR light emitting unit 8) is less than a predetermined nearby distance. The nearby distance is determined depending on the application, but in this embodiment, the eye looking into the EVF 6 is detected as a nearby object, so it can be about 2 to 3 cm, for example.

[0041] Note that a nearby object can be detected not only by the distance to the object, but also by the time or amount of received light that changes depending on the distance to the object. For example, when measuring the distance to an object using a principle similar to that of distance measurement using the ToF (Time of Flight) method, a nearby object can be detected using a proximity threshold that indicates the time equivalent to the round trip time / speed of light for the proximity distance. Furthermore, when detecting the distance to an object using the amount of received light of the NIR light receiving unit 9, a nearby object can be detected using a proximity threshold that indicates the amount of received light corresponding to the proximity distance.

[0042] The NIR control circuit 208 controls the operations of the NIR light emitter 8 and the NIR light receiver 9. The NIR control circuit 208 also outputs the output of the NIR light receiver 9 to the proximity object detection circuit 207.

[0043] The SWIR control circuit 209 controls the operations of the SWIR light emitter 10 and the SWIR light receiver 11. The SWIR control circuit 209 also outputs the output of the SWIR light receiver 11 to the proximity object detection circuit 207.

[0044] The positions, emission directions, and incidence directions of the NIR light emitter 8 and the NIR light receiver 9 are determined so that, when a nearby object is present, the electromagnetic waves emitted by the NIR light emitter 8 are reflected by the nearby object and enter the NIR light receiver 9. The positions, emission directions, and incidence directions of the SWIR light emitter 10 and the SWIR light receiver 11 are also determined in a similar manner.

[0045] The aperture control circuit 212 of the lens unit 1A is included in the aperture driving unit 112, and drives the aperture 111 under the control of the CPU 3.

[0046] <Operation of the proximity object detection circuit> Next, the operation of the proximity object detection circuit 207 to detect a proximity object and determine the type of the proximity object will be described in detail.

[0047] When the imaging device 1 is in, for example, a shooting standby state or a shooting state, the CPU 3 controls the operation of the nearby object detection circuit 207 so as to continuously detect the presence or absence of a nearby object and determine the type of the nearby object.

[0048] Specifically, CPU 3 instructs NIR control circuit 208 and SWIR control circuit 209 to start operation via proximity object detection circuit 207. In response to the instruction, NIR control circuit 208 controls NIR light-emitting unit 8 to emit light periodically for a short period of time. Also, in response to the instruction, SWIR control circuit 209 controls SWIR light-emitting unit 10 to emit light periodically for a short period of time.

[0049] There are no particular restrictions on the light emission cycle of the NIR light-emitting unit 8 and the SWIR light-emitting unit 10, but it can be, for example, one second or less. The light emission cycles of the NIR light-emitting unit 8 and the SWIR light-emitting unit 10 may be the same or different. Furthermore, the light emission timing of the NIR light-emitting unit 8 and the SWIR light-emitting unit 10 may be synchronized or asynchronous. The light emission time may be very short (for example, about several tens of milliseconds), but is determined so that a sufficient amount of light is received when a nearby object is present.

[0050] Here, the NIR light-emitting unit 8 and the SWIR light-emitting unit 10 are caused to emit light periodically. However, light may be emitted continuously, and the light-shielding member may be periodically controlled to be in a light-transmitting state, so that light (electromagnetic waves) is emitted periodically.

[0051] Furthermore, although the NIR light emitting section 8 and the SWIR light emitting section 10 are shown as separate components in FIG. 3, a single light emitting section capable of emitting both near-infrared light and short-wave infrared light may be used. Similarly, the NIR light receiving unit 9 and the SWIR light receiving unit 11 may be a single light receiving unit having sensitivity to both near-infrared light and short-wave infrared light. In this case, by shifting the emission timing of the NIR light emitter 8 and the SWIR light emitter 10, it is possible to determine whether the output of a single light receiving unit corresponds to near-infrared light or short-wave infrared light.

[0052] The NIR control circuit 208 also monitors the output of the NIR light receiving unit 9 and determines whether the output of the NIR light receiving unit 9 exceeds the proximity threshold within a certain time period from the emission of the NIR light emitter 8. If the NIR control circuit 208 determines that the output of the NIR light receiving unit 9 exceeds the proximity threshold, it determines that an object is approaching. If the NIR control circuit 208 does not determine that the output of the NIR light receiving unit 9 exceeds the proximity threshold, it determines that an object is not approaching. Alternatively, the NIR control circuit 208 detects the time from the emission of the NIR light emitter 8 until the output of the NIR light receiving unit 9 exceeds the detection threshold for the reflected wave. The NIR control circuit 208 then determines whether the detected time is equal to or less than the proximity threshold. If the detected time is equal to or less than the proximity threshold, the NIR control circuit 208 determines that an object is approaching. If the detected time is not equal to or less than the proximity threshold, the NIR control circuit 208 determines that an object is not approaching. Note that the certain time period is equal to or less than the light emission period. The NIR control circuit 208 is also a proximity detection means that detects the proximity of an object without using short-wave infrared light.

[0053] The SWIR control circuit 209 (discrimination means) monitors the output of the SWIR light receiving unit 11 and determines whether the output of the SWIR light receiving unit 11 exceeds the detection threshold for reflected waves within a certain period of time from the emission of the SWIR light emitter 10. If it is determined that the output of the SWIR light receiving unit 11 exceeds the detection threshold, the SWIR control circuit 209 detects the rate or proportion of decrease in the amount of received light relative to the amount of emitted light. Note that the certain period of time is set to be equal to or shorter than the emission period. Furthermore, the detection threshold applied to the output of the SWIR light receiving unit 11 and the detection threshold applied to the output of the NIR light receiving unit 9 may be different.

[0054] The NIR light emitter 8 (second emitter) emits near-infrared light (second electromagnetic waves), which are electromagnetic waves with a peak wavelength of 750 nm or more and less than 1000 nm. Near-infrared light is less easily absorbed by water than short-wave infrared light. The NIR light emitter 8 may be, for example, a light-emitting diode. Here, since the nearby object is assumed to be an eye, invisible electromagnetic waves are used so that the emitted light does not interfere with observation through the EVF 6, but in principle visible light could also be used.

[0055] The NIR light receiving unit 9 (first detection means) supplies an output (e.g., a voltage, a count value, etc.) having a magnitude corresponding to the amount of received near-infrared light (incident electromagnetic wave intensity) to the NIR control circuit 208. The NIR light receiving unit 9 has an optical bandpass filter whose transmission range includes the emission peak wavelength of the NIR light emitter 8, and may be a phototransistor or photodiode that outputs a voltage corresponding to the incident intensity. The NIR light receiving unit 9 may also output this voltage as a digital value.

[0056] Instead of using the combination of the NIR light emitter 8 and the NIR light receiver 9, the presence or absence of a nearby object may be detected by using other known configurations such as a capacitance sensor or an ultrasonic sensor.

[0057] Furthermore, the SWIR light emitter 10 (first emitting means) emits shortwave infrared light (first electromagnetic wave), which is an electromagnetic wave with a peak wavelength of 1000 nm or more and less than 2500 nm. The SWIR light emitter 10 may be, for example, a light-emitting diode. Shortwave infrared light is an example of an electromagnetic wave that is easily absorbed by water, and other electromagnetic waves with similar properties, such as microwaves, may also be used. When using shortwave infrared light, the peak wavelength is preferably 1200 nm or more and less than 2500 nm, more preferably 1400 nm or more and less than 2500 nm, and even more preferably 1450 nm or 1940 nm.

[0058] This is because the absorption rate of water for short-wave infrared light increases significantly above 1200 nm, especially above 1400 nm. In particular, 1450 nm (absorption band due to the stretching vibration of HOH) and 1940 nm (absorption band due to the stretching vibration and bending vibration of OH) are peaks in the absorption spectrum of water, and are wavelengths where the absorption rate of light is particularly high.

[0059] The human body is said to be composed of more than 60% water, and therefore has the property of absorbing short-wave infrared light. In particular, the surface of the eyes is covered with tears, and therefore has a higher water content than the skin. In this embodiment, the presence or absence of a nearby object is detected using electromagnetic waves that have a low absorption rate (high reflectance) by water, such as near-infrared light. Furthermore, when a nearby object is detected, the absorption rate (or reflectance) of electromagnetic waves of a wavelength that has a high absorption rate (low reflectance) by water is detected to determine whether the nearby object is a human body.

[0060] In fact, instead of detecting water absorption or reflectance, The difference (reduction amount) between the emitted intensity of the electromagnetic wave and the incident intensity of the electromagnetic wave reflected by a nearby object, The ratio or percentage (%) of the incident intensity of the electromagnetic wave reflected by a nearby object to the outgoing intensity of the electromagnetic wave, It is possible to determine whether a nearby object is a human body by using a value that serves as an index of either of the above. The outgoing intensity may be the amount of emitted light, and the incoming intensity may be the amount of received light. For example, if the amount of reduction is equal to or greater than a first threshold, or the ratio is equal to or less than a second threshold, it can be determined that the absorptance is high (the reflectance is low) and the nearby object is a human body. If the amount of reduction is less than the first threshold, or the ratio is greater than the second threshold, it can be determined that the absorptance is low (the reflectance is high) and the nearby object is not a human body. Here, the first threshold and the second threshold can be set separately.

[0061] The SWIR light receiving unit 11 (second detection means) supplies an output (e.g., a voltage, a count value, etc.) having a magnitude corresponding to the incident intensity of the electromagnetic wave to the SWIR control circuit 209. The SWIR light receiving unit 11 has an optical bandpass filter whose transmission band includes the emission peak wavelength of the SWIR light emitter 10, and may be a phototransistor or photodiode that outputs a voltage corresponding to the incident intensity. The SWIR light receiving unit 11 may also output this voltage as a digital value.

[0062] Here, the detection of a nearby object and the determination of the type of nearby object are performed using two sets of light-emitting units and light-receiving units. However, the detection of a nearby object and the determination of the type of nearby object may also be performed using one set of light-emitting unit and light-receiving unit (e.g., a SWIR light-emitting unit and a SWIR light-receiving unit). For example, if the time difference between the timing of light emission and the timing of detection of reflected light is equal to or less than a proximity threshold and the difference between the amount of light emitted and the amount of light received is less than a first threshold, it can be determined that the nearby object is not a human body (eye). Also, if the time difference is equal to or less than the proximity threshold and the difference between the amount of light emitted and the amount of light received is equal to or greater than the first threshold, it can be determined that the nearby object is a human body (eye). Note that if the time difference exceeds the proximity threshold, it is determined that no nearby object is present.

[0063] Alternatively, one light-emitting unit may be combined with two light-receiving units having different sensitivity wavelengths to detect nearby objects and determine the type of nearby object. For example, a light-emitting unit having an emission wavelength range including wavelengths a and b, a first light-receiving unit having sensitivity to wavelength a, and a second light-receiving unit having sensitivity to wavelength b may be used. The presence or absence of a nearby object can be determined based on the output of one light-receiving unit, and the type of nearby object can be determined based on the output of the other light-receiving unit.

[0064] Note that when determining the type of nearby object after it has been determined that a nearby object is present, there is no need to calculate the difference or ratio described above. If the output of the SWIR light receiving unit 11 does not exceed the SW threshold within a certain time after the SWIR light emitter 10 emits electromagnetic waves, the nearby object can be determined to be a human body. The SW threshold can be determined based on the electromagnetic wave absorption rate or reflectance that must be met if the nearby object is an object containing moisture (here, a human body).

[0065] 3(a) shows a schematic diagram of the state in which the NIR light-emitting unit 8 and the SWIR light-emitting unit 10 emit electromagnetic waves when there is no nearby object. Here, the NIR light-emitting unit 8 and the SWIR light-emitting unit 10 emit electromagnetic waves simultaneously, but their emission periods do not need to overlap. Furthermore, only the NIR light-emitting unit 8 may emit electromagnetic waves until a nearby object is detected. When there is no nearby object, the electromagnetic waves reflected by the object do not enter the NIR light-receiving unit 9 and the SWIR light-receiving unit 11, or, even if they do, the time difference from emission, the incident intensity, or the distance from the object do not satisfy the proximity threshold conditions.

[0066] 3(b) shows a state in which, in the state of FIG. 3(a), an object 300 that does not contain water is approaching the eyepiece 7 of the imaging device 1. In this case, the electromagnetic waves emitted by the NIR light-emitting unit 8 and the SWIR light-emitting unit 10 are reflected by the object 300 and enter the NIR light-receiving unit 9 and the SWIR light-receiving unit 11, respectively. When the output of the NIR light-receiving unit 9 exceeds the proximity threshold, the presence of the approaching object 300 is detected. Furthermore, when the output of the SWIR light-receiving unit 11 exceeds the SW threshold, the approaching object 300 is determined to be not a human body.

[0067] 3(c) shows a state in which object 310, which is an eye, is in proximity to the eyepiece 7 of the imaging device 1 in the state of FIG. 3(a). In this case, the electromagnetic waves emitted by the NIR light emitter 8 and the SWIR light emitter 10 are reflected by the object 310 and enter the NIR light receiver 9 and the SWIR light receiver 11, respectively. The presence of the nearby object 310 is detected when the output of the NIR light receiver 9 exceeds the proximity threshold within the time threshold after the NIR light emitter 8 emits the electromagnetic waves. Furthermore, the output of the SWIR light receiver 11 does not exceed the SW threshold, so the nearby object 310 is identified as a human body.

[0068] If the electromagnetic waves emitted by the SWIR light-emitting unit 10 have a very high water absorption rate, it is necessary to detect the presence of a nearby object using electromagnetic waves with a low water absorption rate (using the NIR light-emitting unit 8 and the NIR light-receiving unit 9). This is because it may be difficult to distinguish, from the output of the SWIR light-receiving unit 11, between the case where there is no nearby object (FIG. 3(a)) and the case where the nearby object is a human body (FIG. 3(c)). On the other hand, if it is possible to distinguish, from the output of the SWIR light-receiving unit 11, between the case where there is no nearby object and the case where the nearby object is a human body, the NIR light-emitting unit 8 and the NIR light-receiving unit 9 are not necessary.

[0069] 4 is a flowchart showing an example in which the detection of the presence or absence of a nearby object and the determination of its type are used to control the state of the imaging device 1. The operations described below are realized by the CPU 3 executing a program. The operations described below can also be performed in parallel with any other operations of the imaging device 1.

[0070] In S101, the CPU 3 instructs the nearby object detection circuit 207 to start detecting a nearby object, and then executes S102. In response to the instruction from the CPU 3, the nearby object detection circuit 207 enables the operation of the NIR control circuit 208. As a result, the NIR control circuit 208 starts periodically emitting electromagnetic waves from the NIR light emitter 8 and monitoring the output of the NIR light receiver 9. The emission period is assumed to be predetermined.

[0071] The NIR control circuit 208 (proximity detection means) determines whether the output of the NIR light receiving unit 9 exceeds the proximity threshold each time the NIR light emitter 8 emits electromagnetic waves. If the NIR control circuit 208 determines that the output of the NIR light receiving unit 9 exceeds the proximity threshold, it notifies the proximity object detection circuit 207 that the presence of a nearby object has been detected. If it is not determined that the output of the NIR light receiving unit 9 exceeds the proximity threshold within a predetermined time after the NIR light emitter 8 emits electromagnetic waves, the NIR control circuit 208 may or may not notify the proximity object detection circuit 207 that the presence of a nearby object has not been detected. The predetermined time is shorter than the emission period. If the presence or absence of a nearby object is detected based on the time difference between the emission timing and the detection timing of reflected light, the presence of a nearby object is deemed to be detected if it is determined that the output of the NIR light receiving unit 9 exceeds the detection threshold within the proximity threshold after the NIR light emitter 8 emits electromagnetic waves. The proximity object detection circuit 207 transfers the notification from the NIR control circuit 208 to the CPU 3.

[0072] In S102, the CPU 3 determines whether or not it has received a notification from the proximity object detection circuit 207 indicating that the presence of a proximity object has been detected, and if it is determined that it has been received, it performs S103, and if it is not determined that it has been received, it performs S107.

[0073] In S103, the CPU 3 instructs the nearby object detection circuit 207 to start the operation of determining the type of nearby object, and then executes S104. In response to the instruction from the CPU 3, the nearby object detection circuit 207 enables the operation of the SWIR control circuit 209. As a result, the SWIR control circuit 209 starts cyclically emitting electromagnetic waves from the SWIR light emitter 10 and monitoring the output of the SWIR light receiver 11. The emission period is assumed to be predetermined. Note that the CPU 3 may also instruct the nearby object detection circuit 207 to stop the operation of detecting nearby objects in S103.

[0074] The SWIR control circuit 209 determines whether the output of the SWIR light receiving unit 11 exceeds the SW threshold each time the SWIR light emitter 10 emits electromagnetic waves. If the SWIR control circuit 209 determines that the output of the SWIR light receiving unit 11 exceeds the SW threshold, it determines that the nearby object is an object that does not contain moisture (not a human body). Furthermore, if the SWIR control circuit 209 does not determine that the output of the SWIR light receiving unit 11 exceeds the SW threshold within a predetermined time, it determines that the nearby object is an object that contains moisture (a human body). The predetermined time is shorter than the emission period. The SWIR control circuit 209 notifies the nearby object detection circuit 207 of the determined type. The nearby object detection circuit 207 transfers the notification from the SWIR control circuit 209 to the CPU 3.

[0075] In S104, the CPU 3 refers to the notification indicating the result of discrimination of the type of the nearby object received from the nearby object detection circuit 207, and if the nearby object is discriminated as a human body, it executes S105, and if not, it executes S106.

[0076] In addition, while executing the processing from S102 onwards, the CPU 3 may execute S107 if it stops receiving a notification from the proximity object detection circuit 207 indicating that the presence of a proximity object has been detected, or if it receives a notification indicating that the presence of a proximity object is not detected.

[0077] In S105, the CPU 3 executes processing when the nearby object is determined to be a human body. For example, the CPU 3 enables the operation of the EVF 6, starts supplying power to the EVF 6, and starts display operation on the EVF 6. The CPU 3 also disables the operation of the display unit 5, and stops supplying power to the display unit 5.

[0078] In S106, the CPU 3 executes processing for when the approaching object is determined to be an object that does not contain moisture (not a human body). In this case, the user of the imaging device 1 is not looking through the EVF 6, but some object is present near the eyepiece 7. For example, this state occurs when the user is wearing the imaging device 1 around their neck or when the user puts the imaging device 1 in a bag without turning off the power. The CPU 3, for example, disables the operation of the EVF 6 and stops the supply of power to the EVF 6 and the display operation on the EVF 6. Furthermore, the CPU 3 may disable the operation of the display unit 5 and stop the supply of power to the display unit 5 if the approaching object is determined to be an object that does not contain moisture for a certain period of time.

[0079] In S107, the CPU 3 executes processing for the case where no nearby object is present. For example, the CPU 3 starts displaying a live view image on the display unit 5 while disabling the operation of the EVF 6 and stopping the supply of power to the EVF 6 and the display operation on the EVF 6. Furthermore, in order to reduce power consumption of the imaging device 1, the CPU 3 may instruct the nearby object detection circuit 207 to end the operation of determining the type of nearby object. This stops the operation of the SWIR control circuit 206, the SWIR emitter 10, and the SWIR receiver 11.

[0080] In S108, the CPU 3 determines whether or not a power-off command has been issued by operating the power switch included in the operation unit 203. If it is determined that a power-off command has been issued, the CPU 3 executes a predetermined power-off process, and if it is not determined that a power-off command has been issued, the CPU 3 executes S101 again.

[0081] Note that the processes described in S105 to S107 that are executed depending on the presence or absence and type of nearby object are merely examples, and other operations may be executed. Furthermore, the operation to be executed may be determined by further considering other information, such as the movement and orientation of the image capture device 1 and information obtained from live view images.

[0082] As described above, according to this embodiment, the type of a nearby object can be determined using a simple configuration that emits electromagnetic waves and detects the reflected waves. Therefore, complex and expensive configurations such as an imaging optical system or an imaging element are not required. Furthermore, if the configuration also serves to detect nearby objects, there is no need to provide a new configuration for determining the type of object.

[0083] ●(Second embodiment) Next, an embodiment in which the present invention is applied to an HMD (head-mounted display device) will be described. Note that the present invention can also be applied to eyeglass-type displays called XR goggles, XR glasses, smart glasses, etc. XR is a general term for virtual reality technologies such as VR (Virtual Reality), AR (Augmented Reality), and MR (Mixed Reality).

[0084] Fig. 5 is a diagram showing an example of the configuration of the HMD 300 according to this embodiment. Fig. 5 shows a vertical cross section of the part that is located in front of the face when the HMD 300 is worn, and does not show components (band and temples) for supporting the HMD 300 on the head or ears. Furthermore, the same reference numerals as in Fig. 2 are used to designate the same components as those in the imaging device 1.

[0085] The HMD 300 has a left-eye display unit 311 and a right-eye display unit 312. The configurations of the display units 311 and 312 may differ depending on whether the HMD 300 is an immersive type, an optical see-through type, or a video see-through type.

[0086] In this embodiment, the NIR light emitter 8, NIR light receiver 9, SWIR light emitter 10, and SWIR light receiver 11 are positioned near the user's nose when the HMD 300 is in use. This is to prevent electromagnetic waves emitted by the NIR light emitter 8 and SWIR light receiver 10 from being reflected by the eyeglass lenses when the user wears eyeglasses and the HMD 300. If electromagnetic waves are reflected by the eyeglass lenses, a nearby object may be determined to be a human body even though it is actually a human body. Therefore, nearby object detection and type determination are generally performed near the nose, where eyeglass lenses are not located. While nearby object detection and type determination may be performed near the head or forehead, because hair has a lower moisture content than skin, more accurate determination is possible by performing nearby object detection and type determination in areas where hair is less likely to be in proximity.

[0087] 5 is a vertical cross-sectional view, so the light-emitting unit and light-receiving unit are directly visible, but in reality there is a cover member in front of them. The cover member is made of a material that is transparent to the electromagnetic waves emitted by the NIR light-emitting unit 8 and the SWIR light-emitting unit 10, or has holes in the direction of emission and incidence of the electromagnetic waves.

[0088] When the HMD 300 is of the video see-through type, the HMD 300 has a functional configuration that is almost the same as that of the imaging device 1, except that the EVF 6 and the display unit 5 are replaced with display units 311 and 312. Therefore, the operations described in the first embodiment with reference to FIG. 4 can be performed by the CPU 3, the nearby object detection circuit, the NIR control circuit 208, the SWIR control circuit 209, and the like of the HMD 300.

[0089] Then, in S105 to S107 of FIG. 4, the CPU 3 can execute the following processes depending on whether an object is present close to the HMD 300 and whether the close object is determined to be a human body.

[0090] In S105, the CPU 3 executes a process for when the nearby object is determined to be a human body. For example, the CPU 3 enables the operation of the display units 311 and 312, and starts supplying power to the display units 311 and 312 and displaying the images on the display units 311 and 312.

[0091] In S106, the CPU 3 executes processing for when the approaching object is determined to be an object that does not contain moisture (not a human body). In this case, the CPU 3, for example, disables the operation of the display units 311 and 312, and stops the supply of power to the display units 311 and 312 and the display operation on the display units 311 and 312. Furthermore, when the approaching object is determined to be an object that does not contain moisture for a certain period of time, the CPU 3 turns off the power of the HMD 300 or puts the HMD 300 into a sleep state.

[0092] In S107, the CPU 3 executes processing for the case where no nearby object is present. For example, the CPU 3 executes the same processing as when the nearby object is determined to be an object that does not contain water (not a human body). In addition, to reduce power consumption of the MHD 300, the CPU 3 may instruct the nearby object detection circuit 207 to end the operation of determining the type of nearby object. This stops the operations of the SWIR control circuit 206, the SWIR emitter 10, and the SWIR receiver 11.

[0093] According to this embodiment, the same effects as those of the first embodiment can be achieved in the HMD.

[0094] (Other embodiments) In the above-described embodiment, the presence or absence of an object in the vicinity of the electronic device and the type of the object are determined, but a change in the distance to the object may also be detected from a change over time in the time difference between the emission of the electromagnetic wave and the detection of the reflected wave, or a change over time in the strength of the reflected wave.

[0095] Furthermore, the operations described in the above embodiments as being performed by a CPU may be performed by a plurality of hardware components sharing the processing.

[0096] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0097] The disclosure of the present embodiment includes the following electronic device, control method for the electronic device, and program. (Item 1) a first emitting means for emitting a first electromagnetic wave having a property of being absorbed by water; a first detecting means for detecting a reflected wave of the first electromagnetic wave; a discrimination means for discriminating whether the object that reflected the first electromagnetic wave is an object that contains water or an object that does not contain water, depending on the intensity of the reflected wave; An electronic device comprising: (Item 2) Item 10. The electronic device according to item 1, wherein the first electromagnetic wave is short-wave infrared light or microwave. (Item 3) Item 1. The electronic device according to item 1, wherein the first electromagnetic wave is short-wave infrared light having a peak wavelength of 1000 nm or more and less than 2500 nm. (Item 4) The electronic device described in any one of items 1 to 3, characterized in that the discrimination means performs the discrimination when the distance between the object and the electronic device is less than a threshold value, and does not perform the discrimination when the distance between the object and the electronic device is equal to or greater than the threshold value. (Item 5) The electronic device described in item 4, characterized in that the discrimination means determines whether the distance between the object and the electronic device is less than a threshold value based on the time difference between the emission of the first electromagnetic wave and the detection of the reflected wave, or the intensity of the reflected wave. (Item 6) 5. The electronic device according to item 4, further comprising a proximity detection means for determining whether the distance between the object and the electronic device is less than a threshold value without using a reflected wave of the first electromagnetic wave. (Item 7) a second emitting means for emitting a second electromagnetic wave that is less easily absorbed by water than the first electromagnetic wave; and a second detection means for detecting a reflected wave of the second electromagnetic wave, the proximity detection means determines whether the distance to the electronic device is less than a threshold value based on the time difference between the emission of the second electromagnetic wave and the detection of the reflected wave of the second electromagnetic wave, or based on the intensity of the reflected wave. 7. The electronic device according to item 6. (Item 8) 8. The electronic device according to item 7, wherein the second electromagnetic wave is near-infrared light having a peak wavelength of 750 nm or more and less than 1000 nm. (Item 9) 9. The electronic device described in any one of items 1 to 8, wherein the first emission means and the first detection means are provided in a location close to a user of the electronic device when the electronic device is in use. (Item 10) Item 10. The electronic device described in item 9, wherein the electronic device is a head-mounted display device, and the first emission means and the first detection means are provided in a position close to the user's nose when the user wears the electronic device. (Item 11) 11. The electronic device according to any one of items 1 to 10, further comprising an execution unit that, when the discrimination unit discriminates that the object that reflected the first electromagnetic wave is an object containing moisture, executes processing assuming that the object has been discriminated as a human body. (Item 12) A control method executed by an electronic device, comprising: Emitting a first electromagnetic wave that has a property of being absorbed by water; detecting a reflected wave of the first electromagnetic wave; determining whether the object that reflected the first electromagnetic wave is an object that contains water or an object that does not contain water, according to the intensity of the reflected wave; 1. A method for controlling an electronic device, comprising: (Item 13) 12. A program for causing a computer included in an electronic device having a first emission means for emitting a first electromagnetic wave having a property of being absorbed by water and a first detection means for detecting a reflected wave of the first electromagnetic wave to function as the first detection means included in the electronic device described in any one of items 1 to 11.

[0098] The present invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Therefore, the following claims are appended to clarify the scope of the invention. [Explanation of symbols]

[0099] 1: imaging device, 3: CPU, 8: NIR light emitting unit, 9: NIR light receiving unit, 10: SWIR light emitting unit, 11: SWIR light receiving unit, 300: HMD

Claims

1. a first emitting means for emitting a first electromagnetic wave having a property of being absorbed by water; a first detecting means for detecting a reflected wave of the first electromagnetic wave; a determination means for determining whether the object that has reflected the first electromagnetic wave is an object that contains water or an object that does not contain water, depending on the intensity of the reflected wave; An electronic device comprising:

2. 2. The electronic device according to claim 1, wherein the first electromagnetic wave is short-wave infrared light or microwave.

3. 2. The electronic device according to claim 1, wherein the first electromagnetic wave is short-wave infrared light having a peak wavelength of 1000 nm or more and less than 2500 nm.

4. 2. The electronic device according to claim 1, wherein the discrimination means performs the discrimination when the distance between the object and the electronic device is less than a threshold value, and does not perform the discrimination when the distance between the object and the electronic device is equal to or greater than the threshold value.

5. 5. The electronic device according to claim 4, wherein the discrimination means determines whether the distance between the object and the electronic device is less than a threshold value based on the time difference between the emission of the first electromagnetic wave and the detection of the reflected wave, or based on the intensity of the reflected wave.

6. 5. The electronic device according to claim 4, further comprising a proximity detection unit that determines whether or not the distance between the object and the electronic device is less than a threshold value without using a reflected wave of the first electromagnetic wave.

7. a second emitting means for emitting a second electromagnetic wave that is less easily absorbed by water than the first electromagnetic wave; and a second detection means for detecting a reflected wave of the second electromagnetic wave, the proximity detection means determines whether the distance to the electronic device is less than a threshold value based on the time difference between the emission of the second electromagnetic wave and the detection of the reflected wave of the second electromagnetic wave, or based on the intensity of the reflected wave.

7. The electronic device according to claim 6, wherein the electronic device is a semiconductor device.

8. 8. The electronic device according to claim 7, wherein the second electromagnetic wave is near-infrared light having a peak wavelength of 750 nm or more and less than 1000 nm.

9. 2. The electronic device according to claim 1, wherein the first emitting means and the first detecting means are provided at locations close to a user of the electronic device when the electronic device is in use.

10. 10. The electronic device according to claim 9, wherein the electronic device is a head-mounted display device, and the first emission means and the first detection means are provided in a position close to the nose of the user when the user wears the electronic device.

11. 2. The electronic device according to claim 1, further comprising an execution means for executing processing assuming that the object has been identified as a human body when the discrimination means discriminates that the object that reflected the first electromagnetic wave is an object containing moisture.

12. A control method executed by an electronic device, comprising: Emitting a first electromagnetic wave that has a property of being absorbed by water; detecting a reflected wave of the first electromagnetic wave; determining whether the object that reflected the first electromagnetic wave is an object that contains water or an object that does not contain water, according to the intensity of the reflected wave; 1. A method for controlling an electronic device, comprising:

13. A program for causing a computer possessed by an electronic device having a first emission means for emitting a first electromagnetic wave having a property of being absorbed by water and a first detection means for detecting a reflected wave of the first electromagnetic wave to function as the first detection means possessed by the electronic device described in any one of claims 1 to 11.

Images