Photographing apparatus, and method for controlling photographing apparatus

Abstract

To provide a technique to improve convenience of an apparatus that authenticates and records a photographer.SOLUTION: A photographing apparatus specifies a user of the photographing apparatus, and determines the state of continuation of the use of the photographing apparatus by the user. The photographing apparatus performs control so that an image photographed by the photographing apparatus in a period from when the apparatus specified the user until when the apparatus determines that the use of the apparatus by the user is not continued, and information on the user are recorded in a memory in association with each other.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a technique for authenticating and recording a photographer. [Background technology]

[0002] In recent years, there has been a demand for a method for identifying and recording the photographer of an image or video in order to manage copyrights and guarantee authenticity. Patent Document 1 proposes a method for monitoring the biometric information of the photographer when identifying and recording the photographer in an image file, and if there is a change, it is determined that the photographer has been replaced and the authentication of the photographer is cancelled. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2018-191194 Summary of the Invention [Problem to be solved by the invention]

[0004] In the method of Patent Document 1, authentication may be cancelled between successive shots, reducing the convenience of photography. The present invention provides a technology for improving the convenience of a device that authenticates and records photographers. [Means for solving the problem]

[0005] One aspect of the present invention is an imaging device comprising: an identification means for identifying a user of the imaging device; a determination means for determining whether the user is continuing to use the imaging device; and a control means for controlling the imaging device so that images captured by the imaging device and information about the user are associated and recorded in memory between the time when the identification means identifies the user and the time when the determination means determines that the user is no longer using the device. [Effects of the Invention]

[0006] According to the present invention, it is possible to provide a technique for improving the convenience of a device that authenticates a photographer and records the image. [Brief explanation of the drawings]

[0007] [Figure 1] 1A is a front perspective view of the camera 100, FIG. 1B is a rear perspective view of the camera 100, and FIG. 1C is a diagram illustrating an operation lever. [Figure 2] FIG. 2 is a cross-sectional view of the camera 100 cut along the YZ plane. [Figure 3] FIG. 2 is a block diagram showing the electrical configuration within the camera 100. [Figure 4] FIG. 1 is a block diagram showing an example of the functional configuration of a camera 100. [Figure 5] 1 is a flowchart of processing performed by the camera 100. [Figure 6] 10 is a flowchart showing details of the process in step S503. [Figure 7] 10 is a flowchart showing details of the process in step S503. [Figure 8] 10 is a flowchart showing details of the process in step S503. [Figure 9] (a) is a diagram showing an example of a screen displayed on the display panel in the "continued use by user P" state, (b) is a diagram showing an example of a screen displayed on the display panel in the "unidentified user" state, and (c) is a diagram showing an example of a screen displayed when it is determined that user P of camera 100 is not continuing to use camera 100. [Figure 10] FIG. 10 is a diagram showing an example of a screen displayed on the display panel when a setting mode is touched. [Figure 11] FIG. 20 is a diagram showing an example of the configuration of a camera device according to the sixth embodiment. [Figure 12] FIG. 13 is a block diagram showing an example of the electrical configuration of a camera device according to a sixth embodiment. [Figure 13] 13 is a flowchart showing a procedure for determining a user's continuous use state in the sixth embodiment. [Figure 14]13 is a flowchart showing a procedure for determining a user's continuous use state in the seventh embodiment. [Figure 15] FIG. 1 is a diagram illustrating the principle of an eyeball image analysis method. [Figure 16] 1 is a schematic diagram of an eyeball image projected onto an eyeball image sensor. [Figure 17] 1 is a schematic flowchart of the present eye image analysis. [Figure 18] 13 is a flowchart showing a procedure for determining a user's continuous use state in the eighth embodiment. [Figure 19] FIG. 23 is a diagram showing an example of the configuration of a camera device according to the ninth embodiment. [Figure 20] FIG. 23 is a block diagram showing an example of the electrical configuration of a camera device according to a ninth embodiment. [Figure 21] 13 is a flowchart showing a procedure for determining a user's continuous use state in the ninth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention claimed. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0009] [First embodiment] First, an example of the appearance of a camera 100 (digital still camera; interchangeable lens camera), which is an example of an imaging device according to this embodiment, will be described with reference to Fig. 1. Fig. 1(a) is a front perspective view of the camera 100, and Fig. 1(b) is a rear perspective view of the camera 100. For the sake of explanation, in Fig. 1, the optical axis direction of the camera 100 is defined as the Z-axis direction, and of the two axes perpendicular to the Z-axis, the horizontal direction is defined as the X-axis and the vertical direction is defined as the Y-axis.

[0010] 1(a), camera 100 has photographing lens unit 100A and camera housing 100B. Camera housing 100B is provided with release button 101, which is an operating member that receives operational input for photographing from the user of camera 100 (photographer).

[0011] As shown in FIG. 1(b), an eyepiece 102 (finder) through which the user looks to view a display device 214 (display panel) included in the camera housing 100B is disposed on the back of the camera housing 100B. Also disposed on the back of the camera housing 100B are operation members 103-105 that accept various operations from the user. The operation member 103 is, for example, a touch panel that accepts touch operations from the user, and the operation member 104 is, for example, an operation lever that can be pushed down in all directions as shown in FIG. 1(c). The operation member 105 is, for example, a four-way key that can be pressed in each of four directions. The operation member 103 (touch panel) is equipped with a display panel (for example, a liquid crystal panel), and various information such as images and text is displayed on the display panel.

[0012] 2 is a cross-sectional view of camera 100 cut along the YZ plane defined by the Y axis and Z axis shown in Fig. 1(a), and shows a rough internal configuration of camera 100, including the configuration related to the following description. In Fig. 2, functional units similar to those shown in Fig. 1 are assigned the same reference numerals as those functional units, and descriptions of those functional units will be omitted.

[0013] The photographing lens unit 100A includes two lenses 201 and 202, an aperture 203, an aperture driver 204, a lens drive motor 205, a lens driver 206, a photocoupler 207, a pulse plate 208, a mount contact 209, and a focus adjustment circuit 210.

[0014] Lens drive member 206 includes a drive gear, and photocoupler 207 detects the rotation of pulse plate 208, which is linked to lens drive member 206, and transmits the detection result to focus adjustment circuit 210. Focus adjustment circuit 210 drives lens drive motor 205 based on the detection result from photocoupler 207 and information (lens drive amount information) from camera housing 100B, moving lens 201 and changing the focus position. Mount contact 209 is an interface for communication between photographing lens unit 100A and camera housing 100B. While FIG. 2 shows two lenses 201 and 202 for simplicity, more than two lenses are actually included in photographing lens unit 100A. Aperture drive unit 204 drives and controls aperture 203 to open and close the aperture.

[0015] The camera housing 100B includes an image sensor 211, a CPU 212, a memory unit 213, a display device 214, and a display device drive circuit 215. The image sensor 211 photoelectrically converts an image of a subject that has entered through the lenses 201 and 202 into a RAW image. The CPU 212 is a central processing unit of a microcomputer and controls the overall operation of the camera 100 using computer programs and data stored in the memory unit 213. The memory unit 213 is a memory unit that can store various information. For example, the CPU 212 performs various image processing, including development processing, on a RAW image output from the image sensor 211 to generate a captured image and stores it in the memory unit 213. The display device 214 displays various information, such as images and text. For example, the display device 214 is a liquid crystal panel that displays the captured image and various information. The display device drive circuit 215 drives and controls the display device 214. A user can view the screen of the display device 214 through the eyepiece 102.

[0016] Camera housing 100B further includes light sources 216a and 216b, a beam splitter 217, a light receiving lens 218, and an eye imaging element 219. Light sources 216a and 216b are light sources conventionally used in single-lens reflex cameras to detect the line of sight from the relationship between the pupil and a reflection image of light reflected from the cornea (corneal reflection image), and are light sources for illuminating eyeball 220 of a user looking through the viewfinder (eyepiece 102). For example, light sources 216a and 216b are infrared light-emitting diodes that emit infrared light that is insensitive to the user, and are arranged around eyepiece 102. An optical eye image (an optical image formed by light emitted from light sources 216a and 216b and reflected by eyeball 220), which is an optical image of the illuminated eyeball 220, passes through eyepiece 102 and is reflected by beam splitter 217. The eye optical image is then formed by the light receiving lens 218 on the eye imaging element 219, which has a plurality of photoelectric conversion elements (e.g., CCD or CMOS) arranged two-dimensionally. The light receiving lens 218 positions the pupil of the eyeball 220 and the eye imaging element 219 in a conjugate imaging relationship. The line of sight of the eyeball 220 can be detected from the position of the corneal reflection image in the eye optical image formed on the eye imaging element 219, and information about the line of sight is obtained as a result of the detection. For example, the information about the line of sight may include at least one of information indicating the line of sight direction (the direction of the line of sight) and information indicating the viewpoint (the position where the line of sight is fixed) on the screen of the display device 214. The viewpoint can be regarded as the position at which the user is looking or as the line of sight position. The eye imaging element 219 then outputs the formed eye optical image (an image obtained by capturing the eyeball 220) as an eye image (an analog image signal).

[0017] Next, the electrical configuration within camera 100 will be described using the block diagram of Fig. 3. CPU 212 can transmit signals via mount contacts 209 to focus adjustment circuit 210 arranged in photographing lens unit 100A and to aperture control circuit 306 included in aperture drive section 204 within photographing lens unit 100A. For example, CPU 212 can transmit information on the lens drive amount to focus adjustment circuit 210 via mount contacts 209. CPU 212 can also transmit information on the drive amount of aperture 203 to aperture control circuit 306 via mount contacts 209. CPU 212 also controls the operation of each of the other sections.

[0018] The memory unit 213 is a memory unit capable of storing captured images, eye images, and gaze correction parameters, which are parameters for correcting the detected gaze. The gaze detection circuit 301 A / D converts the eye image (analog image signal) output from the eye imaging element 219 when an eye optical image is formed on the eye imaging element 219 to generate an eye image, which is a digital image signal. The gaze detection circuit 301 then extracts feature points necessary for gaze detection from the eye image, which is the generated digital image signal, detects the user's gaze from the positions of the feature points, and outputs information about the gaze as a result of the detection. The gaze detection circuit 301 also stores the eye image, which is the generated digital image signal, in the memory unit 213.

[0019] The photometry circuit 302 performs predetermined processing (e.g., amplification, logarithmic compression, and A / D conversion) on the image signal obtained from the image sensor 211, which also functions as a photometry sensor, such as a luminance signal corresponding to the brightness of the field, and sends the result to the CPU 212 as field luminance information.

[0020] The autofocus detection circuit 303 A / D converts output signals from multiple detection elements (multiple pixels) used for phase difference detection included in the image sensor 211, and calculates the distance to the subject corresponding to each focus detection point from the A / D converted output signals. This is a well-known technique known as image plane phase difference AF.

[0021] The signal input circuit 304 is connected to the switches SW1 and SW2 of the release button 101. The switch SW1 outputs an ON signal when the release button 101 is pressed for the first time, starting the preparatory operations for shooting (e.g., photometry and distance measurement) of the camera 100. The switch SW2 outputs an ON signal when the release button 101 is pressed for the second time, starting the shooting operation. The ON signals from the switches SW1 and SW2 are input to the signal input circuit 304 and transmitted to the CPU 212. When the CPU 212 receives the ON signal from the switch SW1, it controls the operation of each unit to start the preparatory operations for shooting of the camera 100. When the CPU 212 receives the ON signal from the switch SW2, it controls the operation of each unit to start the shooting operation. When the switch SW1 is turned ON, line-of-sight detection may be started. The light source drive circuit 305 drives and controls the light sources 216a and 216b.

[0022] Furthermore, operation members 103 to 105 are also connected to CPU 212. When the user operates operation members 103 to 105, operation members 103 to 105 output operation signals according to the operation from the user to CPU 212, and CPU 212 performs processing (control) according to the operation signals. For example, CPU 212 moves a frame (item, indicator) displayed at the viewpoint according to the operation signal.

[0023] An example of the functional configuration of the camera 100 related to the processing described below is shown in the block diagram of Fig. 4. In this embodiment, each functional unit shown in Fig. 4 is implemented in hardware. Each functional unit can be implemented in one or more pieces of hardware shown in Figs. 2 and 3, and operates under the control of the CPU 212.

[0024] 4 may be implemented as software (computer program). In this case, the computer program is stored in the memory unit 213, and the CPU 212 executes the computer program to realize the functions of the corresponding functional units.

[0025] The process performed by camera 100 according to this embodiment to identify the user of camera 100 and store the captured image in memory unit 213 in association with the identification result will be described with reference to the flowchart in Fig. 5. The process according to the flowchart in Fig. 5 starts when camera 100 is powered on, completes startup of camera 100, and becomes available for use, and is repeated until the process can no longer be continued due to reasons such as the camera 100 being powered off.

[0026] In step S501, the user identification unit 402 performs iris authentication on the eye image stored in the memory unit 213 to perform processing to identify the user of the camera 100. In step S502, the user identification unit 402 determines whether or not the user of the camera 100 has been successfully identified in step S501.

[0027] If the result of this determination is that the user of the camera 100 has been successfully identified in step S501, the process proceeds to step S503, and if the user of the camera 100 has not been successfully identified in step S501, the process proceeds to step S511.

[0028] For example, the user identification unit 402 identifies the registered iris that is most similar to the iris detected from the eye image as the similar registered iris from among the irises of each user (registered irises) pre-registered in the memory unit 213. Then, the user identification unit 402 identifies the user corresponding to the similar registered iris as the user of the camera 100.

[0029] Note that if the similarity between the iris detected from the eye image and the similar registered iris is less than a threshold, the user identification unit 402 determines that user identification has failed. Also, if no one is looking through the eyepiece 102 and therefore the eye image does not contain an iris, the user identification unit 402 determines that user identification has failed because no iris is recognized from the eye image. Also, if something other than an eyeball is placed near the eyepiece 102, the user identification unit 402 determines that user identification has failed because no iris is recognized from the eye image.

[0030] In step S511, continuous use determination unit 403 determines whether or not a photograph has been taken by photographing unit 401 functioning as photographing lens unit 100A, for example by pressing release button 101. If the result of this determination is that a photograph has been taken by photographing unit 401, the process proceeds to step S512, and if a photograph has not been taken by photographing unit 401, the process proceeds to step S501.

[0031] In step S512, the recording unit 404 assigns, as metadata, photographer information indicating that "the photographer is unidentified" to the captured image captured by the imaging unit 401. The recording unit 404 then records the captured image (captured image file) to which the metadata has been assigned in the memory unit 213. The process then proceeds to step S501.

[0032] On the other hand, in step S503, the continuous use determination unit 403 determines whether the user of the camera 100 identified in step S501 (hereinafter referred to as user P) continues to use the camera 100. Details of the processing in step S503 will be described later.

[0033] If the result of such a determination is that user P continues to use camera 100, processing proceeds to step S521 via step S504; if user P does not continue to use camera 100, processing proceeds to step S501 via step S504.

[0034] In step S521, similar to step S511 above, continuous use determination unit 403 determines whether or not a photograph has been taken by photographing unit 401, for example by pressing release button 101. If the result of this determination is that a photograph has been taken by photographing unit 401, the process proceeds to step S522, and if a photograph has not been taken by photographing unit 401, the process proceeds to step S503.

[0035] In step S522, the recording unit 404 assigns, as metadata, photographer information indicating that "the photographer is user P" to the captured image captured by the imaging unit 401. Then, the recording unit 404 records the captured image (captured image file) to which the metadata has been assigned in the memory unit 213. Then, the process proceeds to step S503.

[0036] In this way, the continued use determination unit 403 determines the period during which user P continues to use the camera 100 as the "range of continued use by user P," and images taken within this range of continued use are assigned metadata containing photographer information such as "the photographer is user P."

[0037] The method for recording photographer information is not limited to adding it to the captured image as metadata. For example, it may be stored in the form of a table in which the hash value of the captured image corresponds to the photographer information, or it may be connected to an external server via a communication channel such as wireless communication and uploaded as a record combining the captured image and the photographer information. Furthermore, methods for preventing tampering and ensuring authenticity, such as encryption or electronic signature, may also be used for this information. Furthermore, in step S512, the recording unit 404 may store the captured image captured by the image capturing unit 401 in the memory unit 213 without adding photographer information indicating that the "photographer is unidentified" to the captured image.

[0038] In this way, the recording unit 404 controls the recording so that images taken by the camera 100 are associated with information about the user and recorded in the memory unit 213 between the time the user of the camera 100 is identified and the time it is determined that the user is no longer using the camera.

[0039] In addition, if the user of camera 100 cannot be identified, recording unit 404 controls the recording so that the image captured by camera 100 and information indicating that the user of camera 100 is unidentified are recorded in memory in association with each other.

[0040] Next, details of the process in step S503 above will be described with reference to the flowchart in Fig. 6. In step S601, the continued use determination unit 403 determines whether the end button on at least one of the operation members 103 to 105 has been pressed by the user P. For example, if the operation member 103 is the display panel shown in Fig. 9(a), and the user P touches button 902 displayed on the display panel, the continued use determination unit 403 determines that the end button on the operation member 103 has been pressed by the user P.

[0041] The end button is not limited to a button on the display panel, but may be provided as a physical button. Also, when an operation lever such as the operation member 104 is pressed, it may be determined that the end button of the operation member 104 has been pressed. Also, when a power button or the like is pressed, it may be determined that the end button has been pressed. If it is determined that the end button has been pressed, the process proceeds to step S605, and if the end button has not been pressed, the process proceeds to step S602.

[0042] In step S602, the continued use determination unit 403 determines whether or not the user P has performed a mode change operation on at least one of the operation members 103 to 105. For example, if the operation member 103 is the display panel shown in Fig. 9(a), when the user P performs a touch operation on a mode different from the current mode among the three modes included in the list 903 displayed on the display panel, the continued use determination unit 403 determines that the user P has performed a mode change operation on the operation member 103.

[0043] 9(a), list 903 includes three modes: a mode for taking pictures (shooting mode), a mode for displaying information related to files stored in memory unit 213 on the display panel (viewing mode), and a mode for setting camera 100 (setting mode). Therefore, when user P touches one of these three modes that is different from the mode currently set in camera 100, continued use determination unit 403 determines that a mode change operation has been performed by user P on operating member 103. When the mode currently set in camera 100 is the shooting mode, the mode change operation by the user is an operation for changing the operating state to a state other than shooting, and therefore this is considered to be an operation by the user to end shooting.

[0044] If the result of such determination is that the user P has performed an operation to change the mode on at least one of the operation members 103 to 105, the process proceeds to step S605. On the other hand, if the user P has not performed an operation to change the mode on at least one of the operation members 103 to 105, the process proceeds to step S604.

[0045] In step S604, the continuous use determination unit 403 determines that the user P continues to use the camera 100. On the other hand, in step S605, the continuous use determination unit 403 determines that the user P does not continue to use the camera 100.

[0046] In addition to the above, there are various other operations that can be performed by user P to indicate the end of use of camera 100, such as sending an end signal from a remote control or a device in wireless communication. These methods may also be combined, or only some of them may be performed.

[0047] In this way, by determining whether the user has performed an operation to end photography and defining the range of continuous use as from the identification of the user until the user performs an operation to end photography, the convenience of the device that stores user information together with the photographed image is improved.

[0048] Note that the method for identifying the user of camera 100 is not limited to iris authentication. For example, camera 100 may identify the user of camera 100 using other biometric information such as a fingerprint, vein pattern, or face, or may identify the user of camera 100 by requiring the user to input authentication information such as a personal identification number or password. Alternatively, camera 100 may identify the user of camera 100 by reading authentication information such as the user's personal identification number or password from a recording medium such as an ID card on which the information is recorded, and performing authentication. An authentication dongle for performing such authentication may be connected to camera 100. Alternatively, the authentication may be performed by wirelessly communicating with an authentication device such as a nearby smartphone or smartwatch, or by combining these methods to perform two-step authentication.

[0049] In this way, the method for identifying the user of camera 100 (authentication method) is not limited to a specific method, and the above authentication may be performed by camera 100 alone, or by communicating with another device.

[0050] Furthermore, in this embodiment, the memory unit 213 is a memory unit built into the camera housing 100B, but it may also be a "memory unit that is physically detachably connected to the camera housing 100B," such as an SD card or USB memory.

[0051] <Modification> Instead of the user pressing the end button to decide to end use of camera 100, the "end of use of camera 100" may be decided when the user releases the continue use button that the user has been continuing to press. In this case, in step S601, it is determined whether the user has released the continue use button that the user has been continuing to press. If the user has released the continue use button that the user has been continuing to press, the process proceeds to step S605, and if the user has not released the continue use button that the user has been continuing to press, the process proceeds to step S602.

[0052] The continue use button can be provided as a touch panel or operating lever applicable to the operating members 103 to 105, and convenience can be improved by, for example, placing the button on the grip of the camera 100 so that the user can easily continue to press it while holding it.

[0053] That is, in this embodiment, if an instruction to continue using the camera 100 (an instruction to continue using the camera) is being input (inputting state) in step S601, the process proceeds to step S602. On the other hand, if the instruction to continue using the camera is no longer being input in the inputting state in step S601, the process proceeds to step S605. Such an instruction to continue using the camera can be input in various ways, as described above.

[0054] [Second embodiment] In the following embodiments, including this embodiment, differences from the first embodiment will be described, and unless otherwise specified below, it will be assumed that they are the same as the first embodiment. In this embodiment, a method for determining the continuous use range based on an action that is not an explicit operation by the user but that presumes the end of shooting will be described. Note that this method may be used in combination with the method for determining the continuous use range in the first embodiment.

[0055] Details of the processing of step S503 according to this embodiment will be described with reference to the flowchart in Fig. 7. In Fig. 7, processing steps that are the same as those shown in Fig. 6 are assigned the same step numbers as those steps, and descriptions of those processing steps will be omitted.

[0056] In step S701, continued use determination unit 403 acquires "information indicating whether photographing lens unit 100A is mounted (connected) to camera housing 100B" from mount contact 209. Continued use determination unit 403 then determines whether the acquired information indicates "photographing lens unit 100A is mounted to camera housing 100B" or "photographing lens unit 100A is not mounted to camera housing 100B."

[0057] If the result of this determination indicates that the acquired information indicates that "photographing lens unit 100A is mounted on camera housing 100B (photographing lens unit 100A has not been removed from camera housing 100B)", the process proceeds to step S702.

[0058] On the other hand, if the acquired information indicates that "photographing lens unit 100A is not mounted on camera housing 100B (photographing lens unit 100A is detached from camera housing 100B)", the process proceeds to step S605.

[0059] In step S702, the continued use determination unit 403 acquires "information indicating the input brightness to the image sensor 211" from the mount contact 209. The continued use determination unit 403 then determines whether the "input brightness to the image sensor 211" indicated by the acquired information is equal to or less than a threshold, for example, whether the input is completely dark due to a lens cap being attached to the photographing lens unit 100A.

[0060] If the result of this determination is that the "input brightness to the image sensor 211" indicated by the acquired information is less than or equal to the threshold value, processing proceeds to step S605; if the "input brightness to the image sensor 211" indicated by the acquired information is greater than the threshold value, processing proceeds to step S703.

[0061] In step S703, the continued use determination unit 403 acquires "information indicating whether or not an external device is connected to the camera 100" from the communication module of the camera 100. Then, the continued use determination unit 403 determines whether or not the acquired information indicates "an external device is connected to the camera 100."

[0062] If the result of this determination is that the acquired information indicates that "an external device is not connected to the camera 100" (if communication between the camera 100 and the external device has been disconnected), the process proceeds to step S605. On the other hand, if the acquired information indicates that "an external device is connected to the camera 100" (if communication between the camera 100 and the external device has not been disconnected), the process proceeds to step S604.

[0063] In this embodiment, three operations have been described as examples of operations for estimating whether or not image capture by camera 100 has ended: "determine whether photographing lens unit 100A is mounted on camera housing 100B," "determine whether the input brightness to image sensor 211 is equal to or lower than a threshold," and "determine whether communication between camera 100 and an external device has been disconnected." However, the operations for estimating whether or not image capture by camera 100 has ended are not limited to these three operations. For example, if it is detected that an accessory other than a lens, such as a flash or a release cable, has been removed from camera housing 100B, it may be determined that image capture by camera 100 has ended. Furthermore, if it is detected that the battery cover has been opened or the SD card has been removed, it may be determined that image capture by camera 100 has ended. Furthermore, these methods may be combined, or only some of them may be performed.

[0064] In this way, the range of continued use is determined by an action that presumes the end of photography, even without an explicit operation by the user, thereby improving the convenience of the device that authenticates and records the photographer.

[0065] [Third embodiment] In this embodiment, a method for determining the continuous use range while the user is performing an operation for shooting will be described. Details of the processing of step S503 according to this embodiment will be described with reference to the flowchart in Fig. 8. In Fig. 8, processing steps that are the same as those shown in Fig. 6 are assigned the same step numbers as those processing steps, and descriptions of those processing steps will be omitted.

[0066] In step S801, the continuous use determination unit 403 determines whether continuous shooting is being performed by long pressing of the release button 101. If the result of this determination is that continuous shooting is being performed by long pressing of the release button 101, the process proceeds to step S802, and if continuous shooting is not being performed by long pressing of the release button 101, the process proceeds to step S803.

[0067] In step S802, the continuous use determination unit 403 determines whether the number of images captured after iris authentication is equal to or greater than a predetermined maximum number of images N. The continuous use determination unit 403 counts up the number of images captured each time an image is captured after iris authentication.

[0068] If the result of this determination is that the number of images taken after iris authentication is equal to or greater than the predetermined maximum number of images N, processing proceeds to step S605; if the number of images taken after iris authentication is less than the predetermined maximum number of images N, processing proceeds to step S803.

[0069] In step S803, the continuous use determination unit 403 determines whether or not the user is gripping the grip portion provided on the camera housing 100B, using a contact sensor provided on the grip portion.

[0070] If the result of this determination is that the user is gripping the grip portion, the process proceeds to step S604, and if the user is not gripping the grip portion, the process proceeds to step S804.

[0071] In step S804, continued use determination unit 403 determines whether the direction (posture) in which camera 100 is facing is changing, using a gyro sensor provided in camera housing 100B. For example, continued use determination unit 403 determines that the direction (posture) in which camera 100 is facing is changing if the difference between the direction measured previously by the gyro sensor and the direction measured this time by the gyro sensor is equal to or greater than a threshold.

[0072] If the result of this determination is that the direction (posture) of the camera 100 has changed, the process proceeds to step S604, and if the direction (posture) of the camera 100 has not changed, the process proceeds to step S805.

[0073] Here, the bottom of the camera housing 100B according to this embodiment is provided with a fixing screw hole and a screw detection sensor for detecting whether or not the fixing screw hole is screwed in. When the camera housing 100B is to be fixed to a device such as a tripod or a pan head, the camera housing 100B can be fixed to the device by attaching the camera housing 100B to the device and then screwing the fixing screw hole.

[0074] In step S805, the continuous use determination unit 403 determines whether or not a screw is fastened in the fixing screw hole using the screw fastening detection sensor. If the result of this determination is that a screw is fastened in the fixing screw hole, the process proceeds to step S604, and if the screw is not fastened in the fixing screw hole, the process proceeds to step S605.

[0075] As described above, according to this embodiment, the range in which the user is performing operations for photographing is determined as the continuous use range, thereby improving the convenience of the device for authenticating and recording the photographer.

[0076] [Fourth embodiment] It can be assumed that camera 100 is being used continuously while camera 100 itself is moving, such as when the user carries camera 100 around or turns camera 100 toward a subject. To this end, camera 100 is equipped with an inertial sensor, gyro, or the like to detect camera movement, and continuous use determination unit 403 determines the period during which movement of camera 100 is detected as the continuous use range.

[0077] Furthermore, the position of the camera 100 may be used to determine that the camera 100 is in continuous use while it is within a specific geographical range. The camera 100 may be equipped with a position measuring device such as a GPS, and the continuous use determination unit 403 may determine that the continuous use range is the period when the position measured by the position measuring device is within a specific range.

[0078] Alternatively, a clock may be installed in camera 100, and a specific time period may be set as the continuous use range. Furthermore, it may be determined that continuous use is occurring as a series of images while similar images are being captured. Continuous use determination unit 403 evaluates the similarity of consecutively captured images based on differential images and statistics, and determines the period during which images with a similarity below a threshold are captured as the continuous use range.

[0079] The above-mentioned methods are merely examples of methods for determining the continuous use range, and are not intended to be limiting. In this way, by determining the continuous use range using various information, the convenience of the device for authenticating and recording the photographer is improved.

[0080] [Fifth embodiment] In this embodiment, a screen displayed in relation to the processing in the above embodiment will be described. As an example, in the flowchart of FIG. 5, the period during which the loop in steps S501 and S502 is repeated without identifying the user will be referred to as a "user unidentified state." Furthermore, the period during which the loop in steps S503 and S504 continues to determine that use is continuing will be referred to as a "state of continued use by user P" (user P is the user identified in step S501). Note that display-related processing, i.e., display control such as screen display, screen switching and display changes in response to user operations, etc., is performed by CPU 212.

[0081] Fig. 9(a) is a diagram showing an example of a screen displayed on the display panel serving as the operation member 103 in the "state of continued use by user P." The screen in Fig. 9(a) includes a display 901 including the subject 900, the user of the camera 100 (user P), and the time elapsed since iris authentication, a list 903 for mode selection, and an end button 902. The screen also includes a display 904 showing the number of shots / maximum number of shots N (here, N=100) after iris authentication.

[0082] 9(b) is a diagram showing an example of a screen displayed on the display panel serving as the operation member 103 in the "unidentified user state." Display 901 indicates that the user has not been identified by iris authentication.

[0083] The display 904 allows the user to grasp the number of remaining shots. For example, the camera 100 may change the color or size of the display 904 or make a sound when the number of shots approaches N.

[0084] Furthermore, if the continued use determination unit 403 determines that the user P of the camera 100 has not continued to use the camera 100, the screen shown in FIG. 9C may be displayed on the display panel serving as the operation member 103. The display 901 includes a message indicating that the user P of the camera 100 has ended continued use of the camera 100 and the reason for the end. The display 901 may be displayed for a specified period of time (for example, five seconds). In the example of FIG. 9C, since it was determined in step S701 that the photographing lens unit 100A has been detached from the camera housing 100B, the display 901 indicates this as the reason for the end.

[0085] This allows the user to understand the cause even if an unexpected termination occurs. To make it easier for the user to notice, the entire screen may be highlighted by changing the color, or a sound may be played.

[0086] These displays may be displayed on the display device 214 instead of on the display panel of the operation member 103, and may be superimposed on the viewfinder, or may be displayed on both.

[0087] Another display method may be to provide an LED lamp on the camera 100, which lights up during "continuous use state," turns off during "unidentified user state," and flashes when continued use ends. In this case, similar information can be displayed by changing the color of the LED lamp for each user. To prevent the light from the LED lamp from affecting the photograph, it may be controlled to turn it off while the shutter is being taken.

[0088] In addition, the status of the continuous use range can be notified to the user by a shutter sound. When the release button 101 is pressed, a different shutter sound is made for each user P in the "continuous use state by user P," and a different shutter sound is made in the "unidentified user state." This allows the user to know whether they are within the continuous use range, which user has been authenticated, and who has been recorded as the photographer in the photo. Instead of the shutter sound after the shutter is released, a sound indicating the status can be made to sound when the shutter is pressed halfway.

[0089] Here, when the user touches the setting mode in list 903, the screen shown in Fig. 10 is displayed on the display panel serving as operation member 103. List 1001 displays a list of configurable items, and when the user touches "authentication continuation method" from list 1001, list 1002 is displayed.

[0090] List 1002 is used to individually enable (ON) or disable (OFF) the various methods described above for determining whether the user of camera 100 is continuing to use camera 100, and to set whether to display or hide end button 902. List 1002 also includes a numeric box 1003. The user can input the maximum number of images N to be taken into numeric box 1003. In this way, by displaying the status of camera 100 to users and allowing settings to be made for each user, the convenience of the device for authenticating and recording photographers is further improved.

[0091] [Sixth embodiment] In this embodiment, a method for determining whether or not the camera is being used continuously is described, based on an action that is not an explicit operation by the user but is inferred to indicate the end of image capture. This method may be used in combination with the method for determining whether or not the camera is being used continuously in the first embodiment.

[0092] In this embodiment, in addition to operations using the operating members 103 to 105, the continued use determination unit 403 also acquires detection information from the eye proximity detection unit 300, which is provided near the viewfinder opening as shown in Fig. 11. The acquired detection information from the eye proximity detection unit 300 is sent to the CPU 212 as shown in the block diagram of Fig. 12.

[0093] In this embodiment, the procedure by which the continuous use determination unit 403 determines whether the user P is continuing to use the device will be described with reference to the flowchart of FIG.

[0094] First, in step S501, the continuous use determination unit 403 detects, via the eyepiece detection unit 300, whether the user's eye is placed in contact with the eyepiece lens 102.

[0095] If the eye is placed near the camera, it is determined that use of the camera has started, and the process proceeds to step S502, whereas if the eye is not placed near the camera, the process returns to step S501. As an example, the eye placement detection unit 300 uses a proximity sensor.

[0096] As shown in Figure 11, a proximity sensor is provided as an eyepiece detector 300 in part of the viewfinder's eyepiece, and when a user looks into the viewfinder, their face approaches the proximity sensor. The sensor detects the approach of the face and determines that the user is looking into the viewfinder. The proximity sensor, for example, is configured using a light-emitting element such as an LED and a reflected light intensity detector (such as a CCD) that detects the intensity of light reflected back from the approaching face.

[0097] In one detection operation, the proximity sensor emits light from the light-emitting unit, and when the light is reflected by an object such as a face positioned in front of the sensor and then enters the reflected light intensity detection unit, the intensity of the reflected light is evaluated and it is determined that an object is in proximity if the intensity is higher than a predetermined value. However, the proximity sensor method is not limited to this, and any method such as a laser distance measurement method or a contact detection method can also be applied as long as it is configured to detect that the user's face is in contact with the viewfinder.

[0098] The proximity sensor performs the eye proximity detection operation at predetermined intervals, detecting whether a face is approaching the viewfinder multiple times per second. For example, the detection frequency is 60 times per second. When the proximity sensor detects at this frequency, it can detect the removal of a face from the camera when a user attempts to switch from one user to another as the end of the proximity state. This is because it is virtually impossible for a user to remove their face from the viewfinder and have another user place their eye on the viewfinder in the 1 / 60th of a second between the first detection and the next detection.

[0099] Therefore, in this embodiment, as long as it is detected that the user continues to keep their eye on the viewfinder, it is assumed that the same user is continuing to take pictures, and this is used to determine whether the camera is being used continuously by the same user.

[0100] Next, in step S502, the user identification unit 402 identifies the current user by performing person identification based on image information of the eyeball 220 looking through the eyepiece 102. Person identification based on the image information of the eyeball may be iris authentication using a so-called iris code, or the eyeball image may be input into a deep learning model to perform person identification using AI. Any person identification method based on eyeball image information is applicable.

[0101] Next, in step S503, user identification unit 402 determines whether or not the user was successfully identified in step S502, and if successful, the process proceeds to step S521. The user identified at this time will hereafter be referred to as user P. If authentication fails, for example, because it is determined that the user is different from any of the registered persons, the process proceeds to step S511.

[0102] In step S511, it is determined whether or not a photograph has been taken by the photographing unit 401, for example by pressing the release button 101. If a photograph has been taken, the process proceeds to step S512. If a photograph has not been taken, the process proceeds to step S513, where it is determined whether or not the subject has taken their eyes away from the viewfinder, as will be described later.

[0103] In step S512, which is the step to proceed to if imaging was performed in the previous step, the recording unit 404 records the captured video in the memory unit 213. At this time, metadata of photographer information such as "photographer is unidentified" is added to the electronic data of the video to be recorded, and then the data is recorded in the memory unit 213. Then, the process proceeds to step S513.

[0104] In step S513, it is determined whether the user has moved their eyes away from the viewfinder. This is done using the detection results of the proximity sensor described above. If the proximity sensor detects that the face has moved away from the proximity sensor, it is determined that the user has moved their eyes away from the viewfinder. If it is determined that the user has moved their eyes away, it is determined that the camera is no longer in use, and the process returns to the start of the flow, step S501, where it determines whether the user is still looking into the viewfinder. If the user has not moved their eyes away and is still looking into the viewfinder, it is determined that the camera is still in use, and the process returns to step S502, where it identifies the user.

[0105] In step S521, which is the step to be taken if it is determined in step S503 that the user has been successfully identified, it is determined whether or not a photograph has been taken by the photographing unit 401, for example by pressing the release button 101. If a photograph has been taken, the process proceeds to step S522. If a photograph has not been taken, the process proceeds to step S523, where it is determined whether or not the user has taken their eyes away from the viewfinder, as will be described later.

[0106] In step S522, the recording unit 404 records the captured video in the memory unit 213. At this time, metadata indicating "photographer is P" is added to the electronic data of the video to be recorded, and then the data is recorded in the memory unit 213. Then, the process proceeds to step S523.

[0107] In step S523, it is determined whether the user has moved their eyes away from the viewfinder. This is done using the detection result of the proximity sensor described above. If the proximity sensor detects that the face has moved away from the proximity sensor, it is determined that the user has moved their eyes away from the viewfinder. If it is determined that the user has moved their eyes away, it is determined that the camera is no longer in use, and the process returns to the start of the flow, step S501, where it determines whether the user is still looking into the viewfinder. If the user has not moved their eyes away and is still looking into the viewfinder, it is determined that the camera is still in use, and the process returns to step S521, where it determines whether the release button 101 has been pressed and an image has been taken by the photographing unit 401.

[0108] The gist of this embodiment is that in step S523 above, if the user has not taken their eyes off the camera and the continuous use state is maintained, it is possible to assign metadata indicating that "the photographer is person P" to the electronic data of the video without returning to the user identification step of step S502. In other words, since it is not possible to change users while the eye contact state is maintained, once person authentication is successful, it is permitted to assign metadata indicating that person P, who was the person at the time of successful authentication, is the photographer without identifying the user until the eye contact state is released.

[0109] From the above, it has been shown that in a camera with a viewfinder, the user's continued use status can be determined using the detection results of the eyepiece detection unit 300, which is an example of a proximity sensor, to determine whether the user is looking through the viewfinder, thereby improving the convenience of the device for authenticating and recording the photographer.

[0110] The method for recording photographer information is not limited to adding metadata. For example, it may be stored in the form of a table that associates hash values ​​of video data with photographer information, or it may be connected to an external server via wireless communication and uploaded as a record that combines video data and photographer information. Furthermore, methods for preventing tampering and guaranteeing authenticity, such as encryption or digital signatures, may also be used with this information. Instead of adding information that "the photographer is unidentified," it may simply not be added.

[0111] [Seventh embodiment] In the sixth embodiment, after successful user identification while the device is determined to be in a continuous use state, information about the first successfully identified user is added to images captured thereafter as long as the device remains in the continuous use state. However, even for images captured during the continuous use state before successful user identification and saved as an undetermined photographer, the same photographer can be considered to have been captured during the same period of continuous use, and user information may be added retroactively.

[0112] In this embodiment, the configuration and operation for realizing the above will be described with reference to the flowchart in FIG.

[0113] First, in step S501, the continued use determination unit 403 detects whether the user's eye is placed in contact with the eyepiece 102 using the eyepiece detection unit 300. If the eye is placed in contact with the eyepiece, it is determined that use of the camera has begun, and the process proceeds to step S502; if the eye is not placed in contact with the eyepiece, the process returns to step S501. The eyepiece detection unit 300 has the same configuration as described in the second embodiment, and as an example, a proximity sensor is used.

[0114] In this embodiment, as in the sixth embodiment, the state of continued use by the same user is determined by taking advantage of the fact that as long as it is detected that the user continues to keep their eye on the viewfinder, it is assumed that the same user is continuing to take pictures.

[0115] Next, in step S502, since it was determined in the previous step that the eye had been placed on the viewfinder, it is assumed that use of the camera has begun, and the camera's use state is set to "continue use."

[0116] Next, in step S503, the user identification unit 402 performs person identification based on image information of the eyeball 220 looking through the eyepiece 102, and identifies the current user.

[0117] Next, in step S504, user identification unit 402 determines whether or not the user was successfully identified in step S503, and if successful, the process proceeds to step S521. The user identified at this time will hereafter be referred to as user P. If authentication fails, for example, because it is determined that the user is different from any of the registered persons, the process proceeds to step S511.

[0118] In step S511, it is determined whether or not a photograph has been taken by the photographing unit 401, for example by pressing the release button 101. If a photograph has been taken, the process proceeds to step S512. If a photograph has not been taken, the process proceeds to step S513, where it is determined whether or not the subject has taken their eyes away from the viewfinder, as will be described later.

[0119] In step S512, which is the step to proceed to if imaging was performed in the previous step, the recording unit 404 records the captured video in the memory unit 213. At this time, metadata about the photographer, stating "photographer is unidentified," is added to the electronic data of the video to be recorded, and then the data is recorded in the memory unit 213. Then, the process proceeds to step S513.

[0120] In step S513, it is determined whether the user has moved their eyes away from the viewfinder. This is performed using the detection results of the proximity sensor, as in the sixth embodiment. If the proximity sensor detects that the face has moved away from the proximity sensor, it is determined that the user has moved their eyes away from the viewfinder. If it is determined that the user has moved their eyes away, the process proceeds to step S514. If the user has not moved their eyes away and is still looking into the viewfinder, it is determined that the user is still using the camera, and the process returns to identifying the user in step S503.

[0121] In step S514, which is reached if it is determined in the previous step that the camera is in an eye-away state, it is determined that the camera is no longer in use, and the "continuous use" state is cancelled. The process then returns to the start of the flow, step S501, where the eye-closed state is determined.

[0122] In step S521, which is performed if it is determined in step S504 that the user has been successfully identified, a determination is made as to whether there are any images among the images already recorded in memory unit 213 that were taken during the current "continued use" state and for which the "photographer is unidentified." The image or images taken during the current "continued use" state are those that were taken after the viewfinder was placed in position and before the eyes left the viewfinder. If there are any images that satisfy the above conditions, processing proceeds to step S531; if there are no images, processing proceeds to step S522.

[0123] In step S531, the electronic data of the images that were taken during the current "continued use" state and for which the "photographer is unidentified" was determined in the previous step is changed. Here, the metadata of the electronic data of the images, "photographer is unidentified," is changed to metadata that "photographer is P," and then the images are re-recorded in the memory unit 213.

[0124] As mentioned above, it is believed that the same person is using the camera during the "continuous use" state from when the user places their eye on the viewfinder until they take it away. Therefore, once a user has been successfully identified, images taken before that point can be determined to have been taken by the same person as long as they are taken during the "continuous use" state from when the user places their eye on the viewfinder until they take it away. This is the gist of this embodiment.

[0125] From the above, if there is an image taken between looking through the viewfinder and successfully identifying the user, it can be said that the metadata for that image, "Photographer unidentified," can be changed to "Photographer is P."

[0126] If there is no image that meets the conditions in step S521, the process proceeds to step S522, where it is determined whether or not a photograph has been taken by the photographing unit 401, for example by pressing the release button 101. If a photograph has been taken, the process proceeds to step S523. If a photograph has not been taken, the process proceeds to step S524, where it is determined whether or not the subject has taken their eyes away from the viewfinder, as will be described later.

[0127] In step S523, recording unit 404 records the captured video in memory unit 213. At this time, metadata indicating "photographer is P" is added to the electronic data of the video to be recorded, and then the data is recorded in memory unit 213. Then, the process proceeds to step S524.

[0128] In step S524, it is determined whether the user has moved their eyes away from the viewfinder. This is done using the detection result of the proximity sensor described above. If the proximity sensor detects that the face has moved away from the proximity sensor, it is determined that the user has moved their eyes away from the viewfinder. If it is determined that the user has moved their eyes away, it is determined that the continued use state of the camera has been canceled, and the process proceeds to canceling the "continued use" state in step S525. If the user has not moved their eyes away and is still looking into the viewfinder, it is determined that the user is in the continued use state, and the process returns to step S522, where it is determined whether a photograph has been taken by the photographing unit 401, for example by pressing the release button 101.

[0129] In step S525, which is reached if it is determined in the previous step that the eyes are away from the viewfinder, the camera is deemed to have finished being used and the camera's "continue use" state is cancelled, and the process then returns to the start of the flow, step S501, where the eye-closed state is determined.

[0130] From the above, it has been shown that the convenience of the device for authenticating and recording photographers can be improved by retroactively adding user information to images that were taken before user identification was successful and saved with the photographer unknown, as long as they were taken within the same period of continuous use.

[0131] [Eighth embodiment] In the sixth embodiment, if a person is identified once using camera 100, from the time when eye contact is detected until the time when eye contact is removed, the information "photographed by person P" may be added to the image without further identification for images taken at that time and thereafter.

[0132] However, although camera 100 can detect when an object approaches or moves away, it does not detect that the object is an eyeball. Therefore, if you place an object in front of camera 100 after successfully identifying a person by placing your eye on it, and then take your eye off the object while it remains in the vicinity, and another person takes over and looks through the viewfinder, any images captured thereafter will be given the false information that "the image was taken by person P." In other words, there is room for fraudulent activity, such as impersonation.

[0133] Therefore, in order to prevent the above-mentioned fraudulent acts, in this embodiment, the eyeball image captured by the eyeball image sensor 219 is analyzed, and if the eyeball cannot be recognized for a predetermined period of time or more, the person determination is performed again.

[0134] The method of determining whether the eyeball can be recognized on the image from the eyeball image captured by the eyeball image pickup device 219 will be described below.

[0135] FIG. 15 is a diagram illustrating the principle of the eyeball image analysis method, and corresponds to a summary diagram of the optical system.

[0136] 15, reference numerals 216a and 216b denote light sources such as light-emitting diodes that emit infrared light that is insensitive to the observer, and the light sources are arranged approximately symmetrically with respect to the optical axis of a light-receiving lens 218, and illuminate the observer's eyeball 14. A portion of the illumination light reflected by the eyeball 14 is collected by the light-receiving lens 218 onto an eyeball image-capturing element 219.

[0137] 16A is a schematic diagram of an eyeball image projected onto the eyeball image pickup device 219, and FIG. 16B is a diagram of the output intensity of the CCD in the eyeball image pickup device 219. In FIG.

[0138] Figure 17 shows a schematic flow chart of eye image analysis.

[0139] The eyeball image analysis method will be described below with reference to FIGS.

[0140] <Explanation of analysis operation> 17, when the analysis routine starts, in step S001, the light sources 216a and 216b emit infrared light toward the observer's eyeball 14. An image of the observer's eyeball illuminated by the infrared light is formed on the eyeball image sensor 219 through the light-receiving lens 218, and is subjected to photoelectric conversion by the eyeball image sensor 219, so that the eyeball image can be processed as an electrical signal.

[0141] In step S 002 , the eyeball image signal obtained from the eyeball image pickup device 219 as described above is sent to the CPU 212 .

[0142] In step S003, coordinates of points corresponding to the corneal reflection images Pd and Pe of the light sources 216a and 216b and the pupil center c shown in FIG. 15 are calculated from the information of the eyeball image signal obtained in step S002.

[0143] Infrared light emitted from light sources 216a and 216b illuminates the cornea 142 of the observer's eyeball 14, and corneal reflection images Pd and Pe formed by part of the infrared light reflected from the surface of the cornea 142 are collected by a light receiving lens 218 and formed on an eyeball image sensor 219 (points Pd' and Pe' in the figure). Similarly, light beams from ends a and b of the pupil 141 also form images on the eyeball image sensor 219.

[0144] 16(a) shows an example of a reflected image obtained from the eyeball image pickup device 219, and FIG. 16(b) shows an example of brightness information obtained from the eyeball image pickup device 219 in region α of the example image.

[0145] As shown in the figure, the horizontal direction is the X-axis and the vertical direction is the Y-axis. Here, the coordinates of images Pd' and Pe' formed by the corneal reflections of light sources 216a and 216b in the X-axis direction (horizontal direction) are designated Xd and Xe. Furthermore, the coordinates of images a' and b' formed by the light beams from ends a and b of pupil 14b in the X-axis direction are designated Xa and Xb.

[0146] In the example of luminance information in FIG. 16(b), extremely high levels of luminance are obtained at positions Xd and Xe corresponding to images Pd' and Pe' formed by the corneal reflection images of light sources 216a and 216b.

[0147] The area between coordinates Xa and Xb, which corresponds to the area of ​​pupil 141, has an extremely low level of luminance, except for the positions Xd and Xe. In contrast, the area corresponding to the area of ​​iris 143 outside pupil 141, which has an X coordinate value lower than Xa and an X coordinate value higher than Xb, has intermediate luminance values ​​between the two types of luminance levels.

[0148] From the information on the fluctuation of the brightness level with respect to the X coordinate position, the X coordinates Xd and Xe of the images Pd' and Pe' formed by the corneal reflection images of the light sources 216a and 216b, and the X coordinates Xa and Xb of the images a' and b' at the pupil edge can be obtained.

[0149] Furthermore, when the rotation angle θx of the optical axis of the eyeball 14 relative to the optical axis of the light receiving lens 218 is small, the coordinate Xc of the point (let's call it c') corresponding to the pupil center c imaged on the eyeball imaging element 219 can be expressed as Xc ≒ (Xa + Xb) / 2.

[0150] From the above, it was possible to estimate the X coordinate of c' corresponding to the pupil center imaged on the eye image sensor 219, and the coordinates of the corneal reflection images Pd' and Pe' of the light sources 216a and 216b.

[0151] With the above configuration, it is confirmed that the pupil image and corneal reflection image P are present within the angle of view of the image based on the detected pupil edge and center positions and the position of corneal reflection image P. If these are confirmed to be present within the angle of view, it can be said that it has been confirmed that the eyeball is reflected within the angle of view.

[0152] Furthermore, in step S004, the distance L from the eyeball to the light receiving lens 218 is calculated. The distance L is a value determined by the position of the eyeball 14 relative to the light receiving lens 218, and can be obtained substantially as a function of the distance (Xd-Xe) between the corneal reflection images Pd' and Pe'.

[0153] In addition to determining whether or not the eyeballs are present within the angle of view, it is also determined whether or not the distance L from the eyeballs to the light receiving lens can be detected within a predetermined normal range.

[0154] Next, using the flowchart of Figure 18, the operation in this embodiment will be explained below, in which, in order to prevent fraudulent activities, the eyeball image captured by the eyeball imaging element 219 is analyzed as described above, and if the eyeball cannot be recognized for more than a predetermined time, the person is determined again.

[0155] First, in step S501, the continuous use determination unit 403 detects whether the user's eye is placed in contact with the eyepiece 102 using the eyepiece detection unit 300. If the eye is placed in contact with the eyepiece, it is determined that use of the camera has started, and the process proceeds to step S502; if the eye is not placed in contact with the eyepiece, the process returns to step S501.

[0156] The eyepiece detection unit 300 has the same configuration as that described in the sixth embodiment, and a proximity sensor is used as an example. A proximity sensor is provided as the eyepiece detection unit 300 in part of the viewfinder's eyepiece, and when a user looks into the viewfinder, their face approaches the proximity sensor. The sensor detects the approach of the face, and detects that the user is looking into the viewfinder.

[0157] As an example, the proximity sensor is configured using a light-emitting unit such as an LED and a reflected light intensity detection unit (such as a CCD) that detects the intensity of light reflected back from an approaching face.

[0158] In one detection operation, the proximity sensor emits light from the light-emitting unit, and when the light is reflected off an object such as a face located in front of the sensor and then enters the reflected light intensity detection unit, it evaluates the intensity of the reflected light and determines that an object is in proximity if the intensity is higher than a predetermined value.

[0159] Next, in step S502, the user identification unit 402 performs person identification based on image information of the eyeball 220 looking through the eyepiece 102, and identifies the current user.

[0160] Next, in step S503, user identification unit 402 determines whether or not the user was successfully identified in step S502, and if successful, the process proceeds to step S521. The user identified at this time will hereafter be referred to as user P. If authentication fails, for example, because it is determined that the user is different from any of the registered persons, the process proceeds to step S511.

[0161] In step S511, it is determined whether or not a photograph has been taken by the photographing unit 401, for example by pressing the release button 101. If a photograph has been taken, the process proceeds to step S512. If a photograph has not been taken, the process proceeds to step S513, where it is determined whether or not the subject has taken their eyes away from the viewfinder, as will be described later.

[0162] In step S512, which is the step to proceed to if imaging was performed in the previous step, the recording unit 404 records the captured video in the memory unit 213. At this time, metadata about the photographer, stating "photographer is unidentified," is added to the electronic data of the video to be recorded, and then the data is recorded in the memory unit 213. Then, the process proceeds to step S513.

[0163] In step S513, it is determined whether the user has moved their eyes away from the viewfinder. This is done using the detection results of the proximity sensor described above. If the proximity sensor detects that the face has moved away from the proximity sensor, it is determined that the user has moved their eyes away from the viewfinder. If it is determined that the user has moved their eyes away, it is determined that the camera is no longer in use, and the process returns to the start of the flow, step S501, where the eye-contact state is determined. If the user has not moved their eyes away and is still looking into the viewfinder, it is determined that the camera is still in use, and the process returns to step S502, where the user is identified.

[0164] In step S521, which is the step to be taken if it is determined in step S503 that the user has been successfully identified, it is determined whether or not a photograph has been taken by the photographing unit 401, for example by pressing the release button 101. If a photograph has been taken, the process proceeds to step S522. If a photograph has not been taken, the process proceeds to step S523, where it is determined whether or not the user has taken their eyes away from the viewfinder, as will be described later.

[0165] In step S522, the recording unit 404 records the captured video in the memory unit 213. At this time, metadata indicating "photographer is P" is added to the electronic data of the video to be recorded, and then the data is recorded in the memory unit 213. Then, the process proceeds to step S523.

[0166] In step S523, it is determined whether the user has moved their eyes away from the viewfinder. This is done using the detection results of the proximity sensor described above, and if the proximity sensor detects that the face has moved away from the proximity sensor, it is determined that the user has moved their eyes away from the viewfinder. If it is determined that the user has moved their eyes away, it is determined that the continued use state of the camera has been released, and the process returns to the start of the flow, step S501, where the user is still looking into the viewfinder. If the user has not moved their eyes away and is still looking into the viewfinder, the process proceeds to step S524.

[0167] In step S524, the image obtained from eyeball imaging element 219 is analyzed to determine whether the eyeball has been detected normally for a predetermined period of time or more, and if the eyeball has not been detected normally for the predetermined period of time or more, the process returns to step S502 and starts over from person detection. A determination that the eyeball has not been detected normally indicates that the presence of the pupil and P image within the angle of view cannot be confirmed as described above, or that the distance L from the detected eyeball to the light receiving lens, etc., is not within a normal range as described above.

[0168] This is the gist of this embodiment. In the previous step, the proximity sensor detection results confirmed that the eye-contact state was maintained. However, because there is a possibility of falsification, such as placing an object in front of the proximity sensor, the eye image analysis results are additionally used to determine whether the eyes are visible. The distance is also checked to see if it is appropriate. Furthermore, if the eyes are not visible continuously for a predetermined period of time or more, there is a suspicion of falsification, so person determination is performed again even if the eye-contact state continues. Similarly, if the distance is not appropriate for a predetermined period of time or more, it is deemed to be an abnormality, and person determination is performed again.

[0169] If the eye is currently captured normally in the eyeball image, or if the continuous time during which the eye is not captured normally is less than a predetermined time, the process returns to step S521, which is the step of determining whether or not an image has been captured by the image capturing unit 401.

[0170] The predetermined time is assumed to be, for example, 0.1 to 0.3 seconds, which is about the time it takes for a typical human to blink.

[0171] From the above, it has been shown that by determining the user's continued use state based on the results of the eye proximity detection unit 300, which is an example of a proximity sensor, as well as the analysis results of whether or not the eyeball is captured in the image obtained from the eyeball imaging element 219, it is possible to prevent fraud while improving the convenience of the device.

[0172] [Ninth embodiment] In the sixth embodiment, the method of determining whether the camera is in use or not was explained by detecting eye contact using a proximity sensor included in the viewfinder. However, in this embodiment, the camera 100 includes a fingerprint detection sensor, and the detection results are used to determine whether the camera is in use or not.

[0173] In this embodiment, as shown in FIG. 19, the release button 101 has a fingerprint detection sensor 201, which is used to identify the user and determine the state of continued use by detecting whether a finger is continuously in contact with the release button 101.

[0174] The position of the fingerprint detection sensor 201 is not limited to this, and as another example of placement, it may be placed on the grip part as shown by the fingerprint detection sensor 201' in Figure 19, and is applicable as long as it is placed in a position where the hand naturally touches it when taking a picture.

[0175] In the following, in this embodiment, the fingerprint detection sensor 201 is arranged on the release button 101 as described above.

[0176] As an example, the fingerprint detection sensor 201 is composed of a number of electrodes in the contact area and a protective film (sensor surface) covering the electrodes. When a finger is pressed against the sensor surface shown by the hatched area in Fig. 19(a), electric charges corresponding to the unevenness of the fingerprint generated at each electrode are read out, and a fingerprint image obtained from these electric charges is acquired as detected finger information. The acquired detected finger information is sent from the fingerprint detection sensor 201 to the CPU 212, as shown in the block diagram of Fig. 20.

[0177] The CPU 212 detects the feature points of the fingerprint image, which is the detected finger information thus acquired, calculates the degree of match between the feature point information (hereinafter referred to as the detected fingerprint) and the registered finger information (hereinafter referred to as the registered fingerprint), and determines that the detected fingerprint and the registered fingerprint match if the degree of match is equal to or greater than the authentication threshold.

[0178] The degree of match may be calculated from the number of matching feature points such as gaps and bifurcations between the detected fingerprint and the registered fingerprint, or may be calculated using a pattern matching method, a frequency analysis method, or the like.

[0179] The fingerprint detection sensor is not limited to this type, but may be a sensor that optically reads a fingerprint or a sensor that reads a fingerprint by pressing a finger against it and then sliding it.Furthermore, the sensor is not limited to detecting fingerprints, and other finger authentication methods may be used, such as detecting veins to identify fingers.

[0180] With the above-described configuration, the fingerprint detection sensor 201 first has a contact determination function that determines whether a finger is pressed against the release button 101 based on whether any fingerprint image signal is detected from the sensor. If there is no finger contact, the electrodes on the sensor surface do not generate a charge signal, so contact or non-contact can be determined based on the presence or absence of a charge signal, i.e., a fingerprint image signal. Secondly, as described above, the fingerprint detection sensor 201 has an individual determination function that analyzes the feature points of the fingerprint image and determines whether they match the feature points of a person registered in advance to a predetermined degree or higher.

[0181] Here, while the camera is being used continuously, a finger is placed on the release button 101, and in the fingerprint detection sensor 201, the contact detection operation, which is the first function described above, is performed at predetermined time intervals, and the presence or absence of contact with the release button 101 is detected multiple times per second.

[0182] As an example, detection is performed at a frequency of 60 times per second. If detection is performed at this frequency, the release of a finger from the release button 101 when one user tries to switch to another can be detected as a release of the contact state. This is because it is virtually impossible for a user to release their finger from the release button 101 and have another user touch the release button 101 in their place during the less than 1 / 60 second between one detection and the next detection.

[0183] Therefore, in this embodiment, as long as it is detected that the user is continuing to touch the release button 101, it is assumed that the same user is continuing to take pictures, and this is used to determine whether the camera is being used continuously by the same user.

[0184] In this embodiment, the procedure by which the continuous use determination unit 403 determines whether the user P is continuing to use the device will be described with reference to the flowchart in FIG.

[0185] First, in step S501, the continuous use determination unit 403 determines whether the user's finger is in contact with the release button 101 using the fingerprint detection sensor 201. If the finger is in contact, it is determined that use of the camera has started, and the process proceeds to step S502; if the finger is not in contact, the process returns to step S501.

[0186] As described above, the fingerprint detection sensor has the primary function of detecting whether or not a finger is in contact with the camera, and this detection is performed multiple times per second. For example, this is performed 60 times per second. When detection is performed at this frequency, it can detect the removal of a user's face from the camera, as described above, as a release from the proximity state when a user tries to switch from one user to another.

[0187] Therefore, in this embodiment, the state of continued use by the same user is determined by taking advantage of the fact that as long as it is detected that the user is continuing to touch the release button, it is assumed that the same user is continuing to take pictures.

[0188] Next, in step S502, the user identification unit 402 performs the above-mentioned second function, that is, person determination, from the image of the finger touching the release button 101 acquired by the fingerprint detection sensor 201, and identifies the current user.

[0189] Next, in step S503, user identification unit 402 determines whether or not the user was successfully identified in step S502, and if successful, the process proceeds to step S521. The user identified at this time will hereafter be referred to as user P. If authentication fails, for example, because it is determined that the user is different from any of the registered persons, the process proceeds to step S511.

[0190] In step S511, it is determined whether or not a photograph has been taken by the photographing unit 401, for example by pressing the release button 101. If a photograph has been taken, the process proceeds to step S512. If a photograph has not been taken, the process proceeds to step S513, where it is determined whether or not the finger has been released from the release button, as will be described later.

[0191] In step S512, which is the step to proceed to if imaging was performed in the previous step, the recording unit 404 records the captured video in the memory unit 213. At this time, metadata of photographer information such as "photographer is unidentified" is added to the electronic data of the video to be recorded, and then the data is recorded in the memory unit 213. Then, the process proceeds to step S513.

[0192] In step S513, it is determined whether the finger has been released from the release button 101. This determination is made by the contact determination function, which is the first function of the fingerprint detection sensor 201 described above.

[0193] If it is determined that the finger has been released from the release button 101, it is determined that the camera's continued use state has been released, and the process returns to the start of the flow, step S501, where the fingerprint detection sensor 201 determines whether the user's finger is in contact with the release button 101.

[0194] If the finger is not released from the release button 101 and the user continues to keep the finger in contact with the release button 101, it is determined that the button is in continuous use, and the process returns to identifying the user in step S502.

[0195] In step S521, which is the step to be taken if it is determined in step S503 that the user has been successfully identified, it is determined whether or not a photograph has been taken by the photographing unit 401, for example by pressing the release button 101. If a photograph has been taken, the process proceeds to step S522. If a photograph has not been taken, the process proceeds to step S523, where it is determined whether or not the finger has been released from the release button 101, as will be described later.

[0196] In step S522, the recording unit 404 records the captured video in the memory unit 213. At this time, metadata indicating "photographer is P" is added to the electronic data of the video to be recorded, and then the data is recorded in the memory unit 213. Then, the process proceeds to step S523.

[0197] In step S523, it is determined whether the finger has been released from the release button 101. This determination is made by the contact determination function, which is the first function of the fingerprint detection sensor 201 described above.

[0198] If the contact determination function detects that the finger has been removed from the release button 101, which includes a fingerprint detection sensor, it determines that the camera is no longer in use and returns to the start of the flow, step S501, where the eye contact state is determined.

[0199] If the user has not released the finger from the release button 101 and is still touching the release button 101, it is determined that the button is still in use, and the process returns to step S521 to determine whether or not a photograph has been taken by the photographing unit 401, for example by pressing the release button 101.

[0200] The gist of this embodiment is that in step S523, if the finger has not been released from the release button 101 and the state of continuous use is maintained, it is possible to add metadata indicating that "the photographer is P" to the electronic data of the video without returning to the step of identifying the user in step S502. In other words, since it is impossible to change users while the finger is kept in contact with the release button 101, once person authentication has been successful, it is permitted to add metadata indicating that person P, who was the person at the time of successful authentication, is the photographer without identifying the user until the contact state is released.

[0201] From the above, it has been shown that in a camera equipped with a fingerprint sensor, the user's continued use status can be determined based on the detection result of the fingerprint detection sensor 201 as to whether or not the user is touching the release button 101, thereby improving the convenience of the device for authenticating and recording the photographer.

[0202] The numerical values, processing timing, processing order, processing subject, data (information) configuration / acquisition method / sending destination / sending source / storage location, etc. used in the above embodiments and variant examples are given as examples to provide a concrete explanation, and are not intended to be limited to these examples.

[0203] Furthermore, some or all of the embodiments and modifications described above may be used in appropriate combination, and some or all of the embodiments and modifications described above may be used selectively.

[0204] (Other embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0205] The invention of this specification includes the following image capturing apparatus and image capturing apparatus control method. (Item 1) An imaging device, an identification means for identifying a user of the photographing device; a determination means for determining a continuation state of use of the photographing device by the user; a control means for controlling the recording of images captured by the photographing device in association with information about the user in a memory during the period from when the identification means identifies the user until when the determination means determines that the user has not continued to use the photographing device; and An imaging device comprising: (Item 2) 2. The photographing device according to item 1, wherein the memory is built into the photographing device or is physically detachably connected to the photographing device. (Item 3) the imaging device is connected to the server having the memory via a communication path; The photographing device described in item 1, characterized in that the control means sends a signal to the server depending on the result of the judgment so that the image and the user information are associated and recorded in the memory. (Item 4) 4. The photographing device according to any one of items 1 to 3, wherein the identification unit identifies a user of the photographing device based on biometric information of the user of the photographing device. (Item 5) 5. The photographing device according to item 4, wherein the biometric information includes at least one of iris, fingerprint, vein, and face. (Item 6) The photographing device described in any one of items 1 to 4, characterized in that the identification means identifies the user of the photographing device based on authentication information input by the user of the photographing device or read from a recording medium. (Item 7) The photographing device described in any one of items 1 to 6, characterized in that the determination means determines that the user is continuing to use the photographing device from the time the identification means identifies the user until an instruction to end the photographing device is given. (Item 8) 8. The photographing device according to item 7, wherein the instructions include an instruction to press an end button and an instruction to switch modes. (Item 9) The photographing device described in any one of items 1 to 6, characterized in that the determination means determines that the user is continuing to use the photographing device when an instruction to continue using the photographing device is being input. (Item 10) 10. The photographing device according to item 9, wherein the determination means determines that the user is no longer continuing to use the photographing device when an instruction to continue using the photographing device is no longer input in the state. (Item 11) The photographing device described in any one of items 1 to 6, characterized in that the determination means determines that the user is not continuing to use the photographing device when it detects an action that is presumed to be an end of photographing in the photographing device. (Item 12) The photographing device described in item 1 or 11, characterized in that the determination means determines that the user is not continuing to use the photographing device when it detects that a photographing lens unit is not mounted on the housing of the photographing device. (Item 13) The photographing device described in item 1 or 11, characterized in that the determination means determines that the user is not continuing to use the photographing device if the input brightness to the image sensor of the photographing device is below a threshold value. (Item 14) 12. The photographing device according to item 1 or 11, characterized in that the determination means determines that the user is no longer using the photographing device when communication between the photographing device and the external device is disconnected. (Item 15) The photographing device described in any one of items 1 to 6, characterized in that the determination means determines that the user is not continuing to use the photographing device when the number of photographs taken after the user has been identified reaches a threshold value. (Item 16) The photographing device described in any one of items 1 to 6, characterized in that the determination means determines that the user is continuing to use the photographing device when it determines that the user is holding the grip portion of the photographing device. (Item 17) The photographing device described in any one of items 1 to 6, characterized in that when the determination means determines that the posture of the photographing device is changing, it determines that the user is continuing to use the photographing device. (Item 18) The photographing device described in any one of items 1 to 6, characterized in that the determination means determines that the user is continuing to use the photographing device when it determines that an appliance is screwed to the photographing device. (Item 19) The photographing device described in any one of items 1 to 6, characterized in that the determination means determines that the user is continuing to use the photographing device when it detects movement of the photographing device. (Item 20) The photographing device described in any one of items 1 to 6, characterized in that the determination means determines that the user is continuing to use the photographing device if the location of the photographing device is within a specific geographical range. (Item 21) 21. The photographing device according to any one of items 1 to 20, characterized in that, if the identification means is unable to identify the user of the photographing device, the control means controls the image photographed by the photographing device and information indicating that the user of the photographing device has not been identified to be recorded in memory in association with each other. (Item 22) moreover, 22. The photographing device according to any one of items 1 to 21, further comprising a display control means for displaying a screen for setting a method for determining the continuation state. (Item 23) moreover, an eyeball imaging element for capturing an eyeball image; 5. The photographing device according to item 4, wherein the biometric information is information determined from an eyeball image obtained from the eyeball imaging element. (Item 24) moreover, The viewfinder and an eye-contact detection means for detecting whether or not a user has placed their eye on the finder; a means for detecting the timing at which the eye begins to come into contact with the finder and the timing at which the eye is taken away from the finder based on the result of detection by the eye-contact detection means; and Item 24. The photographing device according to item 23, wherein the determining means determines the period from the eye contact start timing to the eye separation timing as the period of continuous use of the photographing device. (Item 25) moreover, Item 25. The photographing device according to item 24, further comprising a recording means for, if the user of the photographing device is successfully identified during the period of continuous use, linking and recording information about the identified user to images taken from that point until the end of the period of continuous use. (Item 26) moreover, Item 25. The photographing device according to item 24, further comprising a recording means for recording images taken during the period from the start of the period of continuous use to the time when the user of the photographing device is successfully identified and recorded in a recording unit, if such images are present, linking the images to information about the identified user. (Item 27) moreover, a means for determining whether or not an eyeball is captured in an eyeball image obtained from the eyeball image pickup device, Item 25. The photographing device described in item 24, characterized in that if it is determined that the eyeball has not been continuously captured in the eyeball image for a predetermined period of time or more, the period of continuous use of the photographing device is terminated at that point, even if it is not detected that the eye has been taken away from the viewfinder, and the user of the photographing device is identified again. (Item 28) moreover, means for acquiring eyeball information of a user from an eyeball image obtained from the eyeball imaging element; Item 25. The photographing device described in item 24, characterized in that if an abnormality is found in the eyeball information, even if it is not detected that the user has taken their eye off the viewfinder, the period of continued use of the photographing device is terminated at that point and the user of the photographing device is identified again. (Item 29) moreover, Release button and Fingerprint detection sensor and 29. The photographing device according to item 28, wherein the fingerprint detection sensor is disposed at the position of the release button. (Item 30) moreover, means for detecting whether a user's finger has come into contact with the release button; means for detecting the timing at which the user's finger starts to touch the release button and the timing at which the user releases the finger from the release button; and 30. The photographing device according to item 29, wherein the determination means determines the period from the contact start timing to the timing when the finger is released from the release button as the period of continuous use of the photographing device. (Item 31) moreover, Item 31. The photographing device according to item 30, further comprising a recording means for, if the user of the photographing device is successfully identified during the period of continuous use, linking and recording information about the identified user to images taken from that point until the end of the period of continuous use. (Item 32) A method for controlling an imaging device, comprising: an identification step in which an identification means of the photographing device identifies a user of the photographing device; a determination step in which a determination means of the photographing device determines a continuation state of use of the photographing device by the user; a control step in which a control means of the photographing device controls the photographing device so that images photographed by the photographing device and information about the user are associated with each other and recorded in a memory during the period from when the user is identified in the identification step until when it is determined in the determination step that the user is no longer using the photographing device; A control method for an imaging device comprising:

[0206] The invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0207] 401: Photographing unit 402: User identification unit 403: Continued use determination unit 404: Recording unit

Claims

1. An imaging device, an identification means for identifying a user of the photographing device; a determination means for determining a continuation state of use of the photographing device by the user; a control means for controlling the recording of images captured by the photographing device and information about the user in a memory in association with each other during the period from when the identification means identifies the user until when the determination means determines that the user has not continued to use the photographing device; and An imaging device comprising:

2. 2. The photographing device according to claim 1, wherein the memory is built into the photographing device or is physically detachably connected to the photographing device.

3. the imaging device is connected to the server having the memory via a communication path; 2. The photographing device according to claim 1, wherein the control means transmits a signal to the server in accordance with the result of the determination so that the image and the user information are recorded in association with each other in the memory.

4. 2. The photographing device according to claim 1, wherein the identifying unit identifies the user of the photographing device based on biometric information of the user of the photographing device.

5. 5. The image capturing apparatus according to claim 4, wherein the biometric information includes at least one of an iris, a fingerprint, veins, and a face.

6. 2. The photographing device according to claim 1, wherein the identification unit identifies the user of the photographing device based on authentication information input by the user of the photographing device or read from a recording medium.

7. 2. The photographing device according to claim 1, wherein the determination means determines that the user is continuing to use the photographing device from the time the identification means identifies the user until an instruction to end the photographing device is given.

8. 8. The photographing apparatus according to claim 7, wherein the instruction includes an instruction to press an end button and an instruction to switch modes.

9. 2. The photographing device according to claim 1, wherein the determining means determines that the user is continuing to use the photographing device when an instruction to continue using the photographing device is being input.

10. 10. The photographing device according to claim 9, wherein the determining means determines that the user is no longer continuing to use the photographing device when an instruction to continue using the photographing device is no longer input in the state.

11. 2. The photographing device according to claim 1, wherein the determination means determines that the user is not continuing to use the photographing device when it detects an action that is presumed to be an end of photographing in the photographing device.

12. 2. The photographing device according to claim 1, wherein the determination means determines that the user is not continuing to use the photographing device when it detects that a photographing lens unit is not mounted on the housing of the photographing device.

13. 2. The photographing device according to claim 1, wherein the determining means determines that the user is not continuing to use the photographing device when the input brightness to the image sensor of the photographing device is equal to or lower than a threshold value.

14. 2. The photographing device according to claim 1, wherein the determining means determines that the user is no longer using the photographing device when communication between the photographing device and the external device is disconnected.

15. 2. The photographing device according to claim 1, wherein the determining means determines that the user is no longer using the photographing device when the number of photographs taken after the user is identified reaches a threshold value.

16. 2. The photographing device according to claim 1, wherein the determining means determines that the user is continuing to use the photographing device when it determines that the user is gripping a grip portion of the photographing device.

17. 2. The photographing device according to claim 1, wherein the determining means determines that the user is continuing to use the photographing device when it determines that the attitude of the photographing device is changing.

18. 2. The photographing device according to claim 1, wherein the determining means determines that the user is continuing to use the photographing device when it determines that a device is screwed to the photographing device.

19. 2. The photographing device according to claim 1, wherein the determining means determines that the user is continuing to use the photographing device when the determining means detects a movement of the photographing device.

20. 2. The photographing device according to claim 1, wherein the determining means determines that the user is continuing to use the photographing device when the position of the photographing device is within a specific geographical range.

21. The photographing device according to claim 1, characterized in that, if the identification means is unable to identify the user of the photographing device, the control means controls the image photographed by the photographing device and information indicating that the user of the photographing device has not been identified to be recorded in memory in association with each other.

22. moreover, 22. The photographing apparatus according to claim 1, further comprising a display control means for displaying a screen for setting a method for determining the continuation state.

23. moreover, an eyeball imaging element for capturing an eyeball image; 5. The photographing apparatus according to claim 4, wherein the biological information is determined from an eyeball image obtained from the eyeball image pickup device.

24. moreover, The viewfinder and an eye-contact detection means for detecting whether or not a user has placed their eye on the finder; a means for detecting the timing at which the eye begins to come into contact with the finder and the timing at which the eye is taken away from the finder based on the result of detection by the eye-contact detection means; and 24. The photographing device according to claim 23, wherein the determining means determines a period from the eye contact start timing to the eye separation timing as a period of continuous use of the photographing device.

25. moreover, The photographing device according to claim 24, further comprising a recording means for, if the user of the photographing device is successfully identified during the period of continuous use, linking information about the identified user to images taken from that point until the end of the period of continuous use and recording the images.

26. moreover, 25. The photographing device according to claim 24, further comprising a recording means for recording, if the user of the photographing device is successfully identified during the period of continuous use, images captured during the period from the start of the period of continuous use to the time when the user was successfully identified and recorded in a recording unit, linking those images to information about the identified user.

27. moreover, a means for determining whether or not an eyeball is captured in an eyeball image obtained from the eyeball image pickup device, The photographing device according to claim 24, characterized in that if it is determined that the eyeball has not been continuously captured in the eyeball image for a predetermined period of time or more, the period of continuous use of the photographing device is terminated at that point, even if it is not detected that the eye has been taken away from the viewfinder, and the user of the photographing device is identified again.

28. moreover, means for acquiring eyeball information of a user from an eyeball image obtained from the eyeball imaging element; The photographing device according to claim 24, characterized in that if an abnormality is found in the eyeball information, even if it is not detected that the user has taken their eye away from the viewfinder, the period of continued use of the photographing device is terminated at that point and the user of the photographing device is identified again.

29. moreover, Release button and Fingerprint detection sensor and 29. The photographing device according to claim 28, wherein the fingerprint detection sensor is disposed at the position of the release button.

30. moreover, means for detecting whether a user's finger has come into contact with the release button; means for detecting the timing at which the user's finger starts to touch the release button and the timing at which the user releases the finger from the release button; and 30. The photographing device according to claim 29, wherein the determining means determines the period from the contact start timing to the timing when the finger is released from the release button as the period of continuous use of the photographing device.

31. moreover, The photographing device according to claim 30, further comprising a recording means for, if the user of the photographing device is successfully identified during the period of continuous use, linking information about the identified user to images taken from that point until the end of the period of continuous use and recording the images.

32. A control method for an imaging device, comprising: an identification step in which an identification means of the photographing device identifies a user of the photographing device; a determination step in which a determination means of the photographing device determines a continuation state of use of the photographing device by the user; a control step in which a control means of the photographing device controls the photographing device so that images photographed by the photographing device and information about the user are associated with each other and recorded in a memory during the period from when the user is identified in the identification step until when it is determined in the determination step that the user is no longer using the photographing device; A control method for an imaging device comprising:

Images