Photographing apparatus, information processing apparatus, and method for controlling photographing apparatus

Abstract

To provide a technique to improve convenience of an apparatus that authenticates and records a photographer.SOLUTION: A photographing apparatus specifies a user of the photographing apparatus and determines the state of continuation of the use of the photographing apparatus by the user. The photographing apparatus performs control so that an image photographed by the photographing apparatus in a period from when the apparatus specifies the user until when the apparatus determines that the use of the apparatus by the user is not continued, and information on the user are recorded in a memory in association with each other.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a technique for authenticating and recording a photographer. [Background technology]

[0002] In recent years, there has been a demand for a method for identifying and recording the photographer of an image or video in order to manage copyrights and guarantee authenticity. Patent Document 1 proposes a method for monitoring the biometric information of the photographer when identifying and recording the photographer in an image file, and if there is a change, it is determined that the photographer has been replaced and the authentication of the photographer is cancelled. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2018-191194 Summary of the Invention [Problem to be solved by the invention]

[0004] In the method of Patent Document 1, authentication may be cancelled between successive shots, reducing the convenience of photography. The present invention provides a technology for improving the convenience of a device that authenticates and records photographers. [Means for solving the problem]

[0005] One aspect of the present invention is an imaging device comprising: an identification means for identifying a user of the imaging device; a determination means for determining whether the user is continuing to use the imaging device; and a control means for controlling the imaging device so that images captured by the imaging device and information about the user are associated and recorded in memory between the time when the identification means identifies the user and the time when the determination means determines that the user is no longer using the device. [Effects of the Invention]

[0006] According to the present invention, it is possible to provide a technique for improving the convenience of a device that authenticates a photographer and records the image. [Brief explanation of the drawings]

[0007] [Figure 1] 1A is a front perspective view of the camera 100, FIG. 1B is a rear perspective view of the camera 100, and FIG. 1C is a diagram illustrating an operation lever. [Figure 2] FIG. 2 is a cross-sectional view of the camera 100 cut along the YZ plane. [Figure 3] FIG. 2 is a block diagram showing the electrical configuration within the camera 100. [Figure 4] FIG. 1 is a block diagram showing an example of the functional configuration of a camera 100. [Figure 5] 1 is a flowchart of processing performed by the camera 100. [Figure 6] 10 is a flowchart showing details of the process in step S503. [Figure 7] 10 is a flowchart showing details of the process in step S503. [Figure 8] 10 is a flowchart showing details of the process in step S503. [Figure 9] (a) is a diagram showing an example of a screen displayed on the display panel in the "continued use by user P" state, (b) is a diagram showing an example of a screen displayed on the display panel in the "unidentified user" state, and (c) is a diagram showing an example of a screen displayed when it is determined that user P of camera 100 is not continuing to use camera 100. [Figure 10] FIG. 10 is a diagram showing an example of a screen displayed on the display panel when a setting mode is touched. [Figure 11] FIG. 13 is a block diagram showing an example of the functional arrangement of a camera 100 according to a sixth embodiment. [Figure 12] 13 is a flowchart showing details of the process in step S503 in the sixth embodiment. [Figure 13] 13 is a flowchart of processing performed by the camera 100 in a modification of the sixth embodiment. [Figure 14]FIG. 13 is a block diagram showing an example of the functional arrangement of a camera 100 according to a seventh embodiment. [Figure 15] 13 is a flowchart showing details of the process of step S503 in the seventh embodiment. [Figure 16] 13 is a flowchart showing details of the process of step S503 in the first modification of the seventh embodiment. [Figure 17] 13 is a flowchart showing details of the process of step S503 in Modifications 2 and 3 of the seventh embodiment. [Figure 18] 13 is a flowchart showing details of the process of step S503 in the eighth embodiment and its first modification example. [Figure 19] 13 is a flowchart showing details of the process of step S503 in Modifications 2 and 3 of the eighth embodiment. [Figure 20] 13 is a flowchart showing details of the process of step S503 in the ninth embodiment and its first modification example. [Figure 21] 13 is a flowchart showing details of the process of step S503 in the second modification of the ninth embodiment. [Figure 22] 20 is a flowchart showing details of step S503 in the tenth embodiment. [Figure 23] FIG. 22 is a block diagram showing an example of the functional arrangement of a camera 100 according to an eleventh embodiment. [Figure 24] 22 is a flowchart showing details of processing in the eleventh embodiment. [Figure 25] 23 shows a display example of the display panel of the eyepiece 102 in the eleventh embodiment. [Figure 26] FIG. 22 is a block diagram showing an example of the functional arrangement of a camera 100 according to a twelfth embodiment. [Figure 27] 22 is a flowchart showing details of processing in the twelfth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention claimed. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0009] [First embodiment] First, an example of the appearance of a camera 100 (digital still camera; interchangeable lens camera), which is an example of an imaging device according to this embodiment, will be described with reference to Fig. 1. Fig. 1(a) is a front perspective view of the camera 100, and Fig. 1(b) is a rear perspective view of the camera 100. For the sake of explanation, in Fig. 1, the optical axis direction of the camera 100 is defined as the Z-axis direction, and of the two axes perpendicular to the Z-axis, the horizontal direction is defined as the X-axis and the vertical direction is defined as the Y-axis.

[0010] 1(a), camera 100 has photographing lens unit 100A and camera housing 100B. Camera housing 100B is provided with release button 101, which is an operating member that receives operational input for photographing from the user of camera 100 (photographer).

[0011] As shown in FIG. 1(b), an eyepiece 102 (finder) through which the user looks to view a display device 214 (display panel) included in the camera housing 100B is disposed on the back of the camera housing 100B. Also disposed on the back of the camera housing 100B are operation members 103-105 that accept various operations from the user. The operation member 103 is, for example, a touch panel that accepts touch operations from the user, and the operation member 104 is, for example, an operation lever that can be pushed down in all directions as shown in FIG. 1(c). The operation member 105 is, for example, a four-way key that can be pressed in each of four directions. The operation member 103 (touch panel) is equipped with a display panel (for example, a liquid crystal panel), and various information such as images and text is displayed on the display panel.

[0012] 2 is a cross-sectional view of camera 100 cut along the YZ plane defined by the Y axis and Z axis shown in Fig. 1(a), and shows a rough internal configuration of camera 100, including the configuration related to the following description. In Fig. 2, functional units similar to those shown in Fig. 1 are assigned the same reference numerals as those functional units, and descriptions of those functional units will be omitted.

[0013] The photographing lens unit 100A includes two lenses 201 and 202, an aperture 203, an aperture driver 204, a lens drive motor 205, a lens driver 206, a photocoupler 207, a pulse plate 208, a mount contact 209, and a focus adjustment circuit 210.

[0014] Lens drive member 206 includes a drive gear, and photocoupler 207 detects the rotation of pulse plate 208, which is linked to lens drive member 206, and transmits the detection result to focus adjustment circuit 210. Focus adjustment circuit 210 drives lens drive motor 205 based on the detection result from photocoupler 207 and information (lens drive amount information) from camera housing 100B, moving lens 201 and changing the focus position. Mount contact 209 is an interface for communication between photographing lens unit 100A and camera housing 100B. While FIG. 2 shows two lenses 201 and 202 for simplicity, more than two lenses are actually included in photographing lens unit 100A. Aperture drive unit 204 drives and controls aperture 203 to open and close the aperture.

[0015] The camera housing 100B includes an image sensor 211, a CPU 212, a memory unit 213, a display device 214, and a display device drive circuit 215. The image sensor 211 photoelectrically converts an image of a subject that has entered through the lenses 201 and 202 into a RAW image. The CPU 212 is a central processing unit of a microcomputer and controls the overall operation of the camera 100 using computer programs and data stored in the memory unit 213. The memory unit 213 is a memory unit that can store various information. For example, the CPU 212 performs various image processing, including development processing, on a RAW image output from the image sensor 211 to generate a captured image and stores it in the memory unit 213. The display device 214 displays various information, such as images and text. For example, the display device 214 is a liquid crystal panel that displays the captured image and various information. The display device drive circuit 215 drives and controls the display device 214. A user can view the screen of the display device 214 through the eyepiece 102.

[0016] Camera housing 100B further includes light sources 216a and 216b, a beam splitter 217, a light receiving lens 218, and an eye imaging element 219. Light sources 216a and 216b are light sources conventionally used in single-lens reflex cameras to detect the line of sight from the relationship between the pupil and a reflection image of light reflected from the cornea (corneal reflection image), and are light sources for illuminating eyeball 220 of a user looking through the viewfinder (eyepiece 102). For example, light sources 216a and 216b are infrared light-emitting diodes that emit infrared light that is insensitive to the user, and are arranged around eyepiece 102. An optical eye image (an optical image formed by light emitted from light sources 216a and 216b and reflected by eyeball 220), which is an optical image of the illuminated eyeball 220, passes through eyepiece 102 and is reflected by beam splitter 217. The eye optical image is then formed by the light receiving lens 218 on the eye imaging element 219, which has a plurality of photoelectric conversion elements (e.g., CCD or CMOS) arranged two-dimensionally. The light receiving lens 218 positions the pupil of the eyeball 220 and the eye imaging element 219 in a conjugate imaging relationship. The line of sight of the eyeball 220 can be detected from the position of the corneal reflection image in the eye optical image formed on the eye imaging element 219, and information about the line of sight is obtained as a result of the detection. For example, the information about the line of sight may include at least one of information indicating the line of sight direction (the direction of the line of sight) and information indicating the viewpoint (the position where the line of sight is fixed) on the screen of the display device 214. The viewpoint can be regarded as the position at which the user is looking or as the line of sight position. The eye imaging element 219 then outputs the formed eye optical image (an image obtained by capturing the eyeball 220) as an eye image (an analog image signal).

[0017] Next, the electrical configuration within camera 100 will be described using the block diagram of Fig. 3. CPU 212 can transmit signals via mount contacts 209 to focus adjustment circuit 210 arranged in photographing lens unit 100A and to aperture control circuit 306 included in aperture drive section 204 within photographing lens unit 100A. For example, CPU 212 can transmit information on the lens drive amount to focus adjustment circuit 210 via mount contacts 209. CPU 212 can also transmit information on the drive amount of aperture 203 to aperture control circuit 306 via mount contacts 209. CPU 212 also controls the operation of each of the other sections.

[0018] The memory unit 213 is a memory unit capable of storing captured images, eye images, and gaze correction parameters, which are parameters for correcting the detected gaze. The gaze detection circuit 301 A / D converts the eye image (analog image signal) output from the eye imaging element 219 when an eye optical image is formed on the eye imaging element 219 to generate an eye image, which is a digital image signal. The gaze detection circuit 301 then extracts feature points necessary for gaze detection from the eye image, which is the generated digital image signal, detects the user's gaze from the positions of the feature points, and outputs information about the gaze as a result of the detection. The gaze detection circuit 301 also stores the eye image, which is the generated digital image signal, in the memory unit 213.

[0019] The photometry circuit 302 performs predetermined processing (e.g., amplification, logarithmic compression, and A / D conversion) on the image signal obtained from the image sensor 211, which also functions as a photometry sensor, such as a luminance signal corresponding to the brightness of the field, and sends the result to the CPU 212 as field luminance information.

[0020] The autofocus detection circuit 303 A / D converts output signals from multiple detection elements (multiple pixels) used for phase difference detection included in the image sensor 211, and calculates the distance to the subject corresponding to each focus detection point from the A / D converted output signals. This is a well-known technique known as image plane phase difference AF.

[0021] The signal input circuit 304 is connected to the switches SW1 and SW2 of the release button 101. The switch SW1 outputs an ON signal when the release button 101 is pressed for the first time, starting the preparatory operations for shooting (e.g., photometry and distance measurement) of the camera 100. The switch SW2 outputs an ON signal when the release button 101 is pressed for the second time, starting the shooting operation. The ON signals from the switches SW1 and SW2 are input to the signal input circuit 304 and transmitted to the CPU 212. When the CPU 212 receives the ON signal from the switch SW1, it controls the operation of each unit to start the preparatory operations for shooting of the camera 100. When the CPU 212 receives the ON signal from the switch SW2, it controls the operation of each unit to start the shooting operation. When the switch SW1 is turned ON, line-of-sight detection may be started. The light source drive circuit 305 drives and controls the light sources 216a and 216b.

[0022] Furthermore, operation members 103 to 105 are also connected to CPU 212. When the user operates operation members 103 to 105, operation members 103 to 105 output operation signals according to the operation from the user to CPU 212, and CPU 212 performs processing (control) according to the operation signals. For example, CPU 212 moves a frame (item, indicator) displayed at the viewpoint according to the operation signal.

[0023] The CPU 212 is also connected to a communication unit 307, a GPS receiving unit 308, a timer 309, and an inertial sensor 310. The communication unit 307 performs wireless communication with information devices such as a user's smartphone, tablet terminal, smartwatch, or PC, and acquires GPS position information of the information device. The wireless communication is assumed to be, but is not limited to, Wi-Fi or Bluetooth. The GPS receiving unit 308 receives radio waves from a GPS satellite and acquires GPS position information of the camera 100. The timer 309 outputs the current time. The inertial sensor 310 (for example, an acceleration sensor and an angular velocity sensor) outputs the acceleration and angular velocity acting on the camera 100 as output values.

[0024] An example of the functional configuration of the camera 100 related to the processing described below is shown in the block diagram of Fig. 4. In this embodiment, each functional unit shown in Fig. 4 is implemented in hardware. Each functional unit can be implemented in one or more pieces of hardware shown in Figs. 2 and 3, and operates under the control of the CPU 212.

[0025] 4 may be implemented as software (computer program). In this case, the computer program is stored in the memory unit 213, and the CPU 212 executes the computer program to realize the functions of the corresponding functional units.

[0026] The process performed by camera 100 according to this embodiment to identify the user of camera 100 and store the captured image in memory unit 213 in association with the identification result will be described with reference to the flowchart in Fig. 5. The process according to the flowchart in Fig. 5 starts when camera 100 is powered on, completes startup of camera 100, and becomes available for use, and is repeated until the process can no longer be continued due to reasons such as the camera 100 being powered off.

[0027] In step S501, the user identification unit 402 performs iris authentication on the eye image stored in the memory unit 213 to perform processing to identify the user of the camera 100. In step S502, the user identification unit 402 determines whether or not the user of the camera 100 has been successfully identified in step S501.

[0028] If the result of this determination is that the user of the camera 100 has been successfully identified in step S501, the process proceeds to step S503, and if the user of the camera 100 has not been successfully identified in step S501, the process proceeds to step S511.

[0029] For example, the user identification unit 402 identifies the registered iris that is most similar to the iris detected from the eye image as the similar registered iris from among the irises of each user (registered irises) pre-registered in the memory unit 213. Then, the user identification unit 402 identifies the user corresponding to the similar registered iris as the user of the camera 100.

[0030] Note that the user identification unit 402 determines that user identification has failed if the similarity between the iris detected from the eye image and the similar registered iris is less than a threshold. Furthermore, if no one is looking through the viewfinder 102 and therefore the eye image does not include an iris, the user identification unit 402 determines that user identification has failed because no iris is recognized from the eye image. Furthermore, if something other than an eyeball is placed near the viewfinder 102, the user identification unit 402 also determines that user identification has failed because no iris is recognized from the eye image.

[0031] In step S511, continuous use determination unit 403 determines whether or not a photograph has been taken by photographing unit 401 functioning as photographing lens unit 100A, for example by pressing release button 101. If the result of this determination is that a photograph has been taken by photographing unit 401, the process proceeds to step S512, and if a photograph has not been taken by photographing unit 401, the process proceeds to step S501.

[0032] In step S512, the recording unit 404 assigns, as metadata, photographer information indicating that "the photographer is unidentified" to the captured image captured by the imaging unit 401. The recording unit 404 then records the captured image (captured image file) to which the metadata has been assigned in the memory unit 213. The process then proceeds to step S501.

[0033] On the other hand, in step S503, the continuous use determination unit 403 determines whether the user of the camera 100 identified in step S501 (hereinafter referred to as user P) continues to use the camera 100. Details of the processing in step S503 will be described later.

[0034] If the result of such a determination is that user P continues to use camera 100, processing proceeds to step S521 via step S504; if user P does not continue to use camera 100, processing proceeds to step S501 via step S504.

[0035] In step S521, similar to step S511 above, continuous use determination unit 403 determines whether or not a photograph has been taken by photographing unit 401, for example by pressing release button 101. If the result of this determination is that a photograph has been taken by photographing unit 401, the process proceeds to step S522, and if a photograph has not been taken by photographing unit 401, the process proceeds to step S503.

[0036] In step S522, the recording unit 404 assigns, as metadata, photographer information indicating that "the photographer is user P" to the captured image captured by the imaging unit 401. Then, the recording unit 404 records the captured image (captured image file) to which the metadata has been assigned in the memory unit 213. Then, the process proceeds to step S503.

[0037] In this way, the continued use determination unit 403 determines the period during which user P continues to use the camera 100 as the "range of continued use by user P," and images taken within this range of continued use are assigned metadata containing photographer information such as "the photographer is user P."

[0038] The method for recording photographer information is not limited to adding it to the captured image as metadata. For example, it may be stored in the form of a table in which the hash value of the captured image corresponds to the photographer information, or it may be connected to an external server via a communication channel such as wireless communication and uploaded as a record combining the captured image and the photographer information. Furthermore, methods for preventing tampering and ensuring authenticity, such as encryption or electronic signature, may also be used with this information. Furthermore, in step S512, the recording unit 404 may store the captured image captured by the image capturing unit 401 in the memory unit 213 without adding photographer information indicating that the "photographer is unidentified" to the captured image.

[0039] In this way, the recording unit 404 controls the recording so that images taken by the camera 100 are associated with information about the user and recorded in the memory unit 213 between the time the user of the camera 100 is identified and the time it is determined that the user is no longer using the camera.

[0040] In addition, if the user of camera 100 cannot be identified, recording unit 404 controls the recording so that the image captured by camera 100 and information indicating that the user of camera 100 is unidentified are recorded in memory in association with each other.

[0041] Next, details of the process in step S503 above will be described with reference to the flowchart in Fig. 6. In step S601, the continued use determination unit 403 determines whether the end button on at least one of the operation members 103 to 105 has been pressed by the user P. For example, if the operation member 103 is the display panel shown in Fig. 9(a), and the user P touches button 902 displayed on the display panel, the continued use determination unit 403 determines that the end button on the operation member 103 has been pressed by the user P.

[0042] The end button is not limited to a button on the display panel, but may be provided as a physical button. Also, when an operation lever such as the operation member 104 is pressed, it may be determined that the end button of the operation member 104 has been pressed. Also, when a power button or the like is pressed, it may be determined that the end button has been pressed. If it is determined that the end button has been pressed, the process proceeds to step S605, and if the end button has not been pressed, the process proceeds to step S602.

[0043] In step S602, the continued use determination unit 403 determines whether or not the user P has performed a mode change operation on at least one of the operation members 103 to 105. For example, if the operation member 103 is the display panel shown in Fig. 9(a), when the user P performs a touch operation on a mode different from the current mode among the three modes included in the list 903 displayed on the display panel, the continued use determination unit 403 determines that the user P has performed a mode change operation on the operation member 103.

[0044] 9(a), list 903 includes three modes: a mode for taking pictures (shooting mode), a mode for displaying information related to files stored in memory unit 213 on the display panel (viewing mode), and a mode for setting camera 100 (setting mode). Therefore, when user P touches one of these three modes that is different from the mode currently set in camera 100, continued use determination unit 403 determines that a mode change operation has been performed by user P on operating member 103. When the mode currently set in camera 100 is the shooting mode, the mode change operation by the user is an operation for changing the operating state to a state other than shooting, and therefore this is considered to be an operation by the user to end shooting.

[0045] If the result of such determination is that the user P has performed an operation to change the mode on at least one of the operation members 103 to 105, the process proceeds to step S605. On the other hand, if the user P has not performed an operation to change the mode on at least one of the operation members 103 to 105, the process proceeds to step S604.

[0046] In step S604, the continuous use determination unit 403 determines that the user P continues to use the camera 100. On the other hand, in step S605, the continuous use determination unit 403 determines that the user P does not continue to use the camera 100.

[0047] In addition to the above, there are various other operations that can be performed by user P to indicate the end of use of camera 100, such as sending an end signal from a remote control or a device in wireless communication. These methods may also be combined, or only some of them may be performed.

[0048] In this way, by determining whether the user has performed an operation to end photography and defining the range of continuous use as from the identification of the user until the user performs an operation to end photography, the convenience of the device that stores user information together with the photographed image is improved.

[0049] Note that the method for identifying the user of camera 100 is not limited to iris authentication. For example, camera 100 may identify the user of camera 100 using other biometric information such as a fingerprint, vein pattern, or face, or may identify the user of camera 100 by requiring the user to input authentication information such as a personal identification number or password. Alternatively, camera 100 may identify the user of camera 100 by reading authentication information such as the user's personal identification number or password from a recording medium such as an ID card on which the information is recorded, and performing authentication. An authentication dongle for performing such authentication may be connected to camera 100. Alternatively, the authentication may be performed by wirelessly communicating with an authentication device such as a nearby smartphone or smartwatch, or by combining these methods to perform two-step authentication.

[0050] In this way, the method for identifying the user of camera 100 (authentication method) is not limited to a specific method, and the above authentication may be performed by camera 100 alone, or by communicating with another device.

[0051] Furthermore, in this embodiment, the memory unit 213 is a memory unit built into the camera housing 100B, but it may also be a "memory unit that is physically detachably connected to the camera housing 100B," such as an SD card or USB memory.

[0052] <Modification> Instead of the user pressing the end button to decide to end use of camera 100, the "end of use of camera 100" may be decided when the user releases the continue use button that the user has been continuing to press. In this case, in step S601, it is determined whether the user has released the continue use button that the user has been continuing to press. If the user has released the continue use button that the user has been continuing to press, the process proceeds to step S605, and if the user has not released the continue use button that the user has been continuing to press, the process proceeds to step S602.

[0053] The continue use button can be provided as a touch panel or operating lever applicable to the operating members 103 to 105, and convenience can be improved by, for example, placing the button on the grip of the camera 100 so that the user can easily continue to press it while holding it.

[0054] That is, in this embodiment, if an instruction to continue using the camera 100 (an instruction to continue using the camera) is being input (inputting state) in step S601, the process proceeds to step S602. On the other hand, if the instruction to continue using the camera is no longer being input in the inputting state in step S601, the process proceeds to step S605. Such an instruction to continue using the camera can be input in various ways, as described above.

[0055] [Second embodiment] In the following embodiments, including this embodiment, differences from the first embodiment will be described, and unless otherwise specified below, it will be assumed that they are the same as the first embodiment. In this embodiment, a method for determining the continuous use range based on an action that is not an explicit operation by the user but that presumes the end of shooting will be described. Note that this method may be used in combination with the method for determining the continuous use range in the first embodiment.

[0056] Details of the processing of step S503 according to this embodiment will be described with reference to the flowchart in Fig. 7. In Fig. 7, processing steps that are the same as those shown in Fig. 6 are assigned the same step numbers as those steps, and descriptions of those processing steps will be omitted.

[0057] In step S701, continued use determination unit 403 acquires "information indicating whether photographing lens unit 100A is mounted (connected) to camera housing 100B" from mount contact 209. Continued use determination unit 403 then determines whether the acquired information indicates "photographing lens unit 100A is mounted to camera housing 100B" or "photographing lens unit 100A is not mounted to camera housing 100B."

[0058] If the result of this determination indicates that the acquired information indicates that "photographing lens unit 100A is mounted on camera housing 100B (photographing lens unit 100A has not been removed from camera housing 100B)", the process proceeds to step S702.

[0059] On the other hand, if the acquired information indicates that "photographing lens unit 100A is not mounted on camera housing 100B (photographing lens unit 100A is detached from camera housing 100B)", the process proceeds to step S605.

[0060] In step S702, the continued use determination unit 403 acquires "information indicating the input brightness to the image sensor 211" from the mount contact 209. The continued use determination unit 403 then determines whether the "input brightness to the image sensor 211" indicated by the acquired information is equal to or less than a threshold, for example, whether the input is completely dark due to a lens cap being attached to the photographing lens unit 100A.

[0061] If the result of this determination is that the "input brightness to the image sensor 211" indicated by the acquired information is less than or equal to the threshold value, processing proceeds to step S605; if the "input brightness to the image sensor 211" indicated by the acquired information is greater than the threshold value, processing proceeds to step S703.

[0062] In step S703, the continued use determination unit 403 acquires "information indicating whether or not an external device is connected to the camera 100" from the communication module of the camera 100. Then, the continued use determination unit 403 determines whether or not the acquired information indicates "an external device is connected to the camera 100."

[0063] If the result of this determination is that the acquired information indicates that "an external device is not connected to the camera 100" (if communication between the camera 100 and the external device has been disconnected), the process proceeds to step S605. On the other hand, if the acquired information indicates that "an external device is connected to the camera 100" (if communication between the camera 100 and the external device has not been disconnected), the process proceeds to step S604.

[0064] In this embodiment, three operations have been described as examples of operations for estimating whether or not image capture by camera 100 has ended: "determine whether photographing lens unit 100A is mounted on camera housing 100B," "determine whether the input brightness to image sensor 211 is equal to or lower than a threshold," and "determine whether communication between camera 100 and an external device has been disconnected." However, the operations for estimating whether or not image capture by camera 100 has ended are not limited to these three operations. For example, if it is detected that an accessory other than a lens, such as a flash or a release cable, has been removed from camera housing 100B, it may be determined that image capture by camera 100 has ended. Furthermore, if it is detected that the battery cover has been opened or the SD card has been removed, it may be determined that image capture by camera 100 has ended. Furthermore, these methods may be combined, or only some of them may be performed.

[0065] In this way, the range of continued use is determined by an action that presumes the end of photography, even without an explicit operation by the user, thereby improving the convenience of the device that authenticates and records the photographer.

[0066] [Third embodiment] In this embodiment, a method for determining the continuous use range while the user is performing an operation for shooting will be described. Details of the processing of step S503 according to this embodiment will be described with reference to the flowchart in Fig. 8. In Fig. 8, processing steps that are the same as those shown in Fig. 6 are assigned the same step numbers as those processing steps, and descriptions of those processing steps will be omitted.

[0067] In step S801, the continuous use determination unit 403 determines whether continuous shooting is being performed by long pressing of the release button 101. If the result of this determination is that continuous shooting is being performed by long pressing of the release button 101, the process proceeds to step S802, and if continuous shooting is not being performed by long pressing of the release button 101, the process proceeds to step S803.

[0068] In step S802, the continuous use determination unit 403 determines whether the number of images captured after iris authentication is equal to or greater than a predetermined maximum number of images N. The continuous use determination unit 403 counts up the number of images captured each time an image is captured after iris authentication.

[0069] If the result of this determination is that the number of images taken after iris authentication is equal to or greater than the predetermined maximum number of images N, processing proceeds to step S605; if the number of images taken after iris authentication is less than the predetermined maximum number of images N, processing proceeds to step S803.

[0070] In step S803, the continuous use determination unit 403 determines whether or not the user is gripping the grip portion provided on the camera housing 100B, using a contact sensor provided on the grip portion.

[0071] If the result of this determination is that the user is gripping the grip portion, the process proceeds to step S604, and if the user is not gripping the grip portion, the process proceeds to step S804.

[0072] In step S804, continued use determination unit 403 determines whether the direction (posture) in which camera 100 is facing is changing, using a gyro sensor provided in camera housing 100B. For example, continued use determination unit 403 determines that the direction (posture) in which camera 100 is facing is changing if the difference between the direction measured previously by the gyro sensor and the direction measured this time by the gyro sensor is equal to or greater than a threshold.

[0073] If the result of this determination is that the direction (posture) of the camera 100 has changed, the process proceeds to step S604, and if the direction (posture) of the camera 100 has not changed, the process proceeds to step S805.

[0074] Here, the bottom of the camera housing 100B according to this embodiment is provided with a fixing screw hole and a screw detection sensor for detecting whether or not the fixing screw hole is screwed in. When the camera housing 100B is to be fixed to a device such as a tripod or a pan head, the camera housing 100B can be fixed to the device by attaching the camera housing 100B to the device and then screwing the fixing screw hole.

[0075] In step S805, the continuous use determination unit 403 determines whether or not a screw is fastened in the fixing screw hole using the screw fastening detection sensor. If the result of this determination is that a screw is fastened in the fixing screw hole, the process proceeds to step S604, and if the screw is not fastened in the fixing screw hole, the process proceeds to step S605.

[0076] As described above, according to this embodiment, the range in which the user is performing operations for photographing is determined as the continuous use range, thereby improving the convenience of the device for authenticating and recording the photographer.

[0077] [Fourth embodiment] It can be assumed that camera 100 is being used continuously while camera 100 itself is moving, such as when the user carries camera 100 around or turns camera 100 toward a subject. To this end, camera 100 is equipped with an inertial sensor, gyro, or the like to detect camera movement, and continuous use determination unit 403 determines the period during which movement of camera 100 is detected as the continuous use range.

[0078] Furthermore, the position of the camera 100 may be used to determine that the camera 100 is in continuous use while it is within a specific geographical range. The camera 100 may be equipped with a position measuring device such as a GPS, and the continuous use determination unit 403 may determine that the continuous use range is the period when the position measured by the position measuring device is within a specific range.

[0079] Alternatively, a clock may be installed in camera 100, and a specific time period may be set as the continuous use range. Furthermore, it may be determined that continuous use is occurring as a series of images while similar images are being captured. Continuous use determination unit 403 evaluates the similarity of consecutively captured images based on differential images and statistics, and determines the period during which images with a similarity below a threshold are captured as the continuous use range.

[0080] The above-mentioned methods are merely examples of methods for determining the continuous use range, and are not intended to be limiting. In this way, by determining the continuous use range using various information, the convenience of the device for authenticating and recording the photographer is improved.

[0081] [Fifth embodiment] In this embodiment, a screen displayed in relation to the processing in the above embodiment will be described. As an example, in the flowchart of FIG. 5, the period during which the loop in steps S501 and S502 is repeated without identifying the user will be referred to as a "user unidentified state." Furthermore, the period during which the loop in steps S503 and S504 continues to determine that use is continuing will be referred to as a "state of continued use by user P" (user P is the user identified in step S501). Note that display-related processing, i.e., display control such as screen display, screen switching and display changes in response to user operations, etc., is performed by CPU 212.

[0082] Fig. 9(a) is a diagram showing an example of a screen displayed on the display panel serving as the operation member 103 in the "state of continued use by user P." The screen in Fig. 9(a) includes a display 901 including the subject 900, the user of the camera 100 (user P), and the time elapsed since iris authentication, a list 903 for mode selection, and an end button 902. The screen also includes a display 904 showing the number of shots / maximum number of shots N (here, N=100) after iris authentication.

[0083] 9(b) is a diagram showing an example of a screen displayed on the display panel serving as the operation member 103 in the "unidentified user state." Display 901 indicates that the user has not been identified by iris authentication.

[0084] The display 904 allows the user to grasp the number of remaining shots. For example, the camera 100 may change the color or size of the display 904 or make a sound when the number of shots approaches N.

[0085] Furthermore, if the continued use determination unit 403 determines that the user P of the camera 100 has not continued to use the camera 100, the screen shown in FIG. 9C may be displayed on the display panel serving as the operation member 103. The display 901 includes a message indicating that the user P of the camera 100 has ended continued use of the camera 100 and the reason for the end. The display 901 may be displayed for a specified period of time (for example, five seconds). In the example of FIG. 9C, since it was determined in step S701 that the photographing lens unit 100A has been detached from the camera housing 100B, the display 901 indicates this as the reason for the end.

[0086] This allows the user to understand the cause even if an unexpected termination occurs. To make it easier for the user to notice, the entire screen may be highlighted by changing the color, or a sound may be played.

[0087] These displays may be displayed on the display device 214 instead of on the display panel of the operation member 103, and may be superimposed on the viewfinder, or may be displayed on both.

[0088] Another display method may be to provide an LED lamp on the camera 100, which lights up during "continuous use state," turns off during "unidentified user state," and flashes when continued use ends. In this case, similar information can be displayed by changing the color of the LED lamp for each user. To prevent the light from the LED lamp from affecting the photograph, it may be controlled to turn it off while the shutter is being taken.

[0089] In addition, the status of the continuous use range can be notified to the user by a shutter sound. When the release button 101 is pressed, a different shutter sound is made for each user P in the "continuous use state by user P," and a different shutter sound is made in the "unidentified user state." This allows the user to know whether they are within the continuous use range, which user has been authenticated, and who has been recorded as the photographer in the photo. Instead of the shutter sound after the shutter is released, a sound indicating the status can be made to sound when the shutter is pressed halfway.

[0090] Here, when the user touches the setting mode in list 903, the screen shown in Fig. 10 is displayed on the display panel serving as operation member 103. List 1001 displays a list of configurable items, and when the user touches "authentication continuation method" from list 1001, list 1002 is displayed.

[0091] List 1002 is used to individually enable (ON) or disable (OFF) the various methods described above for determining whether the user of camera 100 is continuing to use camera 100, and to set whether to display or hide end button 902. List 1002 also includes a numeric box 1003. The user can input the maximum number of images N to be taken into numeric box 1003. In this way, by displaying the status of camera 100 to users and allowing settings to be made for each user, the convenience of the device for authenticating and recording photographers is further improved.

[0092] [Sixth embodiment] In this embodiment, a method for determining a continuous use range based on spatial movement information of a camera will be described. An example of the functional configuration of camera 100 related to the processing described below is shown in the block diagram of Fig. 11. The functional block includes an imaging unit 401, an identification unit 402, a movement acquisition unit 1101, a continuous use determination unit 403, and a recording unit 404. The imaging unit 401, the identification unit 402, and the recording unit 404 are the same as those in the first embodiment, so their description will be omitted.

[0093] The motion acquisition unit 1101 acquires information about the acceleration and angular velocity acting on the camera 100 from the inertial sensor 310. The continuous use determination unit 403 of this embodiment determines whether the user is continuously using the camera 100 based on the motion information acquired by the motion acquisition unit 1101.

[0094] Details of the processing of step S503 according to this embodiment will be described with reference to the flowchart in Fig. 12. In step S1201, the continued use determination unit 403 acquires the conditions for determining that the user P is continuing to use the camera 100. Here, as an example, the condition for determining that the user P is continuing to use the camera 100 is that the camera 100 is determined to be continuing to move based on the criteria described below.

[0095] In step S1202, the motion acquisition unit 1101 acquires, as motion information, the acceleration and angular velocity acting on the camera 100 from the inertial sensor 310. The motion information acquired each time step S1202 is executed is stored in memory as history.

[0096] In step S1203, it is determined whether the motion information acquired in step S1202 satisfies the condition acquired in step S1201. Based on the history of the motion information acquired in step S1202, the continuous use determination unit 403 determines whether the camera 100 is currently moving or stationary. Specifically, if the acquired acceleration has fluctuated within the most recent fixed period of time by a range equal to or greater than a predetermined threshold, or if the latest angular velocity is equal to or greater than a predetermined threshold, it is determined that the camera 100 is currently moving.

[0097] If it is determined that the camera 100 is moving, the process proceeds to step S1204, where it is determined that the user P is "continuing use," and the flow ends. On the other hand, if it is determined that the camera 100 is stationary, the process proceeds to step S1205, where it is determined that the user P is "not continuing use," and the flow ends.

[0098] As a result, for example, while a user is holding camera 100, it is possible to detect shaking of camera 100 and determine that the user is continuing to use it. On the other hand, if the user puts camera 100 down on a desk or the like and then lets go of it, camera 100 comes to a standstill, and it is possible to determine that the user is not continuing to use camera 100.

[0099] According to the method of this embodiment, as long as the camera 100 continues to move, it is determined that the same user is continuing to use the camera, and authentication continues, thereby improving the convenience of the device. Note that in this embodiment, the acceleration and angular velocity acting on the camera 100 are acquired from the inertial sensor 310 as motion information, but the method of acquiring motion information is not limited to this. For example, images captured by the camera 100 may be continuously acquired, and the motion of the camera 100 may be estimated by analyzing optical flow, which is a well-known technique. Alternatively, a differential value of position information acquired by a position measurement unit such as a GPS may be used as motion information.

[0100] Furthermore, in the present embodiment, the condition for determining that user P is continuing to use camera 100 has been described as the condition that camera 100 is continuously moving. However, the determination condition based on movement information is not limited to this. For example, assuming that camera 100 is fixed to a tripod or the like and is intermittently capturing images, it may be determined that user P is continuing to use camera 100 while camera 100 is stationary. Alternatively, a condition may be added that determines that user P has stopped continuing to use camera 100 when a specific movement pattern, such as the user quickly shaking camera 100 twice, is detected. Alternatively, a condition may be set that determines that user P is continuing to use camera 100 while camera 100 is moving according to a pre-registered pattern, such as in programmed shooting using an automatic camera platform. These conditions may be selected in advance or combined based on user operations.

[0101] <Modification> In the sixth embodiment, whether the user continues to use the camera 100 is determined based on pre-set conditions, but in this modified example, the conditions are determined based on movement information at the time the user is identified.

[0102] The camera's appearance in FIG. 1, the camera's configuration in FIG. 2, the camera's electrical configuration in FIG. 3, and the camera's functional configuration in FIG. 11 are all the same as those in the fifth embodiment, and therefore descriptions thereof will be omitted. The processing steps performed by camera 100 according to this embodiment to determine the user's continued use state will be described below using the flowchart in FIG. 13. The steps from step S501 to step S522 are the same as those in the first embodiment, and therefore descriptions thereof will be omitted. Details of the processing in step S503 are the same as those in FIG. 12 described in the sixth embodiment, except for the points described below, and therefore descriptions thereof will be omitted.

[0103] In step S1301, when it is determined in the processing of step S502 that authentication is successful, the motion acquisition unit 1101 acquires motion information. Here, the motion acquisition unit 1101 acquires continuous motion information for a certain period of time. Thereafter, the continuous use determination unit 403 determines from the motion information whether the camera 100 is moving or stationary at the time of successful authentication, based on the same criteria as in step S1203.

[0104] In step S1201, the continuous use determination unit 403 sets a condition for determining that the user P is continuing to use the camera 100, depending on the movement of the camera 100 at the time of successful authentication determined in step S1301. Specifically, if the camera 100 was moving at the time of successful authentication, the condition for determining that the user P is continuing to use the camera 100 is that the camera 100 continues to move. Also, if the camera 100 was stationary at the time of successful authentication, the condition for determining that the user P is continuing to use the camera 100 is that the camera 100 continues to be stationary.

[0105] This allows the continued use determination unit 403 to automatically set the determination conditions without the user P having to actively set them, and authentication is continued under conditions according to the usage status of the camera 100, thereby improving the convenience of the device.

[0106] [Seventh embodiment] In the seventh embodiment, a method will be described in which the camera 100 is paired with an information device such as a smartphone carried by the user of the camera 100, and when the positions of the camera 100 and the information device are within a predetermined distance, it is determined that the user is continuing to use the camera. When the positions of the user and the camera 100 are close to each other, the user remains authenticated, thereby reducing the frequency of authentication. As a result, the convenience of the device is improved.

[0107] An example of the functional configuration of camera 100 related to the processing described below is shown in the block diagram of Fig. 14. The functional block includes an image capturing unit 401, an identification unit 402, an information acquisition unit 1401, a continued use determination unit 403, and a recording unit 404. The image capturing unit 401, the identification unit 402, and the recording unit 404 are the same as those in the first embodiment, and therefore description thereof will be omitted.

[0108] The information acquisition unit 1401 acquires GPS location information of the camera 100 and GPS location information of an information device owned by the user with which the camera 100 is paired from the GPS receiving unit 308. The continuous use determination unit 403 sets the location of the information device acquired by the information acquisition unit 1401 as a reference location, and determines whether the user is continuously using the camera 100 based on this location and the location of the camera 100.

[0109] The following describes the processing steps performed by camera 100 according to this embodiment to determine whether the user is continuing to use the camera. Steps S501 to S522 in Fig. 5 are the same as those in the first embodiment, and therefore descriptions thereof will be omitted. The procedure for determining whether user P is continuing to use the camera in step S503 will be described below with reference to the flowchart in Fig. 15.

[0110] First, in step S1501, the information acquisition unit 1401 acquires the positions of the camera 100 and the information device. The position information of the camera 100 and the information device is GPS position information acquired by the information acquisition unit 1401.

[0111] In step S1502, continued use determination unit 403 determines whether the distance between the position of the information device and camera 100 is equal to or less than a predetermined distance, using the position of the information device as a reference position. If the distance is equal to or less than the predetermined distance, the process proceeds to step S1503, where it is determined that user P is "continuing use," and the flow ends. If the distance exceeds the predetermined distance, the process proceeds to step S1504, where it is determined that user P is "not continuing use," and the flow ends.

[0112] According to the method of this embodiment, when the distance between the information device owned by the user and the camera 100 is short, the user is allowed to continue using the device, and authentication continues between successive photographs, thereby improving the convenience of the device.

[0113] [Modification 1 of the Seventh Embodiment] Next, a first modification of the seventh embodiment will be described. In the seventh embodiment, whether the camera 100 and an information device owned by the user are close to each other is determined based on the GPS position information of both devices, but in this modification, this is determined based on whether communication is possible between the camera 100 and the information device. If short-range communication such as Bluetooth is used for communication between the camera 100 and the information device, it can be determined whether the camera 100 and the information device are close to each other based on whether communication is possible.

[0114] The functional configuration of the camera 100 in Fig. 14 is the same as that of the seventh embodiment. In this modification, the information acquisition unit 1401 in Fig. 14 acquires, from the communication unit 307, information on whether communication is possible between the camera 100 and an information device owned by the user that has already been paired with the camera 100.

[0115] The continuous use determination unit 403 determines whether the user is continuously using the camera 100 based on the communication availability information between the camera 100 and the information device acquired by the information acquisition unit 1401 .

[0116] The procedure for the camera 100 according to this embodiment to determine whether the user is continuing to use the camera 100 is the same as that shown in the flowchart of Fig. 5 in the first embodiment. This flow starts when the camera 100 is paired with an information device owned by the user and communication becomes possible. The procedure for determining whether the user P is continuing to use the camera 100 in step S503 of Fig. 5 will be described below with reference to the flowchart of Fig. 16.

[0117] First, in step S1601, the camera 100 determines whether it can communicate with the paired information device. If communication is possible, the process proceeds to step S1602, where it is determined that the user P is "continuing use," and the flow ends. If communication is not possible, the process proceeds to step S1603, where it is determined that the user P is "not continuing use," and the flow ends.

[0118] According to the method of this modification, even in a situation where the absolute positions of the information device and the camera 100 cannot be measured, authentication continues between a series of photographs, thereby improving the convenience of the device.

[0119] [Modification 2 of the Seventh Embodiment] Next, a second modification of the seventh embodiment will be described. In the first modification of the seventh embodiment, the proximity of the camera 100 to the user is determined based on whether communication is possible between the camera 100 and an information device owned by the user. In this modification, the proximity is determined by measuring the distance between the camera 100 and the user using a distance sensor built into the camera 100. The distance sensor is, for example, an ultrasonic sensor or LiDAR, but other sensors may also be used.

[0120] A distance sensor directly connected to the CPU is added to the electrical configuration of camera 100 in Fig. 3. In this modification, information acquisition unit 1401 in Fig. 14 acquires distance information output by the distance sensor of camera 100. Continuous use determination unit 403 determines whether the user is continuously using camera 100 based on the distance information acquired by information acquisition unit 1401.

[0121] The procedure for the process by which the camera 100 according to this embodiment determines whether the user is continuing to use the camera is the same as that shown in the flowchart of FIG. 5 in the first embodiment. This flow starts when the distance sensor becomes capable of measurement. At the start of this flow, the "predetermined distance" required for the determination by the continued use determination unit 403 is assumed to have been recorded in the memory unit 213 of the camera 100. The procedure for determining whether the user P is continuing to use the camera in step S503 of FIG. 5 will be described below with reference to the flowchart of FIG. 17.

[0122] First, in step S1701, the distance between camera 100 and the user output by the distance sensor is acquired as position information. In step S1702, it is determined whether the distance between the user and camera 100 is equal to or less than a predetermined distance. If it is equal to or less than the predetermined distance, the process proceeds to step S1703, where it is determined that user P is "continuing use," and the flow ends. On the other hand, if the predetermined distance is exceeded, the process proceeds to step S1704, where it is determined that user P is "not continuing use," and the flow ends.

[0123] In this modification, the distance between camera 100 and the user is measured using a distance sensor built into the camera, but other methods may also be used. A camera for detecting the user's face may be provided on the back of camera 100, and the distance between camera 100 and the user may be estimated from the size of the user's face, or the distance between camera 100 and the user may be determined to be close while the face can be detected.

[0124] According to the method of this modification, the camera 100 alone can measure the distance between the camera 100 and the user, and authentication continues between a series of photographs, improving the convenience of the device.

[0125] [Modification 3 of the Seventh Embodiment] Next, a third modification of the seventh embodiment will be described. In the second modification of the seventh embodiment, the distance between the camera 100 and the user is measured by a distance sensor built into the camera, but in this modification, the distance is measured by an external sensor such as a surveillance camera.

[0126] Measurements using a surveillance camera are performed by calculating the coordinates in real space that correspond to the position of the object detected in the captured image using conversion parameters between real space coordinates and camera coordinates obtained in advance by camera calibration.The surveillance camera has a communication function, and the measured distance information can be sent to the camera.

[0127] The functional configuration of the camera 100 in Fig. 14 is the same as that of the seventh embodiment. In this modification, in the information acquisition unit 1401 in Fig. 14, the communication unit 307 communicates with the surveillance camera and acquires the position information of the camera 100 and the user.

[0128] The continuous use determination unit 403 sets the user's position acquired by the information acquisition unit 1401 as a reference position, and determines whether the user is continuously using the camera 100 based on this position and the current position of the camera 100.

[0129] The procedure for the process by which the camera 100 according to this embodiment determines whether the user is continuing to use the camera is the same as the flowchart in FIG. 5 of the first embodiment. This flow starts when the camera 100 and the surveillance camera are able to communicate with each other. At the start of this flow, the "predetermined distance" required for the determination by the continued use determination unit 403 is assumed to have been recorded in the memory unit 213 of the camera 100. The procedure for determining whether the user P is continuing to use the camera in step S503 of FIG. 5 can be represented by the flowchart in FIG. 17, as in the second modification of the first embodiment. In this modification, the position information in step S1701 is the position of the camera 100 and the user obtained from the surveillance camera.

[0130] In this modification, a method based on images captured by a surveillance camera is presented as a method for determining the positions of the camera 100 and the user in a common coordinate system. However, the method for determining the positions of the camera 100 and the user is not limited to this. Another method involves preparing a transmitter such as an optical beacon transmitter, determining the positions of the camera 100 and the user from signals received by receivers fixed to the camera 100 and the user, and then calculating the distance from there.

[0131] According to the method of this modification, authentication continues between a series of photographs even if camera 100 does not have a configuration for measuring the position, thereby improving the convenience of the device.

[0132] [Eighth embodiment] Next, an eighth embodiment will be described. In the seventh embodiment, whether the user of the camera 100 is continuously using the camera 100 is determined based on the short distance between the camera 100 and an information device owned by the user. In this embodiment, whether the user of the camera 100 is continuously using the camera 100 is determined based on the short distance between the camera position at the time of user identification and the reference position. This method makes it possible to reduce the frequency of authentication when changes in the shooting position during a series of shooting are limited. As a result, the convenience of the device is improved. Note that the camera position is based on GPS location information, as in the seventh embodiment.

[0133] The functional configuration of the camera 100 in Fig. 14 is the same as that of the seventh embodiment. In this embodiment, an information acquisition unit 1401 in Fig. 14 acquires GPS position information of the camera 100 from the GPS receiving unit 308.

[0134] The continuous use determination unit 403 sets the position of the camera 100 acquired by the information acquisition unit 1401 when the user is identified by the user identification unit 402 as the reference position, and determines whether the user is continuously using the camera 100 based on this position and the current position of the camera 100.

[0135] The processing procedure by which camera 100 according to this embodiment determines whether a user is continuing to use the camera is the same as that shown in the flowchart of FIG. 5 of the first embodiment. At the start of this flow, the "predetermined distance" required for the determination by continued use determination unit 403 is assumed to have been recorded in memory unit 213 of camera 100. This flow starts when camera 100 is powered on, and is repeated until the processing can no longer be continued, for example, when camera 100 is powered off. The procedure for determining whether user P is continuing to use the camera in step S503 of FIG. 5 will be described below with reference to the flowchart of FIG. 18.

[0136] First, in step S1801, it is determined whether the position of camera 100 at the time when user identification unit 402 identified the user has been acquired. If it has been acquired, the process proceeds to step S1803; if it has not been acquired, the process proceeds to step S1802, where the position of camera 100 is acquired and set as the reference position. In step S1803, the current position of camera 100 is acquired.

[0137] In step S1804, it is determined whether the distance between the reference position and the current position is equal to or less than a predetermined distance. If it is equal to or less than the predetermined distance, the process proceeds to step S1805, where it is determined that the user P is "continuing use," and the flow ends. On the other hand, if it exceeds the predetermined distance, the process proceeds to step S1806, where it is determined that the user P is "not continuing use," and the flow ends.

[0138] According to the method of this embodiment, authentication continues during a series of photographs without measuring the positions of objects other than the device, thereby improving the convenience of the device.

[0139] [Modification 1 of the Eighth Embodiment] Next, a first modification of the eighth embodiment will be described. In the eighth embodiment, the position at the time of user identification was used as the reference position, but in this modification, the position at the start of image capture is used as the reference position. After the user identification unit 402 identifies the user, if there is time before the start of image capture, the user is allowed to continue using the device using a method such as that of the ninth embodiment described below.

[0140] 14 is the same as that of the seventh embodiment. The continuous use determination unit 403 sets the position of the camera 100 acquired by the information acquisition unit 1401 when the shooting unit 401 starts shooting as a reference position, and determines whether the user is continuously using the camera 100 based on this position and the current position of the camera 100.

[0141] The procedure for the process by which the camera 100 according to this embodiment determines whether the user is continuing to use the camera is the same as the flowchart of the first embodiment shown in Fig. 5. This flow starts when the camera 100 is turned on, and is repeated until the process can no longer be continued, for example, when the camera 100 is turned off. It is assumed that the "predetermined distance" required for the determination by the continued use determination unit 403 has already been recorded in the memory unit 213 of the camera 100 at the start of this flow.

[0142] The procedure for determining whether user P is continuing to use the device in step S503 of Fig. 5 can be represented by the flowchart of Fig. 18, similar to the second embodiment. The reference position in this flow is the position at the start of imaging.

[0143] According to the method of this modification, even if the camera 100 moves between the time the user is identified and the time photographing starts, authentication continues during the series of photographs, improving the convenience of the device.

[0144] [Modification 2 of the Eighth Embodiment] Next, a second modification of the eighth embodiment will be described. In the eighth embodiment and the first modification, the position at the time of identifying the user or at the start of shooting was set as the reference position, but in this modification, a position designated by the user on a map with the same coordinates as the coordinates that output the current position of camera 100 is set as the reference position. Note that the map used to designate the reference position may be a map possessed by camera 100, or a map possessed by an information device that can communicate with camera 100.

[0145] The functional configuration of camera 100 in Fig. 14 is the same as that of the seventh embodiment. Continuous use determination unit 403 has a reference position setting unit (not shown in Fig. 14), and determines whether the user is continuously using camera 100 based on this position and the current position of camera 100.

[0146] The processing procedure by which camera 100 according to this embodiment determines whether the user is continuing to use the camera is the same as that shown in the flowchart of Fig. 5 in the first embodiment. At the start of this flow, the reference position and the "predetermined distance" are assumed to have been recorded in memory unit 213 of camera 100. The procedure for determining whether user P is continuing to use the camera in step S503 of Fig. 5 will be described below with reference to the flowchart of Fig. 19.

[0147] First, in step S1901, the current position of camera 100 is obtained. Next, in step S1902, it is determined whether the reference position and the current position are within a predetermined distance. If they are within the predetermined distance, the process proceeds to step S1903, where it is determined that user P is "continuing use," and the flow ends. On the other hand, if the predetermined distance is exceeded, the process proceeds to step S1904, where it is determined that user P is "not continuing use," and the flow ends.

[0148] According to the method of this modification, when a user takes photographs at a specific location, authentication continues between successive photographs, improving the convenience of the device.

[0149] [Modification 3 of the Eighth Embodiment] Next, a third modification of the eighth embodiment will be described. In the second modification of the eighth embodiment, a position designated by the user on a map having the same coordinates as the coordinates at which the current position of the camera 100 is output is used as the reference position, and a reference area in which continuation determination is enabled when the position is within a predetermined range from the reference position is used. In this modification, an area designated by the user on the same map is used as the reference area. The reference area may be designated on a map possessed by the camera 100, or on a map possessed by an information device capable of communicating with the camera 100.

[0150] The functional configuration of camera 100 in Fig. 14 is the same as that of the seventh embodiment. Continuous use determination unit 403 has a reference area setting unit not shown in Fig. 14, and determines whether the user is continuously using camera 100 based on this area and the current position of camera 100.

[0151] The processing procedure by which camera 100 according to this embodiment determines whether a user is continuing to use the camera is the same as that shown in the flowchart of FIG. 5 of the first embodiment. At the start of this flow, it is assumed that the reference area used to determine whether a user is continuing to use the camera has already been recorded in memory unit 213 of camera 100. The procedure for determining whether user P is continuing to use the camera in step S503 of FIG. 5 can be represented by the flowchart of FIG. 19, similar to that of Modification 1 of the eighth embodiment. In this modification, in step S1902, it is determined whether the current position of camera 100 is within the reference area.

[0152] According to the method of this modification, when a user takes photographs within a specific area, authentication continues between successive photographs, improving the convenience of the device.

[0153] [Ninth embodiment] Next, a ninth embodiment will be described. In the seventh and eighth embodiments, continued use was determined based on the position information of the camera 100, but in this embodiment, the determination is made based on time information. The time when the user is identified is used as the reference time, and if a predetermined time has passed since that time, it is determined that the user is continuing to use the device. This method can reduce the frequency of authentication. As a result, the convenience of the device is improved.

[0154] 14 is the same as that of the seventh embodiment. In this embodiment, an information acquisition unit 1401 in FIG.

[0155] The continuous use determination unit 403 sets the time acquired by the information acquisition unit 1401 when the user is identified by the user identification unit 402 as the reference time, and determines whether the user is continuously using the camera 100 based on this time and the current time of the camera 100.

[0156] The processing procedure by which camera 100 according to this embodiment determines whether a user is continuing to use the camera is the same as that shown in the flowchart of FIG. 5 of the first embodiment. At the start of this flow, it is assumed that the "predetermined time" required for determination by continued use determination unit 403 has already been recorded in memory unit 213 of camera 100. This flow starts when camera 100 is powered on, and is repeated until processing can no longer be continued, for example, when camera 100 is powered off. The procedure for determining whether user P is continuing to use the camera in step S503 of FIG. 5 will be described below using the flowchart of FIG. 20.

[0157] First, in step S2001, it is determined whether the time of the camera 100 at the time when the user identification unit 402 identified the user has been acquired. If it has been acquired, the process proceeds to step S2003; if it has not been acquired, the process proceeds to step S2002, where the current time of the camera 100 is acquired and used as the reference time. In step S2003, the current time of the camera 100 is acquired.

[0158] In step S2004, it is determined whether the interval between the reference time and the current time is equal to or less than a predetermined time. If it is equal to or less than the predetermined time, the process proceeds to step S2005, where it is determined that user P is "continuing use," and the flow ends. On the other hand, if it exceeds the predetermined time, the process proceeds to step S2006, where it is determined that user P is "not continuing use," and the flow ends.

[0159] According to the method of this embodiment, when a series of photographs is taken within a limited time, authentication continues between the series of photographs, improving the convenience of the device.

[0160] [Modification 1 of the ninth embodiment] Next, a first modification of the ninth embodiment will be described. In the ninth embodiment, the time when the user is identified is used as the reference time, but in this modification, the time when image capture starts is used as the reference time. After the user is identified by the user identification unit 402, if there is time before image capture starts, the user is allowed to continue using the device using the method of the eighth embodiment described above, or the like.

[0161] 14 is the same as that of the seventh embodiment, and therefore description thereof will be omitted. In this embodiment, an information acquisition unit 1401 in FIG.

[0162] The continuous use determination unit 403 sets the time acquired by the information acquisition unit 1401 when the photographing unit 401 starts photographing as the reference time, and determines whether the user is continuously using the camera 100 based on this time and the current time of the camera 100.

[0163] The processing procedure by which camera 100 according to this embodiment determines whether a user is continuing to use the camera is the same as that shown in the flowchart of FIG. 5 of the first embodiment. At the start of this flow, the "predetermined time" required for determination by continued use determination unit 403 is assumed to have been recorded in memory unit 213 of camera 100. The procedure for determining whether user P is continuing to use the camera in step S503 of FIG. 5 can be expressed by the flowchart of FIG. 20, as in the third embodiment. The reference time in this flow is the time when image capture begins.

[0164] According to the method of this modification, even if the camera 100 moves between the time the user is identified and the time photographing starts, authentication continues during the series of photographs, improving the convenience of the device.

[0165] [Modification 2 of the ninth embodiment] Next, a second modification of the ninth embodiment will be described. In the ninth embodiment, continuous use is determined when a predetermined time has elapsed since the reference time, but in this embodiment, continuous use is determined when the end time specified by the user in the camera 100 is reached.

[0166] 14 is the same as that of the seventh embodiment. In this embodiment, an information acquisition unit 1401 in FIG.

[0167] 14, and determines whether the user is continuously using the camera 100 based on the reference time and the current time of the camera 100. Here, the reference time is the scheduled end time of shooting.

[0168] The processing procedure by which camera 100 according to this embodiment determines whether a user is continuing to use the camera is the same as that shown in the flowchart of Fig. 5 in the first embodiment. It is assumed that the end time has already been recorded in memory unit 213 of camera 100 at the start of this flow. The procedure for determining whether user P is continuing to use the camera in step S503 of Fig. 5 will be described below with reference to the flowchart of Fig. 21.

[0169] First, in step S2101, the current time of the camera 100 is obtained. Next, in step S2102, it is determined whether the current time is before the reference time (scheduled end time). If it is before the end time, the process proceeds to step S2103, where it is determined that the user P is "continuing use," and the flow ends. On the other hand, if the reference time (scheduled end time) has passed, the process proceeds to step S2104, where it is determined that the user P is "not continuing use," and the flow ends.

[0170] According to the method of this modification, when the time when the series of photographs will end is known, authentication continues between the series of photographs, improving the convenience of the device.

[0171] [Tenth embodiment] Next, a tenth embodiment will be described. In the seventh and eighth embodiments, continued use was determined based on the position information of the camera 100, and in the ninth embodiment, it was determined based on the time information of the camera 100, but in this embodiment, it is determined based on both the position and time information.

[0172] The functional configuration of the camera 100 in Fig. 14 is the same as that of the seventh embodiment. In this embodiment, an information acquisition unit 1401 in Fig. 14 acquires GPS position information of the camera 100 from the GPS receiving unit 308 and time information from the timer 309.

[0173] The continuous use determination unit 403 sets the position acquired by the information acquisition unit 1401 when the user is identified by the user identification unit 402 as the reference position. Furthermore, the time acquired by the information acquisition unit 1401 when the user is identified by the user identification unit 402 is set as the reference time. Based on the reference position and the current position of the camera 100, and the reference time and the current time of the camera 100, it is determined whether the user is using the camera 100 continuously.

[0174] The processing procedure by which camera 100 according to this embodiment determines whether a user is continuing to use the camera is the same as that shown in the flowchart of FIG. 5 of the first embodiment. At the start of this flow, it is assumed that the "predetermined distance" and "predetermined time" required for the determination by continued use determination unit 403 have already been recorded in memory unit 213 of camera 100. This flow starts when camera 100 is powered on, and is repeated until the processing can no longer be continued, for example, when camera 100 is powered off. The procedure for determining whether user P is continuing to use the camera in step S503 of FIG. 5 will be described below using the flowchart of FIG. 22.

[0175] First, in step S2201, it is determined whether the position of camera 100 at the time when user identification unit 402 identified the user has been acquired. If it has been acquired, the process proceeds to step S2203, and if it has not been acquired, the process proceeds to step S2202, where the position of camera 100 is acquired and used as the reference position.

[0176] In step S2203, the current position of the camera 100 is acquired. In step S2204, it is determined whether the reference position and the current position are within a predetermined distance. If they are within the predetermined distance, the process proceeds to step S2205. If they exceed the predetermined distance, the process proceeds to step S2210, where it is determined that user P is "not continuing use," and the flow ends.

[0177] In step S2205, it is determined whether the time of the camera 100 at the time when the user identification unit 402 identified the user has been acquired. If it has been acquired, the process proceeds to step S2207, and if it has not been acquired, the process proceeds to step S2206, where the time of the camera 100 is acquired and used as the reference time.

[0178] In step S2207, the current time of the camera 100 is obtained. In step S2208, it is determined whether the reference time and the current time are within a predetermined time. If they are within the predetermined time, the process proceeds to step S2209, where it is determined that the user P is "continuing use," and the flow ends. On the other hand, if the predetermined time has passed, the process proceeds to step S2210, where it is determined that the user P is "not continuing use," and the flow ends.

[0179] According to the method of this embodiment, even if it is not possible to foresee whether the position of the camera 100 or the shooting time is limited, authentication continues between successive shots, thereby improving the convenience of the device.

[0180] [Eleventh embodiment] In this embodiment, an example of determining continued use using a captured image will be described. Specifically, the user at the time of capture, i.e., the photographer, is identified using the image similarity. In this embodiment, image similarity is an index indicating whether the images themselves are similar, and is also influenced by the time of capture and the circumstances at the time of capture. There are two main reasons for using image similarity. One is that the subject intended by the photographer in a single capture is likely to be essentially the same regardless of whether authentication is possible. The other is that even if the subject is different, the capture settings and image composition when the same user captures similar scenes are likely to be similar.

[0181] An example of the functional configuration of camera 100 related to the processing described below is shown in the block diagram of Fig. 23. Camera 100 is composed of an imaging unit 401, an identification unit 402, a calculation unit 2301, a continued use determination unit 403, and a recording unit 404, which function by electrically controlling each unit of camera 100 by CPU 212.

[0182] If the user identification unit 402 fails to identify the user, the calculation unit 2301 calculates the similarity between the image captured by the photographing unit 401 and one or more images linked to photographer information among the images recorded by the recording unit 404. If the user identification unit 402 succeeds in identifying the user, the calculation unit 2301 calculates the similarity between the image captured by the photographing unit 401 and one or more images not linked to photographer information among the images recorded by the recording unit 404.

[0183] If any of the similarities calculated by the calculation unit 2301 exceeds a predetermined threshold, the continuous use determination unit 403 determines that the photographer of the two or more images used to calculate the similarity is the same.

[0184] The processing procedure performed by the camera 100 according to this embodiment from the start of shooting to the completion of linking of photographer information will be described below using the flowchart in Fig. 24. In step S2401, the shooting unit 401 acquires an image by converting an imaging signal from the image sensor 211 into video information.

[0185] In step S2402, the user identification unit 402 acquires feature quantities of the user's eyeball 220 captured by the eye imaging element 219. More specifically, feature extraction is performed on the eye image (eye image signal; electrical signal of the eye image) acquired from the eye imaging element 219 via the gaze detection circuit 301 to acquire the feature quantities. A neural network is used for the feature extraction. For example, a convolutional neural network (CNN), which is a type of neural network, is used. CNN extracts abstract information from an input image by repeatedly performing a process consisting of convolution, activation, and pooling on the input image. In this case, a processing unit consisting of convolution, activation, and pooling is often called a layer. There are several well-known activation processing methods used in this case, but a method called rectified linear unit (ReLU) may be used, for example. There are also several well-known pooling methods, but a method called max pooling may be used, for example. For example, the CNN structure may be ResNet, which is introduced in non-patent literature (K. He, X. Zhang, S. Ren, and J. Sun. Identity mappings in deep residual networks. In ECCV, 2016). Alternatively, a neural network known as VisionTransformer (ViT), which is described in non-patent literature (Alexey Dosovitskiy, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR, 2021), may be used. The neural network configuration is not limited to these. Information such as the structure and weights of the neural network is stored in the memory unit 213. The weights of this neural network are acquired in advance by learning.For example, various people's eye images are acquired in advance for training, and training is performed using a method such as ArcFace, which is shown in non-patent document (J. Deng, J. Guo, N. Xue, and S. Zafeiriou. Arcface: Additive angular margin loss for deep face recognition. In CVPR, 2019).

[0186] Although a method using a neural network is exemplified as a method for personal authentication using eye images, well-known methods such as iris authentication (for example, Patent 03307936) may also be used. Personal authentication methods are not limited to these.

[0187] In step S2403, the user identification unit 402 compares the feature acquired in step S2402 with the feature of each user, i.e., the registered user, stored in advance in the memory unit 213, to determine the similarity. In this embodiment, the feature is normalized and the similarity (i.e., cosine similarity) is calculated by taking the inner product, but other methods may be used. For example, a neural network trained to calculate similarity using the feature as input, pattern matching, frequency analysis, Euclidean distance, or Manhattan distance may be used, and is not limited to a specific form. In this step, similarity is not calculated if acquisition of the feature failed in step S2402.

[0188] In step S2404, user identification unit 402 determines whether any of the similarities calculated in step S2403 exceeds a predetermined threshold value. If any of the similarities exceeds a predetermined threshold value, the process proceeds to step S2405; otherwise, the process proceeds to step S2411.

[0189] In step S2405, the continuous use determination unit 403 determines that the user who captured the image acquired by the image capturing unit 401 in step S2401 is the same as the registered person corresponding to the highest similarity.

[0190] In step S2406, the recording unit 404 checks whether or not there are any images not yet linked to photographer information among the images recorded in the memory 213. If there are any, the process proceeds to step S2407; if not, the process proceeds to step S2410.

[0191] In step S2407, the calculation unit 2301 calculates the similarity between the image acquired by the photographing unit 401 in step S2401 and one or more images previously acquired by the photographing unit 401 that are not linked to photographer information and that were confirmed by the recording unit 404 in step S2406. Here, features are extracted from each image, and the similarity is calculated using the same method as in step S2403. A known neural network is used to extract features from each image, and feature values ​​related to the subject itself are acquired. These feature values ​​are assumed to be close values ​​if the subjects themselves are similar, and distant values ​​if they are not. Note that, although feature values ​​related to the subject itself are acquired here, feature values ​​related to the subject's orientation, position, movement, composition, and background information may also be acquired. Alternatively, these feature values ​​may be used in combination. Furthermore, although a method using a neural network is exemplified as a feature extraction method here, known feature extraction methods such as HoG and SIFT may also be used. While the method used here for calculating image similarity is a method using feature extraction and inner products using a neural network, other methods, such as squared error of pixel values ​​or Kullback-Leibler divergence of pixel value histograms, may also be used. Additionally, the calculated similarity may be adjusted using metadata added when the images were captured, such as the shooting time, shooting location, shutter speed, ISO sensitivity, aperture value, focal length, and lens specifications (e.g., lowering the similarity when there is a large difference in shooting time compared to when there is not). Alternatively, the similarity may be calculated solely from this metadata (e.g., if the shooting time difference is 1 second, the similarity is 0.99, and if it is 10 seconds, the similarity is 0.8). The similarity can be calculated using the function f shown in Equation 1 below, taking into account information obtained from the images and the shooting conditions, so that it is inversely proportional to the image itself and the amount of deviation in the metadata.

[0192] (Equation 1) (Image) similarity = f(I1,I2)

[0193] Here, I1 is information obtained from the image with an unidentified photographer (information that can be extracted from pixels (information obtained by image recognition (the subject itself, the subject's orientation, the subject's movement, composition, background information, and their features), pixel values, histograms), and metadata assigned to the image (shooting time, shooting location, shutter speed, ISO sensitivity, aperture value, focal length, lens specifications)), and I2 is information obtained from N (N≧1) images with an identified photographer. The function f may be obtained by manually weighting each explanatory variable, or may be obtained by learning using a known machine learning method such as a neural network or random forest. In Equation 1, the similarity is calculated using one function, but a combination of multiple similar functions may also be used. The similarity may be in any format or calculation method as long as it can determine whether two images were taken by the same user. This embodiment does not limit the methods of feature extraction and image similarity calculation.

[0194] In step S2409, the continued use determination unit 403 determines that the user who captured all of the images whose similarity was determined to exceed the threshold in step S508 is the same user who captured the images acquired by the imaging unit 401 in step S2401.

[0195] In step S2410, the recording unit 407 links the photographer information to the image for which the user was newly identified in step S2405, step S2409, or step S2414 in a format that cannot be tampered with, and then ends the process. Methods for linking in a format that cannot be tampered with are described in non-patent documents (Coalition for Content Provenance and Authenticity (C2PA), "C2PA Specifications" and<Technical Specifications Version 1.2> , [online], November 3, 2022, [Retrieved January 23, 2023], Internet<URL:https: / / c2pa.org / specifications / specifications / 1.2 / specs / C2PA_Specification.html> ) is used. C2PA is a method of adding metadata to an image that indicates the editing details that have been performed on the image in order to authenticate the origin, history, and provenance of the image. Therefore, photographer information may be recorded in accordance with C2PA. However, other methods of recording may also be used. Alternatively, the image and photographer information may be recorded in separate files. Alternatively, photographer information may be managed in a database. The method of recording photographer information is not limited to these.

[0196] In step S2411, the recording unit 404 checks whether any images recorded in the memory 213 are already associated with photographer information. If any images are already associated with photographer information, the process proceeds to step S2412; if not, the process ends (the images are associated with the fact that the photographer is unidentified, as necessary).

[0197] In step S2412, the calculation unit 2301 calculates the similarity between the image acquired by the photographing unit 401 in step S2401 and the N (N≧1) images with photographer information linked that were confirmed by the recording unit 404 in step S2411. The similarity calculation is performed using the method described in step S2407.

[0198] In step S2413, the continuous use determination unit 403 determines whether any of the similarities calculated by the calculation unit 2301 in step S2412 exceeds a predetermined threshold value. If any of the similarities exceeds a predetermined threshold value, the process proceeds to step S2414; if not, the process ends (the image is associated with the fact that the photographer is unidentified, as necessary).

[0199] In step S2414, the continued use determination unit 403 determines that the photographer of the image corresponding to the highest similarity among those determined to have a similarity greater than the threshold in step S2413 is the same as the photographer of the image acquired by the photographing unit 401 in step S2401.

[0200] As a result, even if information suitable for photographer authentication, such as an image of the eyes looking through the viewfinder, cannot be obtained because the user's attention is focused on taking the photo rather than on authentication when taking the photo (for example, when a perfect photo opportunity suddenly comes up when photographing sports or wild birds), it is still possible to identify the photographer, i.e., to link the photographer's information to the captured image, thereby improving the convenience of the device for authenticating and recording the photographer.

[0201] [Modification 1 of the 11th embodiment] In the above description, the photographing unit 401 acquires an image by converting an image signal from the image sensor 211 into video information. However, a video may be acquired instead of an image. In this case, the calculation unit 2301 may calculate the similarity using a specific frame extracted from the video or all frames of the video, and may take the average of these as the final similarity.

[0202] [Modification 2 of the eleventh embodiment] In the above description, the recording unit 404 links, in a format that prevents tampering, photographer information to an image that has been determined by the continuous use determination unit 403 to have been taken by the same photographer as another image. However, photographer information may also be linked to images other than this one. Specifically, images taken between the capture of two images determined to have been taken by the same photographer may also be determined to have been taken by the same photographer, and the same photographer information may be linked to them.

[0203] [Modification 3 of the eleventh embodiment] In the above description, the user identification unit 402 acquires features of the user's eyeball 220. However, features may be acquired from sources other than the eyeball 220. For example, a separate camera may be installed on the camera 100 (e.g., above the operation member 103) and features may be extracted from a facial image of the user acquired through the camera. Alternatively, features may be extracted from an iris image extracted from an image of the user's eye. Alternatively, a microphone may be installed on the camera 100 (e.g., above the operation member 103) and features may be extracted from voiceprint information of the user acquired through the microphone. Alternatively, a fingerprint sensor may be installed on the camera 100 (e.g., on the release button 101 or operation member 103) and features may be extracted from fingerprint information of the user acquired through the microphone. Alternatively, a vein sensor may be installed on the camera 100 (e.g., above the release button 101) and features may be extracted from vein information of the user acquired through the microphone. Alternatively, features may be extracted from a video showing the movement of the user's eyeball 220, captured in time series by the eye imaging element 219. Alternatively, the feature amount may be extracted from operation information specific to the user obtained through the operation members 103, 104, and 105.

[0204] [Modification 4 of the eleventh embodiment] In the above description, the user identification unit 402 calculates similarity using feature amounts and identifies the user. However, the user may be identified without using feature amounts or similarity. For example, the user may be identified by having the user enter a password or PIN code. Alternatively, the smartphone and camera 100 may be configured to be paired via Bluetooth connection or the like, and the results of personal authentication performed on the smartphone may be shared with the camera 100 upon Bluetooth connection. The user may also be identified using a mechanism such as FIDO authentication.

[0205] [Modification 5 of the 11th embodiment] In the above, the user identification unit 402 identifies the user when capturing an image. However, the user may also be identified in two stages: outside of capture and during capture. The advantage of performing identification in two stages is that by identifying in advance from among multiple registrants a unique registrant who will likely be photographed under strict conditions outside of capture, it becomes possible to determine whether the unique registrant and the user are the same under relatively relaxed conditions during capture, thereby improving the accuracy of identification during capture. Furthermore, since there is only one registrant to compare during capture, it is possible to shorten the processing time (because the processing time is composed of the total of biometric authentication and image similarity calculation).

[0206] [Modification 6 of the eleventh embodiment] In the above, there is no way for the user to know at the time of shooting that the recording unit 404 has linked the photographer information to the image. In response to this, the camera 100 may be provided with a configuration that notifies the user that the photographer information has been linked to the image at the time of shooting. Specifically, a display such as that shown in FIG. 25 may be displayed on the display panel of the eyepiece 102. FIGS. 25(a), (b), and (c) respectively show: FIG. 25(a): Example of display when the photographer of the current image is identified by the eye image, but the photographer of the previous image is not identified by the image similarity. Fig. 25(b): Example of display when the photographer of the current image is identified by the eye image, and the photographer of the past image is also identified by the image similarity. Figure 25(c): Example of display when the photographer of the current image is identified based on the image similarity. Applying these to the above processing flow, FIG. 25(a): Step S2404 is Yes, but Step S2406 or Step S2408 is No FIG. 25(b): Step S2404, Step S2406, and Step S2408 all say "Yes." FIG. 25(c): Step S2404 is No, but Steps S2411 and S2413 are Yes This allows the user to know that photographer information has been linked to the image at the time of shooting, thereby improving the efficiency of shooting. However, this is merely an example, and other methods may be used to notify the user. For example, a display equivalent to that shown in FIG. 25 may be displayed on the operation member 103. Alternatively, a speaker may be installed in a suitable position on the camera 100 (for example, above the operation member 103), and a sound corresponding to the linking status may be emitted from the speaker when the linking of photographer information is completed. Alternatively, the camera 100 may be configured to be paired with an external device such as a smartphone or smartwatch via Bluetooth connection, and the external device may issue a notification corresponding to the linking status when the linking is completed.

[0207] [Twelfth embodiment] In the eleventh embodiment, a method for identifying a user using the similarity of images during image capture was described. However, there are cases where it is desirable to identify a user using the similarity of images even after image capture has been completed. For example, this may occur when it is necessary to move on to another capture immediately after image capture, or when images recorded in the memory unit 213 are transferred to the memory unit of an external information processing device via portable media such as an SD card after capture. In this embodiment, a method for identifying a user using the similarity of images in such cases will be described.

[0208] An example of the functional configuration of camera 100 (or another information processing device) related to the processing described below is shown in the block diagram of Fig. 26. Camera 100 is composed of image selection unit 2601, calculation unit 2301, continued use determination unit 403, and recording unit 404, which function by electrically controlling each unit of camera 100 by CPU 212.

[0209] The image selection unit 2601 selects images that have been linked to photographer information and images that have not been linked to photographer information from among the images recorded in the memory 212. The calculation unit 2301 calculates the similarity between the images that have been selected by the image selection unit 2601 and the images that have not been linked to photographer information.

[0210] The processing procedure performed by the camera 100 (or another information processing device) according to this embodiment until the linking of photographer information is completed will be described below with reference to the flowchart of FIG.

[0211] In step S2701, the image selection unit 2601 checks whether or not there are any images not linked to photographer information among the images recorded in the memory 212. If there are any images, the process proceeds to step S2702; if not, the process ends.

[0212] In step S2702, the image selection unit 2601 checks whether or not there is an image to which photographer information has been linked among the images recorded in the memory 212. If there is, the process proceeds to step S2703; if not, the process ends.

[0213] In step S2703, the image selection unit 2601 selects M images (M≧1) for which the photographer is to be identified from among the images not linked to photographer information confirmed in step S2701, based on the input information obtained from the operation members 103, 104, and 105.

[0214] In step S2704, the image selection unit 2601 selects N images (N≧1) taken by the same photographer from among the images linked to photographer information confirmed in step S2702, based on the input information obtained from the operation members 103, 104, and 105.

[0215] In step S2705, the calculation unit 2301 calculates the similarity between the M images selected by the image selection unit 2601 in step S2703 and the N images selected by the image selection unit 2601 in step S2704. To calculate the similarity, the method described in step S2407 is applied to the M images in order (repeated M times).

[0216] In step S2706, the continuous use determination unit 403 determines whether any of the M similarities calculated by the calculation unit 2301 in step S2705 exceeds a predetermined threshold value. If any of the similarities exceeds a predetermined threshold value, the process proceeds to step S2707; otherwise, the process ends.

[0217] In step S2707, the continuous use determination unit 403 determines that the photographer of all images not linked to photographer information corresponding to X (X≦M) similarities determined to exceed the threshold in step S2706 is the same as the photographer of the N images.

[0218] In step S2708, the recording unit 404 links the photographer information to the image for which the photographer was newly identified in step S2707 in a format that cannot be tampered with, and ends the process.

[0219] As a result, even when not taking a picture, it becomes possible to link an image with photographer information (photographer authentication), improving the convenience of a device that authenticates and records the photographer.

[0220] The numerical values, processing timing, processing order, processing subject, data (information) configuration / acquisition method / sending destination / sending source / storage location, etc. used in the above embodiments and variant examples are given as examples to provide a concrete explanation, and are not intended to be limited to these examples.

[0221] Furthermore, some or all of the embodiments and modifications described above may be used in appropriate combination, and some or all of the embodiments and modifications described above may be used selectively.

[0222] (Other embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0223] The invention of this specification includes the following image capturing apparatus and image capturing apparatus control method. (Item 1) An imaging device, an identification means for identifying a user of the photographing device; a determination means for determining a continuation state of use of the photographing device by the user; a control means for controlling the recording of images captured by the photographing device and information about the user in a memory in association with each other during the period from when the identification means identifies the user until when the determination means determines that the user has not continued to use the photographing device; and An imaging device comprising: (Item 2) 2. The photographing device according to item 1, wherein the memory is built into the photographing device or is physically detachably connected to the photographing device. (Item 3) the imaging device is connected to the server having the memory via a communication path; The photographing device described in item 1, characterized in that the control means sends a signal to the server depending on the result of the judgment so that the image and the user information are associated and recorded in the memory. (Item 4) 4. The photographing device according to any one of items 1 to 3, wherein the identification unit identifies a user of the photographing device based on biometric information of the user of the photographing device. (Item 5) 5. The photographing device according to item 4, wherein the biometric information includes at least one of iris, fingerprint, vein, and face. (Item 6) The photographing device described in any one of items 1 to 4, characterized in that the identification means identifies the user of the photographing device based on authentication information input by the user of the photographing device or read from a recording medium. (Item 7) The photographing device described in any one of items 1 to 6, characterized in that the determination means determines that the user is continuing to use the photographing device from the time the identification means identifies the user until an instruction to end the photographing device is given. (Item 8) 8. The photographing device according to item 7, wherein the instructions include an instruction to press an end button and an instruction to switch modes. (Item 9) The photographing device described in any one of items 1 to 6, characterized in that the determination means determines that the user is continuing to use the photographing device when an instruction to continue using the photographing device is being input. (Item 10) 10. The photographing device according to item 9, wherein the determination means determines that the user is no longer continuing to use the photographing device when an instruction to continue using the photographing device is no longer input in the state. (Item 11) The photographing device described in any one of items 1 to 6, characterized in that the determination means determines that the user is not continuing to use the photographing device when it detects an action that is presumed to be an end of photographing in the photographing device. (Item 12) The photographing device described in item 1 or 11, characterized in that the determination means determines that the user is not continuing to use the photographing device when it detects that a photographing lens unit is not mounted on the housing of the photographing device. (Item 13) The photographing device described in item 1 or 11, characterized in that the determination means determines that the user is not continuing to use the photographing device if the input brightness to the image sensor of the photographing device is below a threshold value. (Item 14) 12. The photographing device according to item 1 or 11, wherein the determination means determines that the user is no longer using the photographing device when communication between the photographing device and the external device is disconnected. (Item 15) The photographing device described in any one of items 1 to 6, characterized in that the determination means determines that the user is not continuing to use the photographing device when the number of photographs taken after the user has been identified reaches a threshold value. (Item 16) The photographing device described in any one of items 1 to 6, characterized in that the determination means determines that the user is continuing to use the photographing device when it determines that the user is holding the grip portion of the photographing device. (Item 17) The photographing device described in any one of items 1 to 6, characterized in that when the determination means determines that the posture of the photographing device is changing, it determines that the user is continuing to use the photographing device. (Item 18) The photographing device described in any one of items 1 to 6, characterized in that the determination means determines that the user is continuing to use the photographing device when it determines that an appliance is screwed to the photographing device. (Item 19) The photographing device described in any one of items 1 to 6, characterized in that the determination means determines that the user is continuing to use the photographing device when it detects movement of the photographing device. (Item 20) The photographing device described in any one of items 1 to 6, characterized in that the determination means determines that the user is continuing to use the photographing device if the location of the photographing device is within a specific geographical range. (Item 21) 21. The photographing device according to any one of items 1 to 20, characterized in that, if the identification means is unable to identify the user of the photographing device, the control means controls the image photographed by the photographing device and information indicating that the user of the photographing device has not been identified to be recorded in memory in association with each other. (Item 22) moreover, 22. The photographing device according to any one of items 1 to 21, further comprising a display control means for displaying a screen for setting a method for determining the continuation state. (Item 23) Further, a motion acquisition means for acquiring motion information of the image capturing device is provided, 2. The photographing device according to item 1, wherein the determining means determines a continuation state of use of the photographing device by the user based on the movement information acquired by the acquiring means. (Item 24) Item 24. The photographing device according to item 23, wherein the movement acquisition means acquires movement information based on output values ​​of an acceleration sensor and / or an angular velocity sensor. (Item 25) Item 24. The photographing device described in item 23, characterized in that the determination means determines a period during which the photographing device is continuously in operation, including the time when the identification means identifies the user, as a period during which the photographing device is continuously in use by the user. (Item 26) Item 24. The photographing device described in item 23, characterized in that the determination means determines a period of time during which the photographing device is continuously stationary, including the time when the identification means identifies the user, as a period of time during which the photographing device is continuously used by the user. (Item 27) Item 24. The photographing device described in item 23, characterized in that the determination means determines a period of time during which the photographing device is continuously used by the user to be a period including the time when the identification means identifies the user, and the period of time during which the photographing device is moving in the same manner as the time when the identification means identifies the user. (Item 28) Item 24. The photographing device according to item 23, wherein the determination means detects a specific movement pattern from the movement information and determines that the user has not used the device continuously before and after the pattern. (Item 29) The photographing device described in item 1 further comprises an acquisition means for acquiring either or both of information indicating the position of the photographing device and information indicating the time, and the determination means determines whether the user is continuously using the photographing device based on the information acquired by the acquisition means. (Item 30) 30. The photographing device according to item 29, further comprising a position setting means for setting a reference position, wherein the determination means determines whether the user is continuously using the photographing device based on the distance between the position of the photographing device acquired by the acquisition means and the reference position set by the position setting means. (Item 31) Item 31. The photographing device according to item 30, wherein the reference position is the position of the photographing device at the time when the identification means identifies the user of the photographing device, or the position of the photographing device at the time when the photographing device starts photographing after the identification means identifies the user of the photographing device, or the position of the user of the photographing device. (Item 32) Item 30. The photographing device according to item 29, further comprising an area setting means for setting the placement of a reference area, wherein the determination means determines that the user is continuously using the photographing device if the position of the photographing device acquired by the acquisition means is within the area set by the area setting means. (Item 33) The photographing device described in item 1 further comprises an acquisition means for acquiring a distance between the photographing device and the user, and the determination means determines that the user is continuously using the photographing device based on the distance acquired by the acquisition means. (Item 34) Item 30. The photographing device described in item 29 further comprises a time setting means for setting a reference time, and the determination means determines whether the user is continuously using the photographing device based on the difference between the time acquired by the acquisition means and the time set by the time setting means. (Item 35) Item 35. The photographing device according to item 34, wherein the reference time set by the time setting means is the time when the identification means identifies the user of the photographing device, or the time when the photographing device starts photographing after the identification means identifies the user of the photographing device, or the time when the identification means expects the user of the photographing device to end photographing. (Item 36) The photographing device described in item 1 further comprises an image similarity calculation means for calculating the similarity between information obtained from the images, and the determination means, when the identification means is unable to identify the user, determines the continued state of use of the photographing device by the user based on the results of the image similarity calculation means for images not linked to photographer information and images linked to photographer information. (Item 37) Item 37. The photographing device according to item 36, wherein the image similarity calculation means uses, as information obtained from the image, information that can be extracted from pixels, metadata assigned to the image, or a combination of these. (Item 38) Item 38. The photographing device according to item 37, wherein the image similarity calculation means uses, as information that can be extracted from the pixels, information obtained by image recognition, pixel values, histograms, or a combination thereof. (Item 39) The photographing device described in item 37 or 38, characterized in that the image similarity calculation means uses one or a combination of the shooting time, shooting location, shutter speed, ISO sensitivity, aperture value, focal length, and lens specifications as metadata to be assigned to the image. (Item 40) Item 39. The photographing device according to item 38, wherein the image similarity calculation means uses, as the information obtained by the image recognition, one or a combination of the subject itself, the orientation of the subject, the positioning of the subject, the movement of the subject, the composition, and background information. (Item 41) Item 37. The photographing device according to item 36, wherein the image similarity calculation means calculates the similarity between information obtained from some or all frames of a moving image photographed by the photographing device. (Item 42) The photographing device according to item 1 further comprises a notification unit that notifies a user when photographer information is linked to an image by the control unit. (Item 43) An information processing device comprising: a recording means for recording images; a selection means for selecting at least two images from the images recorded by said recording means; an image similarity calculation means for calculating the similarity between information obtained from the images selected by said selection means; a determination means for determining whether the images are taken by the same photographer based on the result of said image similarity calculation means; and a control means for linking photographer information to images to which photographer information has not been linked based on the result of said determination means. (Item 44) A control method for an imaging device, comprising: an identification step in which an identification means of the photographing device identifies a user of the photographing device; a determination step in which a determination means of the photographing device determines a continuation state of use of the photographing device by the user; a control step in which a control means of the photographing device controls the photographing device so that images photographed by the photographing device and information about the user are associated with each other and recorded in a memory during the period from when the user is identified in the identification step until when it is determined in the determination step that the user is no longer using the photographing device; A control method for an imaging device comprising: (Item 45) A control method for a photographing device that photographs an image, comprising: an identification step of identifying a user of the photographing device; an acquisition step of acquiring either information indicating the position of the photographing device or information indicating the time, or both; a determination step of determining whether the user is continuously using the photographing device based on the information acquired in the acquisition step; and a recording step of recording a photographed image in association with information about the user based on the determination result of the determination step.

[0224] The invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0225] 401: Photographing unit 402: User identification unit 403: Continued use determination unit 404: Recording unit

Claims

1. An imaging device, an identification means for identifying a user of the photographing device; a determination means for determining a continuation state of use of the photographing device by the user; a control means for controlling the recording of images captured by the photographing device and information about the user in a memory in association with each other during the period from when the identification means identifies the user until when the determination means determines that the user has not continued to use the photographing device; and An imaging device comprising:

2. 2. The photographing device according to claim 1, wherein the memory is built into the photographing device or is physically detachably connected to the photographing device.

3. the imaging device is connected to the server having the memory via a communication path; 2. The photographing device according to claim 1, wherein the control means transmits a signal to the server in accordance with the result of the determination so that the image and the user information are recorded in association with each other in the memory.

4. 2. The photographing device according to claim 1, wherein the identifying unit identifies the user of the photographing device based on biometric information of the user of the photographing device.

5. 5. The image capturing apparatus according to claim 4, wherein the biometric information includes at least one of an iris, a fingerprint, veins, and a face.

6. 2. The photographing device according to claim 1, wherein the identification unit identifies the user of the photographing device based on authentication information input by the user of the photographing device or read from a recording medium.

7. 2. The photographing device according to claim 1, wherein the determination means determines that the user is continuing to use the photographing device from the time the identification means identifies the user until an instruction to end the photographing device is given.

8. 8. The photographing apparatus according to claim 7, wherein the instruction includes an instruction to press an end button and an instruction to switch modes.

9. 2. The photographing device according to claim 1, wherein the determining means determines that the user is continuing to use the photographing device when an instruction to continue using the photographing device is being input.

10. 10. The photographing device according to claim 9, wherein the determining means determines that the user is no longer continuing to use the photographing device when an instruction to continue using the photographing device is no longer input in the state.

11. 2. The photographing device according to claim 1, wherein the determination means determines that the user is not continuing to use the photographing device when it detects an action that is presumed to be an end of photographing in the photographing device.

12. 2. The photographing device according to claim 1, wherein the determination means determines that the user is not continuing to use the photographing device when it detects that a photographing lens unit is not mounted on the housing of the photographing device.

13. 2. The photographing device according to claim 1, wherein the determining means determines that the user is not continuing to use the photographing device when the input brightness to the image sensor of the photographing device is equal to or lower than a threshold value.

14. 2. The photographing device according to claim 1, wherein the determining means determines that the user is no longer using the photographing device when communication between the photographing device and the external device is disconnected.

15. 2. The photographing device according to claim 1, wherein the determining means determines that the user is no longer using the photographing device when the number of photographs taken after the user is identified reaches a threshold value.

16. 2. The photographing device according to claim 1, wherein the determining means determines that the user is continuing to use the photographing device when it determines that the user is gripping a grip portion of the photographing device.

17. 2. The photographing device according to claim 1, wherein the determining means determines that the user is continuing to use the photographing device when it determines that the attitude of the photographing device is changing.

18. 2. The photographing device according to claim 1, wherein the determining means determines that the user is continuing to use the photographing device when it determines that a device is screwed to the photographing device.

19. 2. The photographing device according to claim 1, wherein the determining means determines that the user is continuing to use the photographing device when the determining means detects a movement of the photographing device.

20. 2. The photographing device according to claim 1, wherein the determining means determines that the user is continuing to use the photographing device when the position of the photographing device is within a specific geographical range.

21. The photographing device according to claim 1, characterized in that, if the identification means is unable to identify the user of the photographing device, the control means controls the image photographed by the photographing device and information indicating that the user of the photographing device has not been identified to be recorded in memory in association with each other.

22. moreover, 2. The photographing apparatus according to claim 1, further comprising a display control means for displaying a screen for setting a method for determining the continuation state.

23. Further, a motion acquisition means for acquiring motion information of the imaging device is provided, 2. The photographing device according to claim 1, wherein the determining means determines a continuation state of use of the photographing device by the user based on the movement information acquired by the acquiring means.

24. 24. The photographing apparatus according to claim 23, wherein the movement acquisition means acquires the movement information based on an output value of an acceleration sensor and / or an angular velocity sensor.

25. The photographing device according to claim 23, wherein the determination means determines a period of time during which the photographing device is continuously in operation, including a time point at which the identification means identifies the user, as a period of time during which the photographing device is continuously in use by the user.

26. The photographing device according to claim 23, wherein the determination means determines a period of time during which the photographing device is continuously in use by the user, including a time point at which the identification means identifies the user, and a period during which the photographing device is continuously stationary.

27. The photographing device according to claim 23, characterized in that the determination means determines a period of continuous use by the user to be a period including the time when the identification means identifies the user, and the period of continuous use by the user is a period of the photographing device in which the same type of movement as the movement of the photographing device at the time when the identification means identifies the user.

28. 24. The photographing device according to claim 23, wherein the determining means detects a specific movement pattern from the movement information and determines that the user has not used the device continuously before and after the pattern.

29. The photographing device according to claim 1, further comprising an acquisition means for acquiring either or both of information indicating the location of the photographing device and information indicating the time, and the determination means determines whether the user is continuously using the photographing device based on the information acquired by the acquisition means.

30. 30. The photographing device according to claim 29, further comprising a position setting means for setting a reference position, wherein the determination means determines whether the user is continuously using the photographing device based on the distance between the position of the photographing device acquired by the acquisition means and the reference position set by the position setting means.

31. The photographing device according to claim 30, wherein the reference position is the position of the photographing device at the time when the identification means identifies the user of the photographing device, or the position of the photographing device at the time when the photographing device starts photographing after the identification means identifies the user of the photographing device, or the position of the user of the photographing device.

32. 30. The photographing device according to claim 29, further comprising an area setting means for setting the placement of a reference area, wherein the determination means determines that the user is continuously using the photographing device if the position of the photographing device acquired by the acquisition means is within the area set by the area setting means.

33. The photographing device according to claim 1, further comprising an acquisition means for acquiring a distance between the photographing device and the user, and the determination means for determining that the user is continuously using the photographing device based on the distance acquired by the acquisition means.

34. 30. The photographing device according to claim 29, further comprising a time setting means for setting a reference time, wherein the determination means determines whether the user is continuously using the photographing device based on the difference between the time acquired by the acquisition means and the time set by the time setting means.

35. 35. The photographing device according to claim 34, wherein the reference time set by the time setting means is the time when the identification means identifies the user of the photographing device, or the time when the photographing device starts photographing after the identification means identifies the user of the photographing device, or the time when the identification means expects the user of the photographing device to end photographing.

36. The photographing device of claim 1 further comprises an image similarity calculation means for calculating the similarity between information obtained from the images, and the determination means, when the identification means is unable to identify the user, determines the continued status of the user's use of the photographing device based on the results of the image similarity calculation means for images not linked to photographer information and images linked to photographer information.

37. 37. The photographing apparatus according to claim 36, wherein said image similarity calculation means uses, as information obtained from said image, information that can be extracted from pixels, metadata attached to the image, or a combination of these.

38. 38. The photographing apparatus according to claim 37, wherein said image similarity calculation means uses, as the information that can be extracted from said pixels, information obtained by image recognition, pixel values, histograms, or a combination thereof.

39. The photographing device according to claim 37, wherein the image similarity calculation means uses, as metadata to be assigned to the image, one or a combination of the following: shooting time, shooting location, shutter speed, ISO sensitivity, aperture value, focal length, and lens specifications.

40. 39. The photographing device according to claim 38, wherein the image similarity calculation means uses, as the information obtained by the image recognition, any one or a combination of the subject itself, the orientation of the subject, the positioning of the subject, the movement of the subject, the composition, and background information.

41. 37. The image capturing device according to claim 36, wherein said image similarity calculation means calculates the similarity between information obtained from some or all frames of a moving image captured by said image capturing device.

42. 2. The photographing apparatus according to claim 1, further comprising a notification unit that notifies a user when the photographer information is linked to the image by the control unit.

43. An information processing device comprising: a recording means for recording images; a selection means for selecting at least two images from the images recorded by said recording means; an image similarity calculation means for calculating the similarity between information obtained from the images selected by said selection means; a determination means for determining whether the images are taken by the same photographer based on the result of said image similarity calculation means; and a control means for linking photographer information to images to which photographer information has not been linked based on the result of said determination means.

44. A method for controlling an imaging device, comprising: an identification step in which an identification means of the photographing device identifies a user of the photographing device; a determination step in which a determination means of the photographing device determines a continuation state of use of the photographing device by the user; a control step in which a control means of the photographing device controls the photographing device so that images photographed by the photographing device and information about the user are associated with each other and recorded in a memory during the period from when the user is identified in the identification step until when it is determined in the determination step that the user is no longer using the photographing device; A control method for an imaging device comprising:

45. A control method for a photographing device that photographs an image, comprising: an identification step of identifying a user of the photographing device; an acquisition step of acquiring either information indicating the position of the photographing device or information indicating the time, or both; a determination step of determining whether the user is continuously using the photographing device based on the information acquired in the acquisition step; and a recording step of recording a photographed image in association with information about the user based on the determination result of the determination step.

Images