Imaging device, information processing device, method for controlling imaging device, computer-readable storage medium, and computer program product

Abstract

The invention relates to an imaging apparatus, an information processing apparatus, a control method of the imaging apparatus, a computer-readable storage medium, and a computer program product. The image pickup apparatus includes: a recognition unit configured to recognize a user of the image pickup apparatus; a determination unit configured to determine a state in which the user continues to use the imaging apparatus; and a control unit configured to perform control, and a recording unit configured to record an image captured by the imaging device in a memory in association with information of the user between a time when the recognition unit recognizes the user and a time when the determination unit determines that the use of the imaging device by the user is not being continued.

Description

TECHNICAL FIELD

[0001] The present application relates to an imaging device, an information processing device, a control method for an imaging device, a computer-readable storage medium, and a computer program product, and particularly to authentication and recording of a photographer. BACKGROUND

[0002] In recent years, there is a demand for a technology for identifying a person who has taken an image or a video and recording the person for the purpose of managing copyright or ensuring authenticity. Japanese Patent Application Publication No. 2018-191194 proposes a method for monitoring biometric information of a photographer when identifying the photographer and recording information of the photographer in an image file, and in a case where there is a change in the monitoring result, judging that the photographer has changed and canceling authentication of the photographer.

[0003] According to the method proposed in Japanese Patent Application Publication No. 2018-191194, since authentication is canceled during taking a series of images, convenience of imaging can decrease. SUMMARY

[0004] The present application provides a technology for improving convenience of an apparatus for authenticating and recording a photographer.

[0005] According to a first aspect of the present disclosure, there is provided an imaging device including an identification unit configured to identify a user of the imaging device, a judgment unit configured to judge a state of continued use of the imaging device by the user, and a control unit configured to control so that images taken by the imaging device are recorded in association with information of the user in a storage between a time when the identification unit identifies the user and a time when the judgment unit judges that use of the imaging device by the user is not being continued.

[0006] According to a second aspect of the present disclosure, there is provided an information processing device including a recording unit configured to record images, a selection unit configured to select at least two images from the images recorded by the recording unit, an image similarity calculation unit configured to calculate a similarity between information obtained from the images selected by the selection unit, a judgment unit configured to judge whether photographers of the images are the same as each other based on a result of calculation by the image similarity calculation unit, and a control unit configured to associate photographer information with an image not associated with the photographer information based on a result of judgment by the judgment unit.

[0007] According to a third aspect of the present disclosure, there is provided a control method of an imaging device, the control method including: identifying a user of the imaging device; determining a state of continued use of the imaging device by the user; and controlling so that an image captured by the imaging device is recorded in association with information of the user in a memory between a time when the user is identified in the identifying and a time when it is determined in the determining that the use of the imaging device by the user is not being continued.

[0008] According to a fourth aspect of the present disclosure, there is provided a control method of an imaging device configured to capture an image, the control method including: identifying a user of the imaging device; obtaining either or both of information indicating a position of the imaging device and information indicating a time; determining whether the user is continuing to use the imaging device based on the obtained information; and recording information of the user in association with the captured image based on a result of the determining.

[0009] Further features of the present application will become apparent from the following description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF DRAWINGS

[0010] Figure 1A is a front perspective view of the camera 100.

[0011] Figure 1B is a back perspective view of the camera 100.

[0012] Figure 1C is a view showing an operation lever.

[0013] Figure 2 is a cross-sectional view of the camera 100 taken along the YZ plane.

[0014] Figure 3 is a block diagram showing an electrical structure of the camera 100.

[0015] Figure 4 is a block diagram showing an example of a functional structure of the camera 100.

[0016] Figure 5 is a flowchart showing a process performed by the camera 100.

[0017] Figure 6 is a flowchart showing details of the process performed in step S503.

[0018] Figure 7 is a flowchart showing details of the process performed in step S503.

[0019] Figure 8 is a flowchart showing details of the process performed in step S503.

[0020] Figure 9A is a diagram showing a display example of a screen displayed on the display panel in the "user P's continuation use state".

[0021] Figure 9B is a diagram showing a display example of a screen displayed on the display panel in the "user unidentified state".

[0022] Figure 9C is a diagram showing a display example of a screen displayed when it is determined that the user P of the camera 100 is not continuing use of the camera 100.

[0023] Figure 10 is a diagram showing a display example of a screen displayed on the display panel in response to a touch on the setting mode.

[0024] Figure 11 is a block diagram showing a functional configuration example of the camera 100 in the sixth embodiment.

[0025] Figure 12 is a flowchart showing details of the processing performed in step S503 in the sixth embodiment.

[0026] Figure 13 is a flowchart showing the processing performed by the camera 100 in the modified example of the sixth embodiment.

[0027] Figure 14 is a block diagram showing a functional configuration example of the camera 100 in the seventh embodiment.

[0028] Figure 15 is a flowchart showing details of the processing performed in step S503 in the seventh embodiment.

[0029] Figure 16 is a flowchart showing details of the processing performed in step S503 in the modified example 1 of the seventh embodiment.

[0030] Figure 17 is a flowchart showing details of the processing performed in step S503 in the modified example 2 and the modified example 3 of the seventh embodiment.

[0031] Figure 18 is a flowchart showing details of the processing performed in step S503 in the eighth embodiment and the modified example 1 thereof.

[0032] Figure 19 is a flowchart showing details of the processing performed in step S503 in the modified example 2 and the modified example 3 of the eighth embodiment.

[0033] Figure 20is a flowchart showing details of the processing performed in step S503 in the ninth embodiment and its modification 1.

[0034] Figure 21 is a flowchart showing details of the processing performed in step S503 in the modification 2 of the ninth embodiment.

[0035] Figure 22 is a flowchart showing details of step S503 in the tenth embodiment.

[0036] Figure 23 is a block diagram showing a functional configuration example of the camera 100 in the eleventh embodiment.

[0037] Figure 24 is a flowchart showing details of the processing performed in the eleventh embodiment.

[0038] Figure 25A shows a display example of the display panel of the eyepiece 102 in the eleventh embodiment.

[0039] Figure 25B shows a display example of the display panel of the eyepiece 102 in the eleventh embodiment.

[0040] Figure 25C shows a display example of the display panel of the eyepiece 102 in the eleventh embodiment.

[0041] Figure 26 is a block diagram showing a functional configuration example of the camera 100 in the twelfth embodiment.

[0042] Figure 27 is a flowchart showing details of the processing performed in the twelfth embodiment. DETAILED DESCRIPTION

[0043] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments are not intended to limit the scope of the claimed invention. In the embodiments, a plurality of features are described, but the invention is not limited to the invention requiring all such features, and a plurality of such features can be appropriately combined. Furthermore, in the drawings, the same reference signs are assigned to the same or similar structures, and redundant descriptions thereof are omitted.

[0044] (First Embodiment)

[0045] First, an example of the appearance of a camera 100 (a digital still camera; a camera with interchangeable lens) as an example of an image pickup apparatus according to the present embodiment will be described below with reference to Figure 1A , Figure 1B , and Figure 1C . Figure 1A is a front perspective view of the camera 100, andFigure 1B is a back surface perspective view of the camera 100. In Figure 1A and Figure 1B , for convenience of explanation, the direction of the optical axis of the camera 100 is defined as the Z-axis direction, and among two axes orthogonal to the Z-axis, the horizontal direction is defined as the X-axis direction, and the vertical direction is defined as the Y-axis direction.

[0046] As shown in Figure 1A , the camera 100 includes an imaging lens unit 100A and a camera housing 100B. On the camera housing 100B, a release button 101 is provided, which is an operation member for accepting input of an operation by a user (a photographer) of the camera 100 for taking an image.

[0047] As shown in Figure 1B , on the back surface of the camera housing 100B, an eye piece 102 (viewfinder) is provided, through which the user peeks to view a display device 214 (display panel) inside the camera housing 100B. Further, on the back surface of the camera housing 100B, operation members 103 to 105 that accept various operations by the user are provided. The operation member 103 is, for example, a touch panel that accepts a touch operation by the user, and the operation member 104 is, for example, an operation lever that can be tilted in different directions as shown in Figure 1C . The operation member 105 is, for example, a four-direction key that is arranged in four directions respectively and can be pressed down. The operation member 103 (touch panel) includes a display panel (for example, a liquid crystal panel), and various types of information such as images and characters are displayed on the display panel.

[0048] Figure 2 is a cross-sectional view of the camera 100 taken along the YZ plane defined by the Y-axis and the Z-axis shown in Figure 1A , and shows a schematic internal structure of the camera 100 including a configuration related to the following description. In Figure 2 , the functional units that are the same as those shown in Figure 1A , Figure 1B and Figure 1C are denoted by the same reference numerals as those used in Figure 1A , Figure 1B and Figure 1C , and the description of these functional units is omitted.

[0049] The imaging lens unit 100A includes two lenses 201 and 202, a diaphragm 203, a diaphragm driving unit 204, a lens driving motor 205, a lens driving member 206, a photoelectric coupler 207, a pulse board 208, a mounting contact 209, and a focus adjustment circuit 210.

[0050] The lens drive member 206 includes a drive gear, and the photoelectric coupler 207 detects rotation of a pulse plate 208 interlocked with the lens drive member 206, and transmits the detection result to the focus adjustment circuit 210. Based on the detection result received from the photoelectric coupler 207 and information (information indicating a lens drive amount) received from the camera housing 100B, the focus adjustment circuit 210 drives the lens drive motor 205 to move the lens 201 and change the focus position. The mounting contact 209 is an interface for communication between the camera lens unit 100A and the camera housing 100B. Although Figure 2 Two lenses 201 and 202 are shown to simplify the drawing, but the camera lens unit 100A actually includes more than two lenses. The diaphragm drive unit 204 controls the drive of the diaphragm 203 to control opening and closing of the aperture.

[0051] The camera housing 100B includes an image pickup element 211, a CPU 212, a memory 213, a display device 214, and a display device drive circuit 215. The image pickup element 211 photoelectrically converts an image of a subject obtained through the lenses 201 and 202 to convert the image into a RAW image. The CPU 212 is a central processing unit of a microcomputer, and controls the operation of the camera 100 as a whole by using a computer program and data stored in the memory 213. The memory 213 can store various types of information, and the CPU 212 stores, for example, a captured image generated by various types of image processing including development processing on a RAW image output from the image pickup element 211 in the memory 213. The display device 214 displays various types of information such as images and letters. For example, the display device 214 is a liquid crystal panel, and displays the captured image described above and various types of information. The display device drive circuit 215 controls the drive of the display device 214. The user can view the screen on the display device 214 through the eyepiece 102.

[0052] The camera housing 100B further includes light sources 216a and 216b, a light splitter 217, a light-receiving lens 218, and an eye imaging element 219. The light sources 216a and 216b are light sources that have conventionally been used in single-lens reflex cameras to detect a line of sight from a relationship between a reflection image (corneal reflection image) formed by a corneal reflection of light and a pupil, and are used to irradiate an eyeball 220 of a user who peeks through a viewfinder (eyepiece 102). The light sources 216a and 216b are, for example, infrared light-emitting diodes that emit invisible infrared light toward the user, and are disposed around the eyepiece 102. An eye optical image (optical image formed by light emitted from the light sources 216a and 216b and reflected by the eyeball 220) of an optical image of the irradiated eyeball 220 passes through the eyepiece 102 and is reflected by the light splitter 217. The eye optical image is formed on the eye imaging element 219 including a plurality of photoelectric conversion elements (for example, CCD or CMOS) arranged in a two-dimensional array, by the light-receiving lens 218. The pupil of the eyeball 220 and the eye imaging element 219 are positioned in a conjugate imaging relationship by the light-receiving lens 218. From a position of a corneal reflection image in the eye optical image formed on the eye imaging element 219, a line of sight of the eyeball 220 can be detected, and information related to the line of sight can be obtained as a detection result. For example, at least one of information indicating a line-of-sight direction (direction of the line of sight) and information indicating a point of view (point to which the line of sight is directed) on a screen of the display device 214 can be obtained as the information related to the line of sight. The point of view can be considered as a position seen by the user or a position of the line of sight. Then, the eye imaging element 219 outputs the formed eye optical image (photographed image of the eyeball 220) as an eye image (analog image signal).

[0053] Next, the electrical structure in the camera 100 is described below with reference to the block diagram illustrated in FIG. 2. Figure 3 The CPU 212 can transmit a signal to the focus adjustment circuit 210 in the camera lens unit 100A and the diaphragm control circuit 306 included in the diaphragm driving unit 204 in the camera lens unit 100A via the mounting contact 209. For example, the CPU 212 can transmit information indicating a lens driving amount to the focus adjustment circuit 210 via the mounting contact 209. Further, the CPU 212 can transmit information indicating a driving amount of the diaphragm 203 to the diaphragm control circuit 306 via the mounting contact 209. Further, the CPU 212 controls operations of other units.

[0054] The memory 213 can store a captured image, an eye image, and a gaze correction parameter used for correcting a detected gaze, and the like. The gaze detection circuit 301 A / D-converts an eye image (an analog image signal) output from the eye imaging element 219 in a state where an eye optical image is formed on the eye imaging element 219, and generates an eye image as a digital image signal. Then, the gaze detection circuit 301 extracts a feature point necessary for detecting a gaze from the generated eye image (i.e., a digital image signal), detects a gaze of a user from a position of the feature point, and outputs information related to the gaze as a detection result. Further, the gaze detection circuit 301 stores the generated eye image (i.e., a digital image signal) in the memory 213.

[0055] The photometry circuit 302 performs predetermined processing (e.g., amplification, logarithmic compression, and A / D conversion) on an image signal (such as a luminance signal corresponding to the brightness of a subject field of view) obtained from the imaging element 211 also used as a photometry sensor, and transmits a result of the processing as subject field of view brightness information to the CPU 212.

[0056] The automatic focus detection circuit 303 A / D-converts signals output from a plurality of detection elements (a plurality of pixels) included in the imaging element 211 for phase difference detection, and calculates a distance to a subject corresponding to each focus detection point from the output signals subjected to the A / D conversion. This is a known technique called image plane phase difference AF.

[0057] The switches SW1 and SW2 of the release button 101 are connected to the signal input circuit 304. The switch SW1 is a switch for outputting an ON signal to start a camera 100 imaging preparation operation (e.g., photometry and distance measurement) in response to a first stroke of the release button 101. The switch SW2 is a switch for outputting an ON signal to start an imaging operation in response to a second stroke of the release button 101. The ON signals output from the switches SW1 and SW2 are input to the signal input circuit 304 and transmitted to the CPU 212. Upon receiving the ON signal from the switch SW1, the CPU 212 controls the operation of each unit to start the camera 100 imaging preparation operation. Upon receiving the ON signal from the switch SW2, the CPU 212 controls the operation of each unit to start the imaging operation. The detection of the gaze can be started in response to the switch SW1 turning on. The light source drive circuit 305 controls the drive of the light sources 216a and 216b.

[0058] Further, the operation members 103 to 105 are also connected to the CPU 212. When the user operates the operation members 103 to 105, the operation members 103 to 105 output operation signals corresponding to the operations performed by the user to the CPU 212, and the CPU 212 performs processing (control) corresponding to the operation signals. For example, the CPU 212 moves the frame (item, pointer) displayed at the viewpoint in response to the operation signals.

[0059] The communication unit 307, the GPS receiving unit 308, the timer 309, and the inertial sensor 310 are also connected to the CPU 212. The communication unit 307 obtains GPS position information of an information device (such as a smartphone, a tablet terminal, a smart watch, or a PC held by the user, or the like) by performing wireless communication with the information device. Examples of the wireless communication include Wi-Fi and Bluetooth, but are not limited to these examples. The GPS receiving unit 308 receives radio waves from GPS satellites and obtains GPS position information of the camera 100. The timer 309 outputs the current time. The inertial sensor 310 (for example, an acceleration sensor and an angular velocity sensor) outputs an acceleration rate and an angular velocity of the camera 100 as output values.

[0060] In Figure 4 a block diagram illustrating an example of a functional configuration of the camera 100 related to the processing described below. In the present embodiment, Figure 4 Each functional unit illustrated in Figure 2 and Figure 3 is implemented by using one or more pieces of hardware, and operates under the control performed by the CPU 212.

[0061] Note that Figure 4 Each functional unit illustrated in

[0062] The processing performed by the camera 100 according to the present embodiment in order to identify the user of the camera 100 and store a captured image in the storage 213 in association with the identification result will be described below with reference to the flowchart illustrated in Figure 5 Note that the processing according to the flowchart illustrated in Figure 5 begins when the power of the camera 100 is turned on and the camera 100 is started to be used, and is repeated until it becomes no longer possible to continue the processing, for example, due to the power of the camera 100 being turned off.

[0063] In step S501, the user identifying unit 402 performs processing for identifying the user of the camera 100 by performing iris recognition on the eye image stored in the storage 213. In step S502, the user identifying unit 402 determines whether the user of the camera 100 has been successfully identified in step S501.

[0064] If it is determined that the user of the camera 100 has been successfully identified in step S501, the processing proceeds to step S503, and if it is determined that the user of the camera 100 has not been successfully identified in step S501, the processing proceeds to step S511.

[0065] For example, the user identifying unit 402 identifies the registered iris that is most similar to the iris detected from the eye image among the irises of the respective users (registered irises) that are registered in advance in the storage 213 as the similar registered iris. Then, the user identifying unit 402 identifies the user corresponding to the similar registered iris as the user of the camera 100.

[0066] Note that if the degree of similarity between the iris detected from the eye image and the similar registered iris is lower than a threshold value, the user identifying unit 402 determines that the identification of the user has failed. Further, if no one is peering through the viewfinder 102 and the eye image does not include an iris, no iris is recognized from the eye image, and therefore, the user identifying unit 402 determines that the identification of the user has failed. Further, if something other than an eyeball is in contact with the viewfinder 102, no iris is recognized from the eye image, and therefore, the user identifying unit 402 determines that the identification of the user has failed.

[0067] In step S511, the continued use determining unit 403 continues to determine whether the imaging unit 401 serving as the camera lens unit 100A has captured an image, for example, in response to the release button 101 being pressed down. If it is determined that the imaging unit 401 has captured an image, the processing proceeds to step S512, and if it is determined that the imaging unit 401 has not captured an image, the processing proceeds to step S501.

[0068] In step S512, the recording unit 404 adds, as metadata, the photographer information indicating that the photographer has not been identified to the image captured by the imaging unit 401. Then, the recording unit 404 records the photographed image (photographed image file) to which the metadata has been added in the storage 213. Then, the processing proceeds to step S501.

[0069] On the other hand, in step S503, the continued use determining unit 403 determines whether the user of the camera 100 identified in step S501 (hereinafter referred to as "user P") is continuing the use of the camera 100. Details of the processing performed in step S503 will be described later.

[0070] If it is determined that the user P is continuing to use the camera 100, the process proceeds to step S521 via step S504, and if it is determined that the user P is not continuing to use the camera 100, the process proceeds to step S501 via step S504.

[0071] In step S521, similarly to step S511 described above, the continued use determination unit 403 determines whether the imaging unit 401 has captured an image, for example, in response to the release button 101 being pressed. If it is determined that the imaging unit 401 has captured an image, the process proceeds to step S522, and if it is determined that the imaging unit 401 has not captured an image, the process proceeds to step S503.

[0072] In step S522, the recording unit 404 adds, as metadata, the photographer information indicating that the photographer is the user P to the image captured by the imaging unit 401. Then, the recording unit 404 records the captured image (a captured image file) to which the metadata is added in the storage 213. Then, the process proceeds to step S503.

[0073] As described above, the continued use determination unit 403 defines the period during which the camera 100 is used by the user P as the "continued use range of the user P", and adds, as metadata, the photographer information indicating that the photographer is the user P to the image captured within the continued use range.

[0074] Note that the method for recording the photographer information is not limited to the method of adding the photographer information as metadata to the captured image. For example, a hash value of the captured image and the photographer information can be stored in a table in association with each other, or a record including the captured image and the photographer information can be uploaded by accessing an external server via a communication path such as wireless communication. A method for preventing forgery or ensuring authenticity such as encryption or electronic signature can also be used in combination with these information. The following configuration can also be possible: in step S512, the recording unit 404 stores the image captured by the imaging unit 401 in the storage 213 without adding the photographer information indicating that the photographer is not identified to the image.

[0075] As described above, the recording unit 404 performs control so that the image captured by the camera 100 is recorded in association with the information of the user in the storage 213 between the time when the user of the camera 100 is identified and the time when it is determined that the use of the camera 100 by the user is not continuing.

[0076] Further, the recording unit 404 performs control so that if the user of the camera 100 is not recognized, the image captured by the camera 100 is recorded in the memory in association with information indicating that the user of the camera 100 is not recognized.

[0077] Next, details of the processing performed in step S503 described above will be explained below with reference to the flowchart shown in Figure 6 In step S601, it is determined using the determination unit 403 whether the user P has instructed the end button included in at least one of the operation members 103 to 105. For example, if the operation member 103 is the display panel as shown in Figure 9A In this case, the list 903 includes three modes, that is, a mode for capturing an image (image capturing mode), a mode for causing the display panel to display information on the file stored in the memory 213 (viewing mode), and a mode for performing a setting of the camera 100 (setting mode). Therefore, in response to the user P touching the mode different from the mode currently set in the camera 100 among the three modes, it is determined using the determination unit 403 that the user P has performed the mode change operation on the operation member 103. If the mode currently set in the camera 100 is the image capturing mode, the mode change operation performed by the user is an operation for changing the operation state of the camera to a state other than the image capturing, and therefore, the operation is considered as an operation performed by the user in order to end the image capturing.

[0078] Note that the end button is not limited to the button displayed on the display panel, and can also be a physical button. It can also be determined that the end button included in the operation lever such as the operation member 104 has been instructed in response to the operation member 104 being pressed. It can also be determined that the end button has been instructed in response to the power button or the like being instructed. If it is determined that the end button has been instructed, the processing proceeds to step S605, and if it is determined that the end button has not been instructed, the processing proceeds to step S602.

[0079] In step S602, it is determined using the determination unit 403 whether the user P has performed a mode change operation on at least one of the operation members 103 to 105. For example, if the operation member 103 is the display panel as shown in Figure 9A In this case, the list 903 includes three modes, that is, a mode for capturing an image (image capturing mode), a mode for causing the display panel to display information on the file stored in the memory 213 (viewing mode), and a mode for performing a setting of the camera 100 (setting mode). Therefore, in response to the user P touching the mode different from the mode currently set in the camera 100 among the three modes, it is determined using the determination unit 403 that the user P has performed the mode change operation on the operation member 103. If the mode currently set in the camera 100 is the image capturing mode, the mode change operation performed by the user is an operation for changing the operation state of the camera to a state other than the image capturing, and therefore, the operation is considered as an operation performed by the user in order to end the image capturing.

[0080] In Figure 9A In this case, the list 903 includes three modes, that is, a mode for capturing an image (image capturing mode), a mode for causing the display panel to display information on the file stored in the memory 213 (viewing mode), and a mode for performing a setting of the camera 100 (setting mode). Therefore, in response to the user P touching the mode different from the mode currently set in the camera 100 among the three modes, it is determined using the determination unit 403 that the user P has performed the mode change operation on the operation member 103. If the mode currently set in the camera 100 is the image capturing mode, the mode change operation performed by the user is an operation for changing the operation state of the camera to a state other than the image capturing, and therefore, the operation is considered as an operation performed by the user in order to end the image capturing.

[0081] If it is determined that the user P has performed a mode change operation on at least one of the operation members 103 to 105, the process proceeds to step S605. On the other hand, if the user P has not performed a mode change operation on the operation members 103 to 105, the process proceeds to step S604.

[0082] In step S604, the continued use determination unit 403 determines that the user P is continuing to use the camera 100. On the other hand, in step S605, the continued use determination unit 403 determines that the user P is not continuing to use the camera 100.

[0083] Note that the operation indicating the end of the use of the camera 100 by the user P includes various operations other than the above-described operation, and the end of the use of the camera 100 can be indicated by an operation for transmitting an end signal from a remote controller or a device that is in wireless communication with the camera. These methods can also be combined or only a part of these methods can be performed.

[0084] Convenience of the device that stores the information of the user together with the captured image is improved by determining whether the user has performed an operation for ending the image capturing and by defining a period between the time when the user is recognized and the time when the user performs the operation for ending the image capturing as the continued use range.

[0085] Note that the method for recognizing the user of the camera 100 is not limited to the iris recognition. For example, the camera 100 can recognize the user of the camera 100 by using other types of biometric information such as a fingerprint, a vein, or a face, or by causing the user to input authentication information such as an identification number or a password. The user of the camera 100 can also be recognized by causing the camera 100 to read authentication information such as an identification number or a password from a recording medium such as an ID card on which the information is recorded. An authentication dongle used for such authentication can also be connected to the camera 100. Alternatively, the above-described authentication can be performed by wireless communication with an authentication device such as a smart phone or a smart watch in the vicinity of the camera 100, or a two-step authentication can be performed by combining these methods.

[0086] As described above, the method for recognizing the user of the camera 100 (authentication method) is not limited to a specific method, and the above-described authentication can be performed by the camera 100 alone or through communication with other devices.

[0087] In the present embodiment, the memory 213 is provided inside the camera housing 100B, but the memory 213 can also be a "memory that is connected to the camera housing 100B in a physically detachable manner" such as an SD card or a USB memory.

[0088] <Modification example>

[0089] Instead of determining the end of the use of the camera 100 in response to the user pressing the end button, it is also possible to determine the end of the use of the camera 100 at the timing at which the user releases the use continuation button that has been continuously pressed by the user. In this case, in step S601, it is judged whether or not the user releases the use continuation button that has been continuously pressed by the user. If the user releases the use continuation button that has been continuously pressed by the user, the process proceeds to step S605, and if the user does not release the use continuation button that has been continuously pressed by the user, the process proceeds to step S602.

[0090] The use continuation button can be a button on a touch panel or an operation lever that is applicable to the operation members 103 to 105, and if, for example, the button is configured to be easily continuously pressed during the user is holding the camera 100 by being provided on the holding portion of the camera 100, it is expected that convenience is improved.

[0091] That is, in step S601 of the present embodiment, in a state (input state) in which an instruction (use continuation instruction) to continue the use of the camera 100 is input, the process proceeds to step S602. On the other hand, if the use continuation instruction is no longer input in the input state in step S601, the process proceeds to step S605. As described above, the use continuation instruction can be input using various methods.

[0092] (Second Embodiment)

[0093] In the following embodiments including the present embodiment, differences from the first embodiment will be described, and matters not particularly mentioned below are similar to those in the first embodiment. In the present embodiment, a method for determining a continuation use range based on an operation on which it is possible to infer that the end of imaging is not an explicit operation by the user, although the operation is not an explicit operation by the user, will be described. Note that the following method can also be used in combination with the method for determining a continuation use range according to the first embodiment.

[0094] The details of the process performed in step S503 according to the present embodiment will be described below with reference to the flowchart illustrated in Figure 7 Figure 7 The same processing steps as those illustrated in Figure 6 are indicated by the same step numbers as those used in Figure 6 , and the description of these processing steps is omitted.

[0095] ​In step S701, the judgment unit 403 continues to obtain "information indicating whether or not the camera lens unit 100A is mounted on (connected to) the camera housing 100B" from the mounting contact 209. Then, the judgment unit 403 continues to judge whether or not the obtained information indicates "the camera lens unit 100A is mounted on the camera housing 100B" or "the camera lens unit 100A is not mounted on the camera housing 100B".

[0096] If it is judged that the obtained information indicates "the camera lens unit 100A is mounted on the camera housing 100B (the camera lens unit 100A is not removed from the camera housing 100B)", the processing proceeds to step S702.

[0097] On the other hand, if the obtained information indicates "the camera lens unit 100A is not mounted on the camera housing 100B (the camera lens unit 100A has been removed from the camera housing 100B)", the processing proceeds to step S605.

[0098] In step S702, the judgment unit 403 continues to obtain "information indicating the brightness of the input to the image pickup element 211" from the mounting contact 209. Then, the judgment unit 403 continues to judge whether or not the "brightness of the input to the image pickup element 211" indicated by the obtained information is equal to or lower than a threshold value, for example, whether or not the camera is in a state where the input is completely dark, for example, due to the lens cap being attached to the camera lens unit 100A.

[0099] If it is judged that the "brightness of the input to the image pickup element 211" indicated by the obtained information is equal to or lower than the threshold value, the processing proceeds to step S605, and if it is judged that the "brightness of the input to the image pickup element 211" indicated by the obtained information is higher than the threshold value, the processing proceeds to step S703.

[0100] In step S703, the judgment unit 403 continues to obtain "information indicating whether or not an external device is connected to the camera 100" from a communication module included in the camera 100. Then, the judgment unit 403 continues to judge whether or not the obtained information indicates "an external device is connected to the camera 100".

[0101] If it is judged that the obtained information indicates "an external device is not connected to the camera 100" (communication between the camera 100 and the external device has been cut off), the processing proceeds to step S605. On the other hand, if the obtained information indicates "an external device is connected to the camera 100" (communication between the camera 100 and the external device has not been cut off), the processing proceeds to step S604.

[0102] Note that in the present embodiment, the following three operations are described as examples of operations for inferring whether or not the imaging by the camera 100 has ended: "judging whether or not the imaging lens unit 100A is mounted on the camera housing 100B", "judging whether or not the luminance of the input to the imaging element 211 is equal to or lower than a threshold value", and "judging whether or not the communication between the camera 100 and the external device has been cut off". However, the operations for inferring whether or not the imaging by the camera 100 has ended are not limited to these three operations. For example, the following configuration is also possible: if it is detected that an accessory such as a strobe or a release cable other than a lens has been removed from the camera housing 100B, it is judged that the imaging by the camera 100 has ended. Alternatively, the following configuration is also possible: if it is detected that an operation for opening a battery cover or an operation for removing an SD card, it is judged that the imaging by the camera 100 has ended. These methods can also be combined or only a part of these methods can be performed.

[0103] As described above, in the case where the continuation use range is determined based on the operation on which it is inferred that the imaging has ended even if the operation is not an explicit operation by the user, the convenience of the device that authenticates and records the photographer is improved.

[0104] (Third Embodiment)

[0105] In the present embodiment, a method for determining the period of the operation by the user who is imaging as the continuation use range will be described. The details of the processing performed in step S503 according to the present embodiment will be described below with reference to the flowchart shown in FIG. 9. In Figure 8 Figure 8 the processing steps shown in FIG. 8 are the same as those in Figure 6 the processing steps shown in FIG. 8 are the same as those in Figure 6 the processing steps shown in FIG. 8 are the same as those in

[0106] In step S801, the continuation use judging unit 403 judges whether or not continuous imaging is being performed in a state where the release button 101 is being held down. If it is judged that continuous imaging is being performed in a state where the release button 101 is being held down, the processing proceeds to step S802, and if it is judged that continuous imaging is not being performed by holding down the release button 101, the processing proceeds to step S803.

[0107] In step S802, the continuation use judging unit 403 judges whether or not the number of images taken after the iris recognition is equal to or greater than a predetermined maximum number N of taken images. After the iris recognition, the continuation use judging unit 403 counts up the number of taken images each time an image is taken.

[0108] ​If it is determined that the number of images taken after the iris recognition is equal to or greater than the predetermined maximum number N of taken images, the process proceeds to step S605, and if it is determined that the number of images taken after the iris recognition is less than the predetermined maximum number N of taken images, the process proceeds to step S803.

[0109] In step S803, the determination unit 403 continues to determine whether the user is holding the grip portion provided on the camera housing 100B by using the contact sensor provided on the grip portion.

[0110] If it is determined that the user is holding the grip portion, the process proceeds to step S604, and if it is determined that the user is not holding the grip portion, the process proceeds to step S804.

[0111] In step S804, the determination unit 403 continues to determine whether the direction (orientation) in which the camera 100 is facing is changing by using the gyro sensor included in the camera housing 100B. For example, if the difference between the direction measured by the gyro sensor previously and the direction measured by the gyro sensor currently is equal to or greater than a threshold value, the determination unit 403 continues to determine that the direction (orientation) in which the camera 100 is facing is changing.

[0112] If it is determined that the direction (orientation) in which the camera 100 is facing is changing, the process proceeds to step S604, and if it is determined that the direction (orientation) in which the camera 100 is facing is not changing, the process proceeds to step S805.

[0113] Here, the bottom portion of the camera housing 100B according to the present embodiment includes a fixing screw hole and a screw detection sensor for detecting whether a screw has been fixed to the fixing screw hole. If the camera housing 100B is fixed to a device such as a tripod or a gimbal, the camera housing 100B can be fixed to the device by attaching the camera housing 100B to the device and then fixing a screw to the fixing screw hole.

[0114] In step S805, the determination unit 403 continues to determine whether a screw has been fixed to the fixing screw hole by using the screw detection sensor. If it is determined that a screw has been fixed to the fixing screw hole, the process proceeds to step S604, and if it is determined that a screw has not been fixed to the fixing screw hole, the process proceeds to step S605.

[0115] As described above, in the present embodiment, the period in which the user is performing the operation for taking an image is determined as the continuation range, and thus the convenience of the device that authenticates and records the photographer is improved.

[0116] (FOURTH EMBODIMENT)

[0117] For example, it can be inferred that the camera 100 is being continuously used during a period when the camera 100 itself is moving, because the user carries the camera 100 or turns the camera 100 toward a subject. Therefore, an inertial sensor or a gyro sensor or the like is installed in the camera 100 to detect the movement of the camera 100, and the continuous use judging unit 403 determines a period when the movement of the camera 100 is detected as a continuous use range.

[0118] It is also possible to judge that the camera 100 is being continuously used during a period when the camera 100 is located within a certain geographical range, for example, by using the position of the camera 100. A position measuring device such as a GPS device is installed in the camera 100, and the continuous use judging unit 403 determines a period when the position measured by the position measuring device is within a certain range as a continuous use range.

[0119] It is also possible to install a clock in the camera 100, and determine a certain time range as a continuous use range. It is also possible to judge that the camera is being continuously used to take a series of images if similar images are being taken. The continuous use judging unit 403 evaluates the similarity between images taken consecutively based on a difference image or a statistical quantity, and determines a period when images with a similarity equal to or lower than a threshold are taken as a continuous use range.

[0120] Note that the above-described methods are examples of methods for determining a continuous use range, and the present application is not intended to be limited to these methods. In a case where a continuous use range is determined by using various types of information as described above, the convenience of the device that authenticates and records a photographer is improved.

[0121] (Fifth Embodiment)

[0122] In the present embodiment, a screen displayed in relation to the processing in the above-described embodiments will be described. Here, for example, in a case where the user is not identified in steps S501 and S502 in the flowchart shown in Fig. 10 and the processing is repeatedly performed in a loop, the state will be referred to as a "user non-identified state". Further, in a case where it is continuously judged that the camera is being continuously used in steps S503 and S504 and the processing is performed in a loop, the state will be referred to as a "user P's continuous use state" (user P is the user identified in step S501). Note that the CPU 212 performs display-related processing, i.e., display control for displaying a screen or switching a screen or changing a display in response to a user operation, for example. Figure 5

[0123] Fig. 13 is a diagram showing a display example of a screen displayed on the display panel as the operation member 103 in the "user P's continuous use state". Figure 9A Figure 9A ​The illustrated screen includes the subject 900, a display 901 showing the user (user P) of the camera 100 and the time elapsed from the iris recognition, a mode selection list 903, and an end button 902. Further, the screen includes a display 904 showing the number of images taken after the iris recognition / the maximum number N of taken images (N = 100 in this example).

[0124] Figure 9B is a diagram showing a display example of a screen displayed on the display panel that is the operation member 103 in the "user non-identified state". The display 901 indicates that the user is not identified through the iris recognition.

[0125] Note that the camera 100 can also be configured to enable the user to grasp the remaining number of images that can be taken by using the display 904 described above by, for example, changing the color or size of the display 904 or emitting a sound when the number of taken images approaches N.

[0126] If the continuation use determination unit 403 determines that the user P of the camera 100 is not continuing the use of the camera 100, the screen illustrated in Figure 9C may also be displayed on the display panel that is the operation member 103. Figure 9C The display 901 includes a message indicating the end of the continuation use of the camera 100 by the user P and the reason for the end. It is also possible for the display 901 to display only for a prescribed period of time (for example, 5 seconds). In the example illustrated in

[0127] This configuration enables the user to understand the reason even if the use of the camera ends unintentionally. In order to make it easier for the user to notice such a situation, for example, the display can be emphasized by changing the color of the entire screen or a sound can be emitted.

[0128] Note that the information described above can also be displayed on the display device 214 instead of being displayed on the display panel of the operation member 103, so that the information is superimposed on the viewfinder, or can be displayed on both the display panel and the display device.

[0129] As another display method, an LED lamp can also be provided on the camera 100, the LED lamp is lit in the "continuation use state", the LED lamp is turned off in the "user non-identified state", and the LED lamp is blinked at the end of the continuation use. At this time, for example, similar information can be displayed by changing the color of the LED lamp for each user. Additional control can also be performed to turn off the LED lamp during the shutter opening, so that the light of the LED lamp will not affect the image.

[0130] As other methods, it is also possible to notify the user of the state of continuation of use of the range by using a shutter sound. Different shutter sounds are set for each user P in the "continuation of use state of user P" as the shutter sound output in response to the release button 101 being pressed down, and a shutter sound different from all of these shutter sounds is output in response to the release button 101 being pressed down in the "user not identified state". This enables the user to recognize whether the current state is within the continuation of use range, and which user has been authenticated and recorded as the photographer of the image. It is also possible to output a sound in response to the release button being half-pressed, instead of outputting a shutter sound after the shutter is opened.

[0131] Here, when the user touches the setting mode included in the list 903, the display panel as the operation member 103 displays the screen shown in FIG. 10. This screen shows a list 1001 of items that can be set, and displays a list 1002 in response to the user touching "method for maintaining authentication" included in the list 1001. Figure 10

[0132] The list 1002 is used to make each of the above-described various methods for judging whether the user of the camera 100 is continuing the use of the camera 100 effective (ON) or ineffective (OFF), and to select whether to display the end button 902. Further, the list 1002 includes a number box 1003. The user can input the maximum number N of photographed images in the number box 1003. In the case where the state of the camera 100 is displayed for the user and each user is allowed to make settings as described above, the convenience of the apparatus that authenticates and records the photographer is further improved.

[0133] (Sixth Embodiment)

[0134] In this embodiment, a method for determining the continuation of use range based on information indicating the movement of the camera in space will be described. An example of the functional structure of the camera 100 related to the processing described below is shown in the block diagram of FIG. 11. The functional blocks include an imaging unit 401, an identification unit 402, a movement obtaining unit 1101, a continuation of use judging unit 403, and a recording unit 404. The imaging unit 401, the identification unit 402, and the recording unit 404 are the same as in the first embodiment, so the description thereof is omitted. Figure 11

[0135] The movement obtaining unit 1101 obtains information indicating the acceleration rate and angular velocity of the camera 100 from the inertial sensor 310. The continuation of use judging unit 403 according to this embodiment judges whether the user is continuing the use of the camera 100 based on the movement information obtained by the movement obtaining unit 1101.

[0136] Reference will be made below to Figure 12 ​​The flowchart shown will be used to explain details of the processing performed in step S503 according to the present embodiment. In step S1201, the condition for judging that the user P is continuing to use the camera is obtained using the judgment unit 403. Here, an example will be explained in which the condition for judging that the user P is continuing to use the camera 100 is judged to be moving based on criteria described later.

[0137] In step S1202, the movement obtaining unit 1101 obtains the acceleration rate and angular velocity of the camera 100 from the inertial sensor 310 as movement information. The movement information obtained each time step S1202 is executed is stored as history in the memory.

[0138] In step S1203, it is judged whether the movement information obtained in step S1202 satisfies the condition obtained in step S1201. The judgment unit 403 judges whether the camera 100 is currently moving or stationary based on the history of the movement information obtained in step S1202. More specifically, if the amount of change in the obtained acceleration rate over a certain period of time recently is equal to or greater than a predetermined threshold value, or the latest angular velocity is equal to or greater than a predetermined threshold value, it is judged that the camera 100 is currently moving.

[0139] If it is judged that the camera 100 is moving, the processing proceeds to step S1204, and it is judged that the user P is "continuing to use the camera", and the flow ends. On the other hand, if it is judged that the camera 100 is stationary, the processing proceeds to step S1205 and it is judged that the user P is "not continuing to use the camera", and the flow ends.

[0140] With this configuration, it is possible to detect camera 100 shaking, for example, during the period when the user is holding the camera 100 with his hand, and judge that the use of the camera is continuing. On the other hand, when the user has placed the camera 100 on a table or the like and removed his hand from the camera, the camera 100 is stationary, so it is possible to judge that the user is not continuing to use the camera 100.

[0141] In the method according to the present embodiment, it is determined that the use of the camera by the same user is continuing and the authentication of the user is maintained during the camera 100 is continuing to move, and thus the convenience of the apparatus is improved. In the present embodiment, the acceleration rate and the angular velocity of the camera 100 are obtained from the inertial sensor 310 as the movement information, but the method for obtaining the movement information is not limited to this example. For example, the following configuration is also possible: the images captured by the camera 100 are continuously obtained, and the movement of the camera 100 is estimated by performing optical flow analysis which is a known technique. Alternatively, the differential value of the position information obtained by a position measuring unit such as a GPS device can also be used.

[0142] In the present embodiment, the condition for determining the continued use of the camera 100 by the user P is that the camera is continuing to move, but the condition for making this determination based on the movement information is not limited to this example. For example, in the case where the camera 100 is fixed to a tripod or the like to intermittently capture images, it can also be determined that the user P is continuing to use the camera 100 during the camera 100 is stationary. Alternatively, a condition for determining that the user P stopped the continued use of the camera 100 in response to detecting a specific movement pattern (for example, in response to the user rapidly shaking the camera 100 twice) can also be added. Alternatively, as in programmed photography by using an automatic pan-tilt head, a condition for determining that the user P is continuing to use the camera 100 during the camera 100 is moving in a pre-registered pattern can also be set. These conditions can be pre-selected or combined based on the operation performed by the user.

[0143] <Modification>

[0144] In the sixth embodiment, it is determined whether the user is continuing to use the camera 100 based on a pre-set condition, but in the present modification, this condition is determined based on the movement information of the time when the user is recognized.

[0145] Figure 1A 、 Figure 1B and Figure 1C the appearance of the camera shown in Figure 2 the structure of the camera shown in Figure 3 the electrical structure of the camera shown in, and Figure 11 the functional structure of the camera shown in are the same as in the sixth embodiment, and thus the description thereof is omitted. The process of the processing performed by the camera 100 according to the present embodiment in order to determine the state of the continued use of the user is explained below with reference to the flowchart shown in Figure 13 the steps from step S501 to step S522 are the same as in the first embodiment, and thus the description thereof is omitted. The details of the processing performed in step S503 are the same as the processing described in the sixth embodiment except for the matters described below. The details of the processing performed in step S504 are the same as the processing described in the first embodiment except for the matters described below. Figure 12Details of the processing shown are the same, and thus the explanation thereof is omitted.

[0146] In step S1301, the movement obtaining unit 1101 obtains movement information at a timing at which the processing in step S502 has determined that the authentication is successful. Here, the movement obtaining unit 1101 obtains information about the continued movement during a certain period of time. Thereafter, the continued use judging unit 403 judges, based on the movement information, whether the camera 100 was moving or stationary at the timing at which the user was successfully authenticated, using the same criteria as those used in step S1203.

[0147] In step S1201, the continued use judging unit 403 sets a condition for judging that the user P is continuing the use of the camera 100, in accordance with the movement of the camera 100 at the timing at which the user was successfully authenticated, which has been judged in step S1301. More specifically, if the camera 100 was moving at the timing at which the user was successfully authenticated, a condition that the camera 100 is continuing to move is set as a condition for judging that the user P is continuing to use the camera 100. If the camera 100 was stationary at the timing at which the user was successfully authenticated, a condition that the camera 100 is continuing to be stationary is set as a condition for judging that the user P is continuing to use the camera 100.

[0148] In this case, even if the user P does not actively set the judging condition, the judging condition can be automatically set by the continued use judging unit 403, and the authentication of the user is maintained in a condition corresponding to the situation of the use of the camera 100, and thus the convenience of the apparatus is improved.

[0149] (SEVENTH EMBODIMENT)

[0150] In the seventh embodiment, a method will be described in which an information apparatus such as a smart phone or the like, which is carried by the user of the camera 100, is paired with the camera 100, and if the position of the camera 100 is within a predetermined distance from the position of the information apparatus, it is judged that the user is continuing to use the camera. By maintaining the state in which the user is authenticated when the position of the user and the position of the camera 100 are close to each other, the frequency of the authentication can be reduced. As a result, the convenience of the apparatus is improved.

[0151] In Figure 14 A functional configuration example of the camera 100 related to the processing described below is shown in a block diagram. The functional blocks include the imaging unit 401, the recognition unit 402, the information obtaining unit 1401, the continued use judging unit 403, and the recording unit 404. The imaging unit 401, the recognition unit 402, and the recording unit 404 are the same as in the first embodiment, and thus the explanation thereof is omitted.

[0152] The information obtaining unit 1401 obtains the GPS position information of the camera 100 and the GPS position information of the information device held by the user and paired with the camera 100 from the GPS receiving unit 308. The continue use judging unit 403 sets the position of the information device obtained by the information obtaining unit 1401 as the reference position and judges whether the user is continuing to use the camera 100 based on the reference position and the position of the camera 100.

[0153] The process of the camera 100 according to the present embodiment for judging the state of the user's continued use will be described below. Figure 5 The steps shown from step S501 to step S522 are the same as in the first embodiment, and thus the description thereof will be omitted. The process for judging that the user P is continuing to use the camera in step S503 will be described below with reference to the flowchart shown in FIG. 17. Figure 15 The process for judging that the user P is continuing to use the camera in step S503 will be described below with reference to the flowchart shown in FIG. 17.

[0154] First, in step S1501, the information obtaining unit 1401 obtains the positions of the camera 100 and the information device. The position information of the camera 100 and the information device is the GPS position information obtained by the information obtaining unit 1401.

[0155] In step S1502, the continue use judging unit 403 judges whether the distance between the camera 100 and the reference position that is the position of the information device is equal to or shorter than a predetermined distance. If the distance is equal to or shorter than the predetermined distance, the process proceeds to step S1503 and judges that the user P is "continuing to use the camera", and the flow ends. If the distance is longer than the predetermined distance, the process proceeds to step S1504 and judges that the user P is "not continuing to use the camera", and the flow ends.

[0156] In the method according to the present embodiment, when the information device held by the user and the camera 100 approach each other, it is judged that the user is continuing to use the camera, and thus the authentication of the user is maintained during the shooting of a series of images, and thus the convenience of the device is improved.

[0157] (Modified Example 1 of the Seventh Embodiment)

[0158] Next, Modified Example 1 of the seventh embodiment will be described below. In the seventh embodiment, it is judged that the camera 100 and the information device held by the user approach each other based on the GPS position information of the camera 100 and the information device, but in the present modified example, it is judged based on whether communication is possible between the camera 100 and the information device. If close-range communication such as Bluetooth is used for the communication between the camera 100 and the information device, it is possible to judge whether the camera 100 and the information device approach each other based on whether communication is possible.

[0159] Figure 14 The functional units of the camera 100 shown are the same as in the seventh embodiment. In this modification example, Figure 14 The information obtaining unit 1401 shown obtains information indicating whether or not communication is possible between the camera 100 and the information device held by the user and paired with the camera 100 from the communication unit 307.

[0160] The continuation use judging unit 403 continues to judge whether or not the user is continuing to use the camera 100 based on the information obtained by the information obtaining unit 1401 indicating whether or not communication is possible between the camera 100 and the information device.

[0161] The process of the processing performed by the camera 100 according to the present embodiment in order to judge the continuation use of the user is the same as the process of the processing in the first embodiment shown in the flowchart of Figure 5 The flowchart starts when the information device held by the user is paired with the camera 100 so that communication is possible therebetween. The following explains the flowchart shown in Figure 16 The process of judging that the user P is continuing the use of the camera in step S503 shown. Figure 5

[0162] First, in step S1601, it is judged whether or not communication is possible between the camera 100 and the paired information device. If communication is possible, the processing proceeds to step S1602 and it is judged that the user P is "continuing the use of the camera", and the flow ends. If communication is not possible, the processing proceeds to step S1603 and it is judged that the user P is "not continuing the use of the camera", and the flow ends.

[0163] In the method according to the present modification example, even if the absolute positions of the information device and the camera 100 cannot be measured, the authentication of the user is maintained during the shooting of a series of images, and thus the convenience of the device is improved.

[0164] (Modification Example 2 of the Seventh Embodiment)

[0165] Next, Modification Example 2 of the seventh embodiment will be explained. In Modification Example 1 of the seventh embodiment, the proximity between the camera 100 and the user is judged based on whether or not communication is possible between the camera 100 and the information device held by the user. In the present modification example, the distance between the camera 100 and the user is measured by using a distance sensor included in the camera 100 to make the judgment. Examples of the distance sensor include an ultrasonic sensor, a LiDAR (Light Detection and Ranging), and the like, but other sensors can also be used.

[0166] A distance sensor directly connected to the CPU is added to​Figure 3 The electrical structure of the camera 100 is shown. In this modification example, Figure 14 The information obtaining unit 1401 obtains distance information output by the distance sensor of the camera 100. The continuation use judging unit 403 continues to judge whether the user is continuing to use the camera 100 based on the distance information obtained by the information obtaining unit 1401.

[0167] The process of the processing performed by the camera 100 according to the present embodiment in order to judge the continuation use of the user is the same as that of the first embodiment shown in the flowchart of Figure 5 The flowchart shown in the first embodiment. The flowchart starts when the distance sensor is enabled to measure the distance. At the start of the flowchart, the "predetermined distance" required for the continuation use judging unit 403 to make the judgment is already recorded in the memory 213 of the camera 100. The following describes the process of judging whether the user P is continuing to use the camera in Figure 17 The flowchart shown in the first embodiment. The flowchart starts when the distance sensor is enabled to measure the distance. At the start of the flowchart, the "predetermined distance" required for the continuation use judging unit 403 to make the judgment is already recorded in the memory 213 of the camera 100. The following describes the process of judging whether the user P is continuing to use the camera in Figure 5 The process of judging that the user P is continuing to use the camera in step S503 shown in the first embodiment.

[0168] First, in step S1701, the distance between the camera 100 and the user output by the distance sensor is obtained as the position information. In step S1702, it is judged whether the distance between the user and the camera 100 is equal to or shorter than the predetermined distance. If the distance is equal to or shorter than the predetermined distance, the processing proceeds to step S1703 and it is judged that the user P is "continuing to use the camera", and the flow ends. On the other hand, if the distance is longer than the predetermined distance, the processing proceeds to step S1704 and it is judged that the user P is "not continuing to use the camera", and the flow ends.

[0169] In the present modification example, the distance between the camera 100 and the user is measured by using the distance sensor included in the camera, but other methods can also be used. A camera for detecting the face of the user can be provided on the back of the camera 100, and the judgment can be made based on the distance between the camera 100 and the user estimated from the size of the face of the user, or it can be judged that the camera 100 and the user are close to each other during the period when the face of the user can be detected.

[0170] In the method according to the present modification example, the distance between the camera 100 and the user can be measured by using only the camera 100, and the authentication of the user is maintained during the shooting of a series of images, and thus the convenience of the device is improved.

[0171] (Modification Example 3 of the Seventh Embodiment)

[0172] Next, a modification 3 of the seventh embodiment will be described below. In the modification 2 of the seventh embodiment, the distance between the camera 100 and the user is measured by using the distance sensor included in the camera, but in the present modification, the distance is measured by using an external sensor such as a monitoring camera.

[0173] The measurement is performed by using the monitoring camera, using the parameters for conversion between the coordinates in the real space and the camera coordinates determined in advance via the camera calibration, to determine the coordinates in the real space corresponding to the position of the object detected in the captured image. Note that the monitoring camera has a communication function and is capable of transmitting the information of the measured distance to the camera.

[0174] Figure 14 The functional structure of the camera 100 shown is the same as in the seventh embodiment. In the present modification, Figure 14 The information obtaining unit 1401 shown obtains the position information of the camera 100 and the user by communication between the communication unit 307 and the monitoring camera.

[0175] The continue use judging unit 403 continues to set the position of the user obtained by the information obtaining unit 1401 as the reference position, and judges whether the user is continuing to use the camera 100 based on the reference position and the current position of the camera 100.

[0176] The process of the processing performed by the camera 100 according to the present embodiment in order to judge the continued use of the user is the same as that in the first embodiment shown in the flowchart of Figure 5 The flowchart of the processing in the first embodiment is started when the camera 100 is made capable of communicating with the monitoring camera. At the start of the flow, the "predetermined distance" required for the judgment by the continue use judging unit 403 is already recorded in the memory 213 of the camera 100. As in the modification 2 of the seventh embodiment, the "predetermined distance" can be recorded in the memory 213 by Figure 17 The flowchart of the processing in the first embodiment is started when the camera 100 is made capable of communicating with the monitoring camera. At the start of the flow, the "predetermined distance" required for the judgment by the continue use judging unit 403 is already recorded in the memory 213 of the camera 100. As in the modification 2 of the seventh embodiment, the "predetermined distance" can be recorded in the memory 213 by Figure 5 The process of judging that the user P is continuing the use of the camera in step S503 shown. In the present modification, the position of the camera 100 and the user obtained from the monitoring camera is the position information in step S1701.

[0177] In the present modification example, a method for determining the positions of the camera 100 and the user based on an image captured by a monitoring camera is explained as an example of a method for determining the positions of the camera and the user on a common coordinate system. However, the method for determining the positions of the camera 100 and the user is not limited to this example. There is also a method for determining the distance between the camera 100 and the user by preparing a transmitter such as an optical beacon transmitter, determining the positions of the camera 100 and the user from signals received by receivers fixed to the camera 100 and the user, and calculating the distance based on the positions.

[0178] In the method according to the present modification example, even if the camera 100 is not provided with a structure for measuring the position, the authentication of the user is maintained during the capturing of a series of images, thus improving the convenience of the apparatus.

[0179] (Eighth Embodiment)

[0180] Next, the eighth embodiment is explained below. In the seventh embodiment, the continued use of the camera 100 by the user of the camera 100 is determined based on the fact that the camera 100 and the information apparatus held by the user approach each other. In the present embodiment, the continued use of the camera 100 by the user of the camera 100 is judged based on the fact that the position of the camera approaches a reference position which is the position of the camera at the time when the user is recognized. With this method, in the case where the change in the imaging position during the capturing of a series of images is limited, the frequency of the authentication can be reduced. As a result, the convenience of the apparatus is improved. Note that, as in the seventh embodiment, the position of the camera is determined based on the GPS position information.

[0181] Figure 14 The functional units of the camera 100 shown are the same as in the seventh embodiment. In the present embodiment, Figure 14 The information obtaining unit 1401 shown obtains the GPS position information of the camera 100 from the GPS receiving unit 308.

[0182] The continued use judging unit 403 sets the position of the camera 100 obtained by the information obtaining unit 1401 when the user recognizing unit 402 recognizes the user as a reference position, and judges whether the user is continuing to use the camera 100 based on the reference position and the current position of the camera 100.

[0183] The process performed by the camera 100 according to the present embodiment for judging the continued use by the user is the same as Figure 5The process of the processing in the first embodiment shown in the flowchart is the same as that of the processing in the second embodiment shown in the flowchart. At the start of this flow, it is judged using the judging unit 403 whether the "predetermined distance" has been recorded in the memory 213 of the camera 100. This flow starts when the power of the camera 100 is turned on, and is repeated until it becomes no longer possible to continue the processing, for example, due to the power of the camera 100 being turned off. The following describes the processing in the first embodiment with reference to the flowchart shown in FIG. 18. Figure 18 Figure 5 The process of judging that the user P is continuing the use of the camera in step S503 shown in the flowchart.

[0184] First, in step S1801, it is judged whether the position of the camera 100 at the timing when the user is identified by the user identifying unit 402 has been obtained. If the position has been obtained, the processing proceeds to step S1803, and if the position has not been obtained, the processing proceeds to step S1802, and the position of the camera 100 is obtained and set as the reference position. In step S1803, the current position of the camera 100 is obtained.

[0185] In step S1804, it is judged whether the distance between the reference position and the current position is equal to or shorter than the predetermined distance. If the distance is equal to or shorter than the predetermined distance, the processing proceeds to step S1805 and it is judged that the user P is "continuing the use of the camera", and the flow ends. On the other hand, if the distance is longer than the predetermined distance, the processing proceeds to step S1806 and it is judged that the user P is "not continuing the use of the camera", and the flow ends.

[0186] In the method according to the present embodiment, the authentication of the user is maintained during the shooting of a series of images without the need to measure the position of an object other than the device, thus improving the convenience of the device.

[0187] (Variation 1 of the Eighth Embodiment)

[0188] Next, Variation 1 of the eighth embodiment is described below. In the eighth embodiment, the position at the timing when the user is identified is used as the reference position, but in the present variation, the position at the start of the imaging is used as the reference position. If there is a time lag between the timing when the user is identified by the user identifying unit 402 and the start of the imaging, the use of the camera by the user is continued, for example, by the method according to the ninth embodiment described below.

[0189] Figure 14 ​The functional units of the camera 100 shown are the same as in the seventh embodiment. The continuing use judging unit 403 continues to use the position of the camera 100 obtained by the information obtaining unit 1401 at the time when the imaging unit 401 starts imaging as a reference position, and judges whether or not the user is continuing to use the camera 100 based on the reference position and the current position of the camera 100.

[0190] The process of the processing performed by the camera 100 according to the present embodiment in order to judge the continuing use of the user is the same as that of the processing in the first embodiment shown in the flowchart of Figure 5 The flowchart starts when the power of the camera 100 is turned on, and repeats until it becomes no longer possible to continue the processing, for example, because the power of the camera 100 is turned off. At the start of the flowchart, the "predetermined distance" required for the judgment by the continuing use judging unit 403 has already been recorded in the memory 213 of the camera 100.

[0191] As in the eighth embodiment, it is possible to maintain the authentication of the user during the shooting of a series of images even if the camera 100 moves during the period from the time when the user is recognized until the start of imaging, and thus to improve the convenience of the device. Figure 18 Figure 5 The process of judging that the user P is continuing the use of the camera in step S503 shown is the same as that in the eighth embodiment.

[0192] In the method according to the present variant, even if the camera 100 moves during the period from the time when the user is recognized until the start of imaging, the authentication of the user is maintained during the shooting of a series of images, and thus the convenience of the device is improved.

[0193] (Variant 2 of the Eighth Embodiment)

[0194] Next, Variant 2 of the eighth embodiment will be described. In the eighth embodiment and Variant 1, the position at the time when the user is recognized or the position at the start of imaging is used as the reference position, but in the present variant, the position of the user specified on a map having the same coordinate system as the coordinate system of the output current position of the camera 100 is set as the reference position. Note that the map used to specify the reference position can be a map held by the camera 100, or a map held by an information device that communicates with the camera 100.

[0195] Figure 14 The functional units of the camera 100 shown are the same as in the seventh embodiment. The continuing use judging unit 403 includes Figure 14 a reference position setting unit not shown in the seventh embodiment, and judges whether or not the user is continuing to use the camera 100 based on the reference position and the current position of the camera 100.

[0196] The process of the processing performed by the camera 100 according to the present embodiment in order to judge the continuing use of the user is the same as that of the processing in the first embodiment shown in the flowchart of​Figure 5 The process of the process in the first embodiment shown in the flowchart of FIG. At the start of this process, the reference position and the "predetermined distance" have been recorded in the memory 213 of the camera 100. Figure 19 The flowchart shown is used to illustrate the Figure 5 In step S503 shown, it is determined that the user P is continuing to use the camera.

[0197] First, in step S1901, the current position of the camera 100 is obtained. Next, in step S1902, a determination is made as to whether the distance between the reference position and the current position is equal to or shorter than a predetermined distance. If the distance is equal to or shorter than the predetermined distance, the process proceeds to step S1903, where it is determined that user P is "continuing to use the camera," and the flow ends. On the other hand, if the distance is longer than the predetermined distance, the process proceeds to step S1904, where it is determined that user P is "not continuing to use the camera," and the flow ends.

[0198] In the method according to this modification, if the user captures images at a specific location, the authentication of the user is maintained during the capturing of a series of images, thereby improving the convenience of the apparatus.

[0199] (Variation 3 of the Eighth Embodiment)

[0200] Next, Variation 3 of the eighth embodiment will be described. In Variation 2 of the eighth embodiment, a user-specified position on a map having the same coordinate system as that used to output the current position of camera 100 is used as a reference position. A predetermined range relative to this reference position is set as a reference area, and if the current position is within the reference area, the decision to continue use is validated. In this variation, the reference area is set as an area specified by the user on the same map. The reference area can be specified on a map stored by camera 100 or on a map stored on an information device capable of communicating with camera 100.

[0201] Figure 14 The functional units of the camera 100 shown are the same as those in the seventh embodiment. The continued use determination unit 403 includes Figure 14 A reference area setting unit not shown in FIG. 1 is used, and based on the reference area and the current position of the camera 100 , it is determined whether the user is continuing to use the camera 100 .

[0202] The process of the camera 100 according to this embodiment to determine the user's continued use is similar to the process of Figure 5The process of the process in the first embodiment shown in the flowchart of FIG. At the beginning of this process, the reference area for judging the continued use of the user has been recorded in the memory 213 of the camera 100. As in the modification 1 of the eighth embodiment, the reference area for judging the continued use of the user has been recorded in the memory 213 of the camera 100. Figure 19 The flow chart shows the Figure 5 In step S503 shown, it is determined that the user P is continuing to use the camera. In this modification, it is determined in step S1902 whether the current position of the camera 100 is within the reference area.

[0203] In the method according to this modification, if the user captures an image within a specific area, authentication of the user is maintained during the capturing of a series of images, thereby improving the convenience of the apparatus.

[0204] (Ninth embodiment)

[0205] Next, the ninth embodiment will be described. While the seventh and eighth embodiments determine continued use based on the camera 100's location information, this embodiment uses time information to determine continued use. The time at which the user is identified is set as a reference time, and if the current time is within a predetermined period from the reference time, the user is determined to be continuing to use the camera. This method reduces the frequency of authentication, resulting in improved device convenience.

[0206] Figure 14 The functional units of the camera 100 shown are the same as those in the seventh embodiment. In this embodiment, Figure 14 The illustrated information obtaining unit 1401 obtains the current time of the camera 100 from the timer 309 .

[0207] The continued use determination unit 403 sets the time obtained by the information obtaining unit 1401 when the user identification unit 402 identifies the user as a reference time, and determines whether the user is continuing to use the camera 100 based on the reference time and the current time of the camera 100 .

[0208] The process of the camera 100 according to this embodiment to determine the user's continued use is similar to the process of Figure 5 The process is the same as that of the first embodiment shown in the flowchart of FIG. At the start of this process, the "predetermined period" required to continue the judgment using the judgment unit 403 has been recorded in the memory 213 of the camera 100. This process starts when the power of the camera 100 is turned on, and is repeated until it becomes impossible to continue the processing, for example, due to the power of the camera 100 being turned off. Figure 20 The flowchart shown is used to illustrate the Figure 5 In step S503 shown, it is determined that the user P is continuing to use the camera.

[0209] First, in step S2001, it is judged whether the time of the camera 100 at which the user recognizing unit 402 recognized the user has been obtained. If the time has been obtained, the process proceeds to step S2003, and if the time has not been obtained, the process proceeds to step S2002, and the current time of the camera 100 is obtained and set as the reference time. In step S2003, the current time of the camera 100 is obtained.

[0210] In step S2004, it is judged whether the period between the reference time and the current time is equal to or shorter than a predetermined period. If the period is equal to or shorter than the predetermined period, the process proceeds to step S2005 and judges that the user P is "continuing the use of the camera", and the flow ends. On the other hand, if the period is longer than the predetermined period, the process proceeds to step S2006 and judges that the user P is "not continuing the use of the camera", and the flow ends.

[0211] In the method according to the present embodiment, if a series of images are shot within a limited period, the authentication of the user is maintained during the shooting of the series of images, and thus the convenience of the device is improved.

[0212] (Variation 1 of the ninth embodiment)

[0213] Next, Variation 1 of the ninth embodiment will be described. In the ninth embodiment, the time at which the user is recognized is used as the reference time, but in the present variation, the time at which the imaging is started is used as the reference time. If there is a time lag between the time at which the user recognizing unit 402 recognized the user and the start of the imaging, the use of the camera by the user is continued, for example, by the method according to the eighth embodiment described above.

[0214] Figure 14 The functional units of the camera 100 shown are the same as in the seventh embodiment, and thus the description thereof is omitted. In the present embodiment, Figure 14 The information obtaining unit 1401 shown obtains the current time of the camera 100 from the timer 309.

[0215] The continued use judging unit 403 sets the time obtained by the information obtaining unit 1401 at the time when the imaging unit 401 starts the imaging as the reference time, and judges whether the user is continuing the use of the camera 100 based on the reference time and the current time of the camera 100.

[0216] The process of the camera 100 according to the present embodiment for judging the continued use by the user is the same as that of the camera 100 according to the seventh embodiment, and thus the description thereof is omitted. Figure 5The process of the processing in the first embodiment shown in the flowchart of FIG. At the beginning of this process, the "predetermined period" required to continue to use the judgment unit 403 for judgment has been recorded in the memory 213 of the camera 100. As in the ninth embodiment, Figure 20 The flow chart shows the Figure 5 In step S503 shown in the figure, it is determined that the user P is continuing to use the camera. The reference time in this flow is the time when the image capture starts.

[0217] In the method according to the present modification, even if the camera 100 moves during the period from when the user is recognized until when image capture starts, user authentication is maintained during capturing a series of images, thus improving the convenience of the apparatus.

[0218] (Variation 2 of the Ninth Embodiment)

[0219] Next, a second variation of the ninth embodiment will be described. In the ninth embodiment, if the current time is within a predetermined period from the reference time, it is determined that the user is continuing to use the camera. However, in the present embodiment, if the current time is not later than the end time set by the user in the camera 100, it is determined that the user is continuing to use the camera.

[0220] Figure 14 The functional units of the camera 100 shown are the same as those in the seventh embodiment. In this embodiment, Figure 14 The illustrated information obtaining unit 1401 obtains the current time of the camera 100 from the timer 309 .

[0221] Continue using the judgment unit 403 includes Figure 14 The reference time setting unit (not shown) is used, and whether the user is continuing to use the camera 100 is determined based on the reference time and the current time of the camera 100. Here, the reference time is the time when the imaging plan ends.

[0222] The process of the camera 100 according to this embodiment to determine the user's continued use is similar to the process of Figure 5 The process of the processing in the first embodiment shown in the flowchart of FIG. When this process starts, the end time has been recorded in the memory 213 of the camera 100. Figure 21 The flowchart shown is used to illustrate the Figure 5 In step S503 shown, it is determined that the user P is continuing to use the camera.

[0223] First, in step S2101, the current time of the camera 100 is obtained. Next, in step S2102, it is determined whether the current time is the reference time (planned end time) or earlier than the reference time. If the current time is the end time or earlier than the end time, the process proceeds to step S2103 and it is determined that the user P is "continuing use of the camera", and the flow ends. On the other hand, if the current time is later than the reference time (planned end time), the process proceeds to step S2104 and it is determined that the user P is "not continuing use of the camera", and the flow ends.

[0224] In the method according to the present modification example, if the time at which the photographing of the series of images will end is known, the authentication of the user is maintained during the photographing of the series of images, and thus the convenience of the apparatus is improved.

[0225] (Tenth Embodiment)

[0226] Next, the tenth embodiment will be described. In the seventh and eighth embodiments, the continuation of use is determined based on the position information of the camera 100, and in the ninth embodiment, the continuation of use is determined based on the time information of the camera 100, but in the present embodiment, the continuation of use is determined based on both the position information and the time information.

[0227] Figure 14 The functional units of the camera 100 shown are the same as in the seventh embodiment. In the present embodiment, Figure 14 The information obtaining unit 1401 shown obtains the GPS position information of the camera 100 from the GPS receiving unit 308, and obtains the time information from the timer 309.

[0228] The continuation of use determining unit 403 sets the position obtained by the information obtaining unit 1401 at the time when the user is recognized by the user recognizing unit 402 as the reference position. Further, the continuation of use determining unit 403 sets the time obtained by the information obtaining unit 1401 at the time when the user is recognized by the user recognizing unit 402 as the reference time. Whether the user is continuing to use the camera 100 is determined based on the reference position, the current position of the camera 100, the reference time, and the current time of the camera 100.

[0229] The process of the processing performed by the camera 100 according to the present embodiment in order to determine the continuation of use by the user is the same as the process of the processing in the first embodiment shown in the flowchart of Figure 5 The process of the processing performed by the camera 100 according to the present embodiment in order to determine the continuation of use by the user is the same as the process of the processing in the first embodiment shown in the flowchart of Figure 22the flowchart shown to explain a method for determining whether or not a user is continuing to use a camera Figure 5 In step S503 shown, it is determined that the user P is continuing to use the camera.

[0230] First, in step S2201, it is determined whether or not the position of the camera 100 at the timing when the user is recognized by the user recognition unit 402 has been obtained. If the position has been obtained, the process proceeds to step S2203, and if the position has not been obtained, the process proceeds to step S2202, and the position of the camera 100 is obtained and set as a reference position.

[0231] In step S2203, the current position of the camera 100 is obtained. In step S2204, it is determined whether or not the distance between the reference position and the current position is equal to or shorter than a predetermined distance. If the distance is equal to or shorter than the predetermined distance, the process proceeds to step S2205. If the distance is longer than the predetermined distance, the process proceeds to step S2210 and it is determined that the user P is "not continuing to use the camera", and the flow ends.

[0232] In step S2205, it is determined whether or not the time of the camera 100 at the timing when the user is recognized by the user recognition unit 402 has been obtained. If the time has been obtained, the process proceeds to step S2207, and if the time has not been obtained, the process proceeds to step S2206, and the time of the camera 100 is obtained and set as a reference time.

[0233] In step S2207, the current time of the camera 100 is obtained. In step S2208, it is determined whether or not the period between the reference time and the current time is equal to or shorter than a predetermined period. If the period is equal to or shorter than the predetermined period, the process proceeds to step S2209 and it is determined that the user P is "continuing to use the camera", and the flow ends. On the other hand, if the period is longer than the predetermined period, the process proceeds to step S2210 and it is determined that the user P is "not continuing to use the camera", and the flow ends.

[0234] In the method according to the present embodiment, even if it is not possible to predict which of the position of the camera 100 and the time for which the camera is used will be restricted, the authentication of the user is maintained during the shooting of a series of images, and thus the convenience of the apparatus is improved.

[0235] (Eleventh Embodiment)

[0236] In the present embodiment, an example of judging the continuation of use by using a captured image will be described. More specifically, the user of the camera at the time of imaging (i.e., the photographer) is identified by using the similarity between images. The similarity between images in the present embodiment is an index indicating whether the images themselves are similar to each other, and is also affected by the timing of the captured image and the condition of the captured image. The similarity between images is used mainly for the following two reasons. First, it is considered that the intended subject of the images taken by the photographer at the same time is basically the same as each other in many cases regardless of whether the authentication is successful or not. Second, even if the subjects of the images are different from each other, it is considered that the user uses similar settings and intends to create similar images when the user takes images of similar scenes.

[0237] In Figure 23 A functional configuration example of the camera 100 related to the processing described below is shown in a block diagram. The camera 100 includes an imaging unit 401, an identification unit 402, a calculation unit 2301, a continuation of use judging unit 403, and a recording unit 404, and these units function when the CPU 212 electrically controls the respective units of the camera 100.

[0238] If the user identification unit 402 fails to identify the user, the calculation unit 2301 calculates the similarity between the image captured by the imaging unit 401 and one or more images of the images recorded in the recording unit 404 that have been associated with the photographer information. If the user identification unit 402 has successfully identified the user, the calculation unit 2301 calculates the similarity between the image captured by the imaging unit 401 and one or more images of the images recorded in the recording unit 404 that have not been associated with the photographer information.

[0239] If any of the similarities calculated by the calculation unit 2301 is higher than a predetermined threshold value determined in advance, the continuation of use judging unit 403 judges that the two or more images used for the similarity calculation are captured by the same photographer.

[0240] The process of the processing from when the camera 100 according to the present embodiment starts imaging until when the photographer information is associated with the image will be described below with reference to the flowchart shown in Figure 24 In step S2401, the imaging unit 401 obtains an image by converting a captured image signal received from the imaging element 211 into image information.

[0241] In step S2402, the user identifying unit 402 obtains a feature of the eyeball 220 of the user who photographed the image by the eye imaging element 219. More specifically, a feature is extracted from an eye image (an electric signal of an eye image; an electric signal of an eye image) obtained from the eye imaging element 219 via the line-of-sight detection circuit 301. For feature extraction, a neural network is used. For example, a convolutional neural network (CNN), which is one of neural networks, is used. In the CNN, abstract information is extracted from an input image by repeatedly performing processing including convolution processing, activation processing, and pooling processing on the input image a plurality of times. At this time, a processing unit constituted by the convolution processing, the activation processing, and the pooling processing is generally referred to as a layer. There are many known methods used as the activation processing, and a method called rectified linear unit (ReLU), for example, can be used. Further, there are many known methods used as the pooling processing, and a method called max pooling, for example, can be used. For example, ResNet introduced in Non-Patent Literature (K. He, X. Zhang, S. Ren, and J. Sun. Identity mappings in deep residual networks. In ECCV, 2016) can be used as the configuration of the CNN. A neural network known as a visual transformer (ViT) described in Non-Patent Literature (Alexey Dosovitskiy, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR, 2021) can also be used. The configuration of the neural network is not limited to these examples. Note that information such as the configuration and the weight of the neural network is held by the memory 213. The weight of the neural network used is obtained in advance by learning. For example, eye images of various persons are obtained in advance for learning, and learning is performed with a method such as ArcFace described in Non-Patent Literature (J. Deng, J. Guo, N. Xue, and S. Zafeiriou. ArcFace: Additive angular margin loss for deep face recognition. In CVPR, 2019) or the like.

[0242] Although a method using a neural network is described as an example of a method for authenticating a person by using an eye image, a publicly known method known as iris recognition (for example, Japanese Patent 3307936), for example, can also be used. The method for authenticating a person is not limited to these examples.

[0243] In step S2403, the user identifying unit 402 compares the feature obtained in step S2402 with the features of the users (i.e., the registered persons) recorded in advance in the storage 213, and determines the degrees of similarity. In the present embodiment, the degrees of similarity (i.e., cosine similarities) are calculated by normalizing the features and calculating inner products, but other methods can also be used. For example, a neural network trained to calculate a degree of similarity from an input feature, pattern matching, frequency analysis, Euclidean distance, Manhattan distance, or the like can also be used, and the method is not limited to a particular method. If the feature is not successfully obtained in step S2402, no degree of similarity is calculated in this step.

[0244] In step S2404, the user identifying unit 402 determines whether any of the degrees of similarity calculated in step S2403 is higher than a predetermined threshold value determined in advance. If any of the degrees of similarity is higher than the threshold value, the processing proceeds to step S2405, otherwise, the processing proceeds to step S2411.

[0245] In step S2405, the user who took the image obtained by the imaging unit 401 in step S2401 is determined to be the same as the registered person corresponding to the highest degree of similarity using the determination unit 403.

[0246] In step S2406, the recording unit 404 confirms whether the images recorded in the storage 213 include one or more images not associated with the photographer information. If the images include such an image, the processing proceeds to step S2407, otherwise, the processing proceeds to step S2410.

[0247] In step S2407, the calculation unit 2301 calculates the degree of similarity between the image obtained by the imaging unit 401 in step S2401 and one or more images confirmed by the recording unit 404 in step S2406 as images obtained by the imaging unit 401 in the past and not associated with the photographer information. Here, features are extracted from each image, and the degree of similarity is calculated using a method similar to that used in step S2403. Features are extracted from each image using a known neural network, and features related to the subject itself are obtained. If the subjects themselves are similar to each other, the features are close to each other, and otherwise, the features are distant from each other. Although features related to the subject itself are obtained in this example, features related to the orientation, arrangement, or motion of the subject, or the composition or background information of the image can also be obtained. A combination of these features can also be used. Furthermore, a method using a neural network is described as an example of a method for extracting features, but a known feature extraction method such as HoG or SIFT can also be used. Furthermore, a method using feature extraction and inner product by a neural network is described as an example of a method for calculating the degree of similarity between images, but for example, a method using the squared error of pixel values or the Kullback-Leibler divergence of pixel value histograms can also be used. In addition, metadata added when each image is captured, such as the time at which the image was captured, the place at which the image was captured, the shutter speed, the ISO sensitivity, the aperture value, the focal length, the lens specification, and the like, can also be used to adjust the calculated degree of similarity (for example, if the difference between the times at which each image was captured is large, the degree of similarity between the images can be made lower than in the case where the difference is small). Alternatively, the degree of similarity can also be calculated based only on these types of metadata (for example, if the difference between the times at which each image was captured is 1 second, the degree of similarity can be determined to be 0.99, and if the difference is 10 seconds, the degree of similarity can be determined to be 0.8). It is sufficient to calculate the degree of similarity by comprehensively considering the information obtained from the images and the conditions under which the images were captured using a function f represented by the following mathematical expression 1 such that the degree of similarity is inversely proportional to the difference between the images themselves or the metadata of each image.

[0248] (Mathematical expression 1)

[0249] Degree of similarity (between images) = f(I1, I2)

[0250] Here, I1 represents information obtained from an image in which a photographer has not been identified (information that can be extracted from pixels (information obtained by image recognition (the subject itself, the orientation of the subject, the movement of the subject, the composition, background information, and features thereof), pixel values, a histogram), or metadata added to the image (the time at which the image was captured, the place at which the image was captured, the shutter speed, the ISO sensitivity, the aperture value, the focal length, the lens specification)), and I2 represents information obtained from N (N≥1) images in which the photographer has been identified. The function f can be obtained by manually determining the weights of the respective explanatory variables or by learning using a known machine learning method such as a neural network or a random forest. Although the degree of similarity is calculated by using a single function in Mathematical Expression 1, a plurality of similarity functions can be used in combination. The degree of similarity can be in any form, and can be calculated using any method as long as it is possible to judge whether or not two images were captured by the same user based on the degree of similarity. The present embodiment is not limited to the method for extracting features and the method for calculating the degree of similarity between images.

[0251] In step S2409, it is judged by the judging unit 403 that the user who captured all the images for which it was judged in step S2408 that the degree of similarity was higher than the threshold value is the same as the user who captured the image obtained by the imaging unit 401 in step S2401.

[0252] In step S2410, the recording unit 407 associates the photographer information with each image for which the user was newly identified in step S2405, S2409, or S2414 in a manner in which the photographer information cannot be tampered with, and then ends the processing. As a method for associating the photographer information in a manner in which the photographer information cannot be tampered with, a method using C2PA described in Non-Patent Literature (Coalition for Content Provenance and Authenticity (C2PA), “C2PA Specifications”, <Technical Specifications Version 1.2>, [online], November 3, 2022 [retrieved January 23, 2023], Internet <URL: https: / / c2pa.org / specifications / specifications / 1.2 / specs / C2PA_Specification.html>) is used. C2PA is a method for adding metadata indicating the content of editing performed on an image to the image to authenticate the origin of the image, the progress of the event, or the history of the image. Thus, the photographer information can also be recorded in accordance with C2PA. However, the photographer information can also be recorded by using other methods. Alternatively, the image and the photographer information can also be recorded in different files. Alternatively, the photographer information can also be managed in a database. The method for recording the photographer information is not limited to these methods.

[0253] In step S2411, the recording unit 404 confirms whether the images recorded in the memory 213 include one or more images that have already been associated with the photographer information. If the images include such an image, the processing proceeds to step S2412, otherwise, the processing ends (as necessary, information indicating that the photographer was not identified is associated with the image).

[0254] In step S2412, the calculation unit 2301 calculates the degree of similarity between the image obtained by the imaging unit 401 in step S2401 and N (N ≥ 1) images that are confirmed by the recording unit 404 in step S2411 as having been associated with the photographer information. The degree of similarity is calculated using the method described with respect to step S2407.

[0255] In step S2413, the determination unit 403 continues to determine whether any of the degrees of similarity calculated by the calculation unit 2301 in step S2412 is higher than a predetermined threshold value determined in advance. If any of the degrees of similarity is higher than the threshold value, the processing proceeds to step S2414, otherwise, the processing ends (as necessary, information indicating that the photographer was not identified is associated with the image).

[0256] In step S2414, it is continued to judge by the continuation use judging unit 403 that the photographer of the image corresponding to the highest degree of similarity in the image judged in step S2413 to have a degree of similarity higher than the threshold value is the same as the photographer of the image obtained by the imaging unit 401 in step S2401.

[0257] According to the above configuration, even if the photographer's attention is directed to imaging rather than authentication at the time the user takes an image (for example, a perfect opportunity suddenly occurs during the user is taking an image of a sports game or a wild bird), and thus information suitable for the authentication of the photographer (such as an image of the eye peering through the viewfinder) cannot be obtained, the photographer can be identified (that is, the information of the photographer is associated with the taken image), and the convenience of the device that authenticates and records the photographer is improved.

[0258] (Variation 1 of the Eleventh Embodiment)

[0259] In the above example, the imaging unit 401 obtains an image by converting a taken image signal received from the imaging element 211 into image information. However, a moving image can be obtained instead of an image. In this case, the calculating unit 2301 can calculate the degree of similarity by using a specific frame extracted from the moving image or all frames of the moving image, and take an average value thereof as the final degree of similarity.

[0260] (Variation 2 of the Eleventh Embodiment)

[0261] In the above example, the recording unit 404 associates the photographer information and the image judged by the continuation use judging unit 403 to have the same photographer as another image in a manner in which the photographer information cannot be tampered with. However, the photographer information can be associated with an image other than such an image. More specifically, it can be judged that an image taken between two images judged to have the same photographer is also taken by the same photographer, and the information of the same photographer can be associated with the image.

[0262] (Variation 3 of the Eleventh Embodiment)

[0263] In the above-described embodiments, the user identifying unit 402 obtains the feature of the eyeball 220 of the user. However, it is also possible to obtain the feature from something other than the eyeball 220. For example, the following configuration is also possible: another camera is installed on the camera 100 (for example, above the operation member 103), and the feature is extracted from an image of the face of the user obtained by using the camera. Alternatively, it is also possible to extract the feature in an iris image extracted from an eye image of the user. Alternatively, the following configuration is also possible: a microphone is installed on the camera 100 (for example, above the operation member 103), and the feature is extracted from voiceprint information of the user obtained by using the microphone. Alternatively, the following configuration is also possible: a fingerprint sensor is installed on the camera 100 (for example, on the release button 101 or the operation member 103), and the feature is extracted from fingerprint information of the user obtained by using the sensor. Alternatively, the following configuration is also possible: a vein sensor is installed on the camera 100 (on the release button 101), and the feature is extracted from vein information of the user obtained by using the sensor. Alternatively, the following configuration is also possible: the feature is extracted from an animation showing the movement of the eyeball 220 of the user, which is captured in time series by the eye imaging element 219. Alternatively, the following configuration is also possible: the feature is extracted from information of an operation unique to the user, which is obtained via the operation members 103, 104, or 105.

[0264] (Variation 4 of the eleventh embodiment)

[0265] In the above-described example, the user identifying unit 402 determines the degree of similarity by using the feature, and identifies the user. However, it is also possible to identify the user without using the feature and the degree of similarity. For example, it is also possible to identify the user by having the user input a password or a PIN code, or the like. Alternatively, the following configuration is also possible: the camera 100 is configured to be able to be paired with a smartphone by a Bluetooth connection or the like, and when the Bluetooth connection is established, the result of personal authentication by the smartphone is shared with the camera 100. It is also possible to identify the user by using a system such as FIDO authentication.

[0266] (Variation 5 of the eleventh embodiment)

[0267] In the above example, the user is identified at the time of imaging by the user identification unit 402. However, the user can also be identified in two steps, that is, at a time when the camera is not being used for imaging and at the time of imaging. The advantage of two-step identification is that if the registered person who is likely to take the image is extracted in advance from a plurality of registered persons based on strict conditions at a time when the camera is not being used for imaging, it is sufficient to judge only whether the user is the same as the registered person under relatively loose conditions at the time of imaging, and thus the accuracy of the identification at the time of imaging can be improved. Furthermore, the user is compared with a single registered person at the time of imaging, and thus the processing time (which is the sum of the time taken to perform biometric authentication and the time taken to calculate the degree of similarity between images) can be reduced.

[0268] (Variation 6 of the eleventh embodiment)

[0269] In the above example, the user cannot notice at the time of imaging that the photographer information is associated with the image by the recording unit 404. In this regard, the camera 100 can also be provided with a configuration for notifying the user at the time of imaging that the photographer information is associated with the image. More specifically, an indication such as an indication shown in FIGS. 21A, 21B, and 21C can also be displayed on the display panel of the eyepiece 102. Figure 25A 、 Figure 25B and Figure 25C are shown below. Figure 25A 、 Figure 25B and Figure 25C are shown below.

[0270] Figure 25A : Display example of a case where the photographer of the current image is identified based on the eye image, but the photographer of the past image is not identified based on the degree of similarity between images.

[0271] Figure 25B : Display example of a case where the photographer of the current image is identified based on the eye image, and the photographer of the past image is also identified based on the degree of similarity between images.

[0272] Figure 25C : Display example of a case where the photographer of the current image is identified based on the degree of similarity between images.

[0273] These correspond to the following cases in the above processing flow.

[0274] Figure 25A : "Yes" in step S2404, but "No" in steps S2406 or S2408.

[0275] Figure 25B : "Yes" in all of steps S2404, S2406, and S2408.

[0276] Figure 25CYES in steps S2404, but YES in steps S2411 and S2413.

[0277] With this configuration, the user can notice that the photographer information is associated with the image at the time of imaging, which improves the efficiency of imaging. However, this is merely an example, and it is also possible to give the user a notification by using other methods. For example, the following configuration is also possible: a display similar to the one illustrated in FIG. 26 is displayed on the operation member 103, and the display is changed to the display similar to the one illustrated in FIG. 27 at the time of associating the photographer information with the image. Alternatively, the following configuration is also possible: a display similar to the one illustrated in FIG. 26 is displayed on the operation member 103, and a display similar to the one illustrated in FIG. 27 is displayed on the display 201 at the time of associating the photographer information with the image. Figure 25A 、 Figure 25B and Figure 25C the display similar to the one illustrated in FIG. 27 is displayed on the display 201 at the time of associating the photographer information with the image. Alternatively, the following configuration is also possible: a speaker is provided at an appropriate position on the camera 100 (for example, above the operation member 103), and audio corresponding to the association state is output from the speaker at the time of associating the photographer information with the image. Alternatively, the following configuration is also possible: the camera 100 is configured to be able to be paired with an external device such as a smartphone or a smartwatch by a Bluetooth connection or the like, and the external device gives a notification corresponding to the association state at the timing of associating the photographer information with the image.

[0278] (Twelfth Embodiment)

[0279] In the eleventh embodiment, a method of identifying a user by using the degree of similarity between images at the time of imaging is explained. However, there are cases where it is desired to identify a user by using the degree of similarity between images after imaging has ended. Examples of such cases include a case where it is necessary to switch to the shooting of other images immediately after imaging has ended, and a case where the images recorded in the memory 213 after imaging are transferred to the memory of an external information processing device via a portable medium such as an SD card. In the present embodiment, a method of identifying a user by using the degree of similarity between images in such a case is explained.

[0280] In the block diagram of Figure 26 , an example of the functional structure of the camera 100 (or another information processing device) related to the processing described below is illustrated. The camera 100 includes an image selection unit 2601, the calculation unit 2301, a continuation use judgment unit 403, and a recording unit 404, and these units function when the CPU 212 electrically controls the units of the camera 100.

[0281] The image selection unit 2601 selects one or more images that have been associated with photographer information and one or more images that have not been associated with photographer information from among the images recorded in the memory 213. The calculation unit 2301 calculates the degree of similarity between the images that have been associated with photographer information and the images that have not been associated with photographer information, which have been selected by the image selection unit 2601.

[0282] Reference is made below toFigure 27 The flowchart shown illustrates the procedure of the processing performed by the camera 100 (or another information processing apparatus) in order to associate the photographer information with the image according to the present embodiment.

[0283] In step S2701, the image selection unit 2601 confirms whether the images recorded in the storage 213 include one or more images that are not associated with the photographer information. If the images include such images, the processing proceeds to step S2702, otherwise, the processing ends.

[0284] In step S2702, the image selection unit 2601 confirms whether the images recorded in the storage 213 include one or more images that have been associated with the photographer information. If the images include such images, the processing proceeds to step S2703, otherwise, the processing ends.

[0285] In step S2703, the image selection unit 2601 selects, based on the input information obtained from the operation member 103, 104, or 105, M (M≥1) images to be used for identifying the photographer from among the images confirmed in step S2701 as not being associated with the photographer information.

[0286] In step S2704, the image selection unit 2601 selects, based on the input information obtained from the operation member 103, 104, or 105, N (N≥1) images that are the same as the photographer from among the images confirmed in step S2702 as being associated with the photographer information.

[0287] In step S2705, the calculation unit 2301 calculates the degree of similarity between the M images selected by the image selection unit 2601 in step S2703 and the N images selected by the image selection unit 2601 in step S2704. The method described with respect to step S2407 is applied to the M images in turn (M times) to calculate the degrees of similarity.

[0288] In step S2706, the determination unit 403 continues to determine whether any of the M degrees of similarity calculated by the calculation unit 2301 in step S2705 is higher than a predetermined threshold value determined in advance. If any of the degrees of similarity is higher than the threshold value, the processing proceeds to step S2707, otherwise, the processing ends.

[0289] In step S2707, the determination unit 403 determines that the photographers of all the images that are not associated with the photographer information and that correspond to the X (X≤M) degrees of similarity determined to be higher than the threshold value in step S2706 are the same as the photographer of the N images.

[0290] In step S2708, the recording unit 404 associates the photographer information with the image in which the photographer has been newly recognized in step S2707 in a manner in which the photographer information cannot be tampered with, and then ends the process.

[0291] The above-described configuration makes it possible to associate the photographer information with the image (authenticate the photographer) even when the camera is not being used for image capturing, and improves the convenience of the device that authenticates and records the photographer.

[0292] The numerical values, processing timings, processing sequences, subjects of the processing, configurations of data (information), methods for obtaining data, transmission destinations and origins of data, storage locations of data, and the like described in the above-described embodiments and modified examples are examples given for the purpose of concretely describing the present application, and are not intended to limit the present application to these examples.

[0293] The above-described embodiments and modified examples can be appropriately combined, in part or in whole, with each other. Furthermore, some or all aspects of the above-described embodiments and modified examples can be selectively used.

[0294] Other Embodiments

[0295] Embodiments of the present application can also be realized by a computer- readable recording medium having recorded thereon a program for implementing the functions of the above-described embodiments, the program being readable by a computer (CPU or MPU) of a system or an apparatus, which functions as the program causes the computer to function. The computer-readable recording medium includes a magnetic recording medium, an optical recording medium, and a carrier wave.

[0296] While the present application has been described with reference to exemplary embodiments, it is to be understood that the application is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be construed as encompassing all such modifications and equivalents as fall within the true spirit and scope of the present application.

Claims

1. A camera device, comprising: an identification unit configured to identify a user of the camera device; a determination unit configured to determine a status of continued use of the camera device by the user; as well as A control unit is configured to control so that the images captured by the camera device between the time when the recognition unit recognizes the user and the time when the judgment unit judges that the user's use of the camera device is not continuing are recorded in the memory in association with the user's information.

2. The imaging device according to claim 1, in, The memory is included in the imaging device or is physically detachably connected to the imaging device.

3. The imaging device according to claim 1, in, The camera is connected to a server including the memory via a communication path, and The control unit transmits a signal to the server according to a result of the determination, so that the image and the user's information are recorded in the memory in association with each other.

4. The imaging device according to claim 1, in, The identification unit identifies a user of the imaging device based on biometric information of the user of the imaging device.

5. The imaging device according to claim 4, in, The biometric information includes information of at least one of iris, fingerprint, vein and face.

6. The imaging device according to claim 1, in, The identification unit identifies a user of the imaging apparatus based on authentication information input by the user of the imaging apparatus or read from a recording medium.

7. The imaging device according to claim 1, in, The determination unit determines that use of the imaging apparatus by the user is continuing during a period from when the recognition unit recognizes the user until when an instruction to end the imaging apparatus is given.

8. The imaging device according to claim 1, in, When it is determined that the user is gripping the grip portion of the imaging device, the determination unit determines that the user is continuing to use the imaging device.

9. The imaging device according to claim 1, in, The determination unit determines that use of the imaging device by the user is continuing when it is determined that the orientation of the imaging device is changing.

10. The imaging device according to claim 1, in, The determination unit determines that use of the imaging device by the user is continuing when movement of the imaging device is detected.

11. The imaging device according to claim 1, in, When the imaging device is located within a specific geographical range, the determination unit determines that the user is continuing to use the imaging device.

12. The imaging device according to claim 1, in, When the recognition unit fails to recognize the user of the imaging device, the control unit controls so that the image captured by the imaging device is recorded in the memory in association with information indicating that the user of the imaging device was not recognized.

13. The camera device according to claim 1, further comprising: a movement obtaining unit configured to obtain movement information of the camera device, The determining unit determines whether the user continues to use the camera device based on the movement information obtained by the movement obtaining unit.

14. The imaging device according to claim 13, in, The movement obtaining unit obtains the movement information based on a value output from an acceleration sensor and / or an angular velocity sensor.

15. The imaging device according to claim 13, in, The judgment unit judges a period during which the imaging device is moving in the same manner as the movement of the imaging device at the time when the recognition unit recognizes the user as a period during which the user is continuing to use the imaging device.

16. The imaging device according to claim 1, further comprising: an obtaining unit configured to obtain either or both of information indicating a position of the imaging device and information indicating a time, The determining unit determines whether the user is continuing to use the camera based on the information obtained by the obtaining unit.

17. The imaging device according to claim 16, further comprising: a position setting unit configured to set a reference position, The determining unit determines whether the user is continuing to use the imaging device based on a distance between the position of the imaging device obtained by the obtaining unit and a reference position set by the position setting unit.

18. The imaging device according to claim 1, further comprising: an image similarity calculation unit configured to calculate similarities between information obtained from images, In which, when the recognition unit fails to recognize the user, the judgment unit judges the status of the user's continued use of the camera device based on the calculation results of the image similarity calculation unit on images not associated with the photographer information and images associated with the photographer information.

19. The imaging device according to claim 18, in, The image similarity calculation unit uses any one or a combination of information extracted from pixels and metadata added to the image as the information obtained from the image.

20. The imaging device according to claim 18, in, The image similarity calculation unit calculates similarities between information obtained from a part or all of frames of a moving image captured by the imaging device.

21. The imaging device according to claim 1, further comprising: a notification unit configured to notify the user that the photographer information is associated with the image, if the control unit associates the photographer information with the image.

22. An information processing device comprising: a recording unit configured to record an image; a selection unit configured to select at least two images from the images recorded by the recording unit; an image similarity calculation unit configured to calculate similarities between information obtained from the images selected by the selection unit; a determination unit configured to determine whether photographers of the images are the same based on a calculation result of the image similarity calculation unit; as well as A control unit is configured to associate the photographer information with the image not associated with the photographer information based on the determination result of the determination unit.

23. A method for controlling a camera device, the method comprising: identifying a user of the camera device; determining a status of continued use of the camera device by the user; as well as Control is performed so that the images captured by the camera device between the time when the user is identified in the identification and the time when it is determined in the judgment that the user's use of the camera device is not continuing are recorded in the memory in association with the user's information.

24. A method for controlling a camera device, wherein the camera device is configured to capture images, the method comprising: identifying a user of the camera device; obtaining either or both of information indicating a position of the imaging device and information indicating a time; determining, based on the obtained information, whether the user is continuing to use the camera device; as well as Based on the result of the determination, the user information is recorded in association with the captured image. 25 . A computer-readable storage medium storing a program for causing a computer to execute the control method according to claim 23 or 24 .

26. A computer program product comprising a program for causing a computer to execute the control method according to claim 23 or 24.

Images