Imaging apparatus and method of controlling imaging apparatus

Abstract

To provide an imaging apparatus which enables a user to previously confirm whether a desired image is photographed or not in automatic photographing based on a photographing instruction from the user.SOLUTION: The imaging apparatus includes a reception means for receiving a photographing instruction by words, an acquisition means for acquiring a reference image on the basis of the words of the photographing instruction, a notification means for presenting the reference image to a user, and a control means for performing control to perform photographing on the basis of the reference image. The acquisition means estimates a feature of an image to be photographed on the basis of a word of the photographing instruction and acquires the reference image showing the estimated feature.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to an imaging device and a control method for an imaging device. [Background technology]

[0002] In recent years, automatic cameras that periodically and continuously capture images of detected subjects without user operation have been developed, and automatic cameras are becoming increasingly common. Since automatic cameras determine whether to capture images themselves, rather than through user operation, there are cases where the images or videos the user expects are not captured. If the images or videos the user expects are not captured, it is desirable for the automatic camera to accept a capture instruction from the user and reflect the capture instruction in subsequent automatic captures.

[0003] Patent Document 1 discloses an image capturing device that automatically determines the appropriate timing for capturing an image by recognizing a smile from a subject included in the image. A user can set a smile detection level for the subject, which is a capturing condition. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2008-311819 Summary of the Invention [Problem to be solved by the invention]

[0005] Even if the smile detection level can be set, the user cannot know the level of the smile that corresponds to the set level until they check the automatically captured image. Also, if the desired image is not captured, it is difficult for the user to determine whether the automatic camera misinterpreted the capture command, or whether the capture command was correctly interpreted but there was no opportunity to capture the image.

[0006] An object of the present invention is to provide an imaging device that allows a user to check in advance whether a desired image will be captured during automatic imaging based on a user's instruction to capture an image. [Means for solving the problem]

[0007] The imaging device of the present invention is characterized by having a receiving means for receiving verbal shooting instructions, an acquisition means for acquiring a reference image based on the verbal shooting instructions, a notification means for presenting the reference image to a user, and a control means for controlling the imaging device to perform the imaging based on the reference image. [Effects of the Invention]

[0008] According to the present invention, in automatic photography based on a photography instruction from a user, the user can confirm in advance whether or not a desired image will be captured. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of an automatic photography system according to a first embodiment. [Figure 2] FIG. 1 is a diagram illustrating an example of the hardware configuration of an automatic photography system according to a first embodiment. [Figure 3A] FIG. 2 is a diagram illustrating an example of the functional configuration of a feature estimation server according to the first embodiment. [Figure 3B] FIG. 2 is a diagram illustrating an example of the functional configuration of a reference image generation server according to the first embodiment. [Figure 3C] 1 is a diagram illustrating an example of the functional configuration of an automatic photography camera according to a first embodiment. [Figure 4] 10A and 10B are diagrams illustrating examples of prompts sent and received by the feature estimation server. [Figure 5] FIG. 10 is a diagram illustrating generation of a reference image. [Figure 6] FIG. 10 is a diagram illustrating the operation of an automatic camera. [Figure 7A] 10 is a flowchart illustrating a shooting control process of an automatic camera. [Figure 7B]10 is a flowchart illustrating a process of a feature estimation server. [Figure 7C] 10 is a flowchart illustrating a process of a reference image generation server. [Figure 8] 10 is a flowchart illustrating an example of a shooting control process based on a reference image. [Figure 9] FIG. 10 is a diagram illustrating an example of the configuration of an automatic photography system according to a second embodiment. [Figure 10] FIG. 2 is a diagram illustrating an example of the hardware configuration of a reference image search server. [Figure 11] FIG. 10 is a diagram illustrating an example of a combination of an image and an image feature. [Figure 12] FIG. 10 is a diagram illustrating an example of the functional configuration of a reference image search server according to the second embodiment. [Figure 13] FIG. 10 is a diagram illustrating an example of a prompt received by a reference image search server. [Figure 14] 10 is a flowchart illustrating a reference image search process. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present invention will be described with reference to the drawings. The components described in each of the following embodiments are examples, and the scope of the present invention is not limited to these embodiments. All of the multiple features described in each embodiment are not necessarily essential, and may be combined in any desired manner.

[0011] <Embodiment 1> (System Configuration) FIG. 1 is a diagram illustrating an example of the configuration of an automatic photography system according to a first embodiment. In the example of FIG. 1, the automatic photography system includes an automatic photography camera 101, a feature estimation server 102, and a reference image generation server 103. The automatic photography camera 101, the feature estimation server 102, and the reference image generation server 103 can communicate with each other via a network 104 such as the Internet. Note that some of the processing of the feature estimation server 102 and the reference image generation server 103 may be executed by the automatic photography camera 101. Furthermore, the feature estimation server 102 and the reference image generation server 103 may be configured as an integral part of the automatic photography camera 101. In other words, the functions of each server may be realized by the automatic photography camera 101.

[0012] The automatic camera 101 is an imaging device that can accept verbal shooting instructions from a user and automatically capture an image in accordance with the shooting instructions. The feature estimation server 102 and the reference image generation server 103 are information processing devices such as a personal computer (PC). The feature estimation server 102 estimates the features of an image to be captured based on the shooting instructions from the user. The feature estimation server 102 transmits the estimated image features to the automatic camera 101. The reference image generation server 103 generates a reference image based on the image features estimated by the feature estimation server 102. The reference image is a confirmation image that allows the user to confirm what kind of image will be captured when the automatic camera 101 captures an image based on the shooting instructions. The reference image generation server 103 transmits the generated reference image to the automatic camera 101.

[0013] When the automatic camera 101 receives a shooting instruction from a user, it transmits the shooting instruction to the feature estimation server 102. When the automatic camera 101 receives the image features estimated by the feature estimation server 102, it transmits the received image features to the reference image generation server 103. When the automatic camera 101 receives the reference image from the reference image generation server 103, it displays the reference image on a display unit provided in the automatic camera 101. The automatic camera 101 may transmit the reference image to an external device provided with a display unit for display.

[0014] (Hardware configuration) FIG. 2 is a diagram illustrating an example of the hardware configuration of each device included in the automatic photography system of FIG. 1. The feature estimation server 102 and the reference image generation server 103 have the same configuration. Below, each configuration of the feature estimation server 102 will be explained, but the reference image generation server 103 It has a similar configuration.

[0015] The feature estimation server 102 includes a CPU 202, a ROM 203, a RAM 204, a HDD 205, a NIC (Network Interface Card) 206, an input unit 207, a display unit 208, and a GPU 209. The components are connected to each other via a system bus 201.

[0016] The CPU 202 loads the programs stored in the ROM 203 into the RAM 204 and executes them to realize various functions of the feature estimation server 102. The ROM 203 stores various programs for realizing processes executed by the feature estimation server 102. The RAM 204 temporarily stores data used by the various programs. The HDD 205 stores trained machine learning models and the like.

[0017] The NIC 206 is connected to the network 104. The NIC 206 transmits and receives data to and from external devices such as the automatic photography camera 101 via the network 104. The NIC 206 includes a communication circuit capable of communicating using communication methods that comply with various standards for transmitting and receiving information via the Internet, dedicated lines, and the like.

[0018] The input unit 207 includes input devices such as a keyboard and a mouse that accept input operations to the feature estimation server 102. The display unit 208 displays various types of information of the feature estimation server 102.

[0019] The GPU 209 can process more data in parallel than the CPU 202, and therefore can perform efficient calculations. The GPU 209 is used, for example, for processing using a machine learning model. The processing using the machine learning model may be executed using the GPU 209 in addition to the CPU 202. Furthermore, the processing using the machine learning model may be executed using either the CPU 202 or the GPU 209.

[0020] The automatic camera 101 includes a CPU 212, a ROM 213, a RAM 214, a HDD 215, a NIC 216, an input unit 217, a lens barrel 218, a motor drive mechanism 219, and a display unit 220. The components are connected to each other via a system bus 211.

[0021] The CPU 212 implements various functions of the automatic camera 101 by expanding programs stored in the ROM 213 into the RAM 214 and executing them. The ROM 213 stores various programs for implementing processes executed by the automatic camera 101. The RAM 214 temporarily stores data used by the various programs. The HDD 215 stores various data such as compressed image signals and compressed audio signals generated by the automatic camera 101.

[0022] The NIC 216 is connected to the network 104. The NIC 216 transmits and receives data to and from external devices such as the feature estimation server 102 and the reference image generation server 103 via the network 104. The NIC 216 includes a communication circuit capable of communicating using communication methods conforming to various standards for transmitting and receiving information via the Internet, dedicated lines, and the like.

[0023] The input unit 217 (receiving means) includes operation members such as buttons and a touch panel that receive input operations for the automatic camera 101. The input unit 217 also includes an image sensor that receives light incident through each lens group and acquires charge information corresponding to the amount of received light as analog image data. The input unit 217 also includes a microphone that converts audio signals from the surroundings of the automatic camera 101 into analog signals and acquires them.

[0024] The lens barrel 218 includes a zoom lens for optically varying the magnification and a lens for focusing. The lens barrel 218 can be driven to rotate relative to a fixed part of the automatic camera 101. The motor drive mechanism 219 is a mechanism for rotating the lens barrel 218 in the pitch direction and in the yaw direction. The motor drive mechanism 219 can rotate the lens barrel 218 in a pan-tilt manner.

[0025] The display unit 220 includes a display panel such as an OLED (Organic Light Emitting Diode), etc. The display unit 220 displays a screen for operating the automatic camera 101, various types of information about the automatic camera 101, etc.

[0026] (Functional configuration) 3A is a diagram illustrating an example of the functional configuration of the feature estimation server 102. Each functional unit of the feature estimation server 102 is realized by the CPU 202 loading a program stored in the ROM 203 into the RAM 204 and executing the program. The feature estimation server 102 includes a transmission / reception unit 301, a feature estimation unit 302, a storage unit 303, and a control unit 304.

[0027] The transmitting / receiving unit 301 communicates with the automatic camera 101 to transmit and receive various data. For example, the transmitting / receiving unit 301 receives a prompt transmitted from the automatic camera 101, and transmits to the automatic camera 101 the features of the image to be captured estimated by the feature estimation unit 302.

[0028] FIG. 4(A) is a diagram illustrating an example of a prompt related to a shooting instruction received by the feature estimation server 102 from the automatic camera 101. The prompt given by the automatic camera 101 to the feature estimation server 102 is an instruction or input given to a program and an AI model. The prompt given to the feature estimation server 102 includes an instruction to estimate image features and supplemental information to supplement the estimation instruction. The supplemental information is, for example, the functions of the automatic camera 101, the subject, the shooting instruction, and the output format of the image features. The prompt may be in a data format such as JSON (JavaScript (registered trademark) Object Notation).

[0029] The feature estimation unit 302 estimates the features of an image captured by the automatic camera 101 based on a prompt received from the automatic camera 101. The image features are information that the automatic camera 101 uses to determine what to capture and when to capture the image. The image features include the type of subject, the subject's facial expression, the subject's movement, the subject's posture, the capture composition, and the angle of view. FIG. 4(B) is a diagram illustrating an example of a prompt regarding image features that the feature estimation server 102 sends to the automatic camera 101. In the example of FIG. 4(B), "person" is specified as the type of subject, and "smile" is specified as the subject's facial expression. The image features may be in a data format such as JSON.

[0030] The feature estimation unit 302 may estimate the features of an image to be captured based on the input shooting instructions, using a learning model such as a generative AI. The learning model used to estimate the image features may be, for example, a model that has been pre-trained to learn general language patterns from large amounts of text data, and then fine-tuned by additionally learning data appropriate for the task of estimating image features. The feature estimation unit 302 may use an existing generative AI service to estimate the features of the captured image. The feature estimation unit 302 may use, for example, Chat GPT as the generative AI service.

[0031] The memory unit 303 stores the trained model. The control unit 304 executes various processes to control the functional units of the feature estimation server 102 and to control data transfer between the functional units.

[0032] FIG. 3B is a diagram illustrating an example of the functional configuration of the reference image generation server 103. Each functional unit of the reference image generation server 103 is realized by the CPU 202 expanding a program stored in the ROM 203 into the RAM 204 and executing the program. The reference image generation server 103 includes a transmission / reception unit 311, a reference image generation unit 312, a storage unit 313, and a control unit 314.

[0033] The transmitting / receiving unit 311 communicates with the automatic camera 101 to transmit and receive various data. For example, the transmitting / receiving unit 311 receives a prompt transmitted from the automatic camera 101, and transmits the reference image generated by the reference image generating unit 312 to the automatic camera 101. When the reference image is to be displayed on an external device, the transmitting / receiving unit 311 transmits the reference image generated by the reference image generating unit 312 to the external device.

[0034] FIG. 5(A) is a diagram illustrating a prompt that the reference image generation server 103 receives from the automatic camera 101. The prompt that the automatic camera 101 gives to the reference image generation server 103 includes an instruction to generate a reference image and supplemental information to supplement the instruction. The supplemental information is, for example, features to be included in the reference image to be generated and a sample image to be used to generate the reference image. In the example of FIG. 5(A), the features to be included in the reference image are "person" and "smile." The sample image is an image to be used as the background of the reference image.

[0035] The reference image generation unit 312 generates a reference image based on a prompt regarding the generation of the reference image. The reference image is generated as an example of an image that is expected to be captured when the automatic camera 101 captures a photo based on a shooting instruction. By checking the reference image, the user can confirm what kind of image will be automatically captured based on the shooting instruction. FIG. 5(B) is a diagram illustrating an example of a reference image. Based on the prompt shown in FIG. 5(A), the reference image generation unit 312 can generate a reference image such as that shown in FIG. 5(B), which shows a smiling person with a sample image as the background.

[0036] The sample image is used by the reference image generation server 103 to generate a reference image. The sample image is, for example, an image of the surroundings captured by the automatic camera 101. The sample image may also be an image previously captured to match the capture instruction (an image captured to show features estimated based on the words of the capture instruction). Images previously captured to match the capture instruction are recorded and accumulated in the HDD 215.

[0037] The reference image generation unit 312 may generate a reference image based on the features of an input image using a learning model such as a generative AI. The learning model used to generate the reference image may be, for example, a model that has previously learned image generation patterns from large amounts of text data, and then fine-tuned by additionally learning data corresponding to the task of generating a reference image.

[0038] The reference image generation unit 312 may generate a reference image by using an existing image generation AI service. Diffusion or the like may also be used.

[0039] The storage unit 313 stores the trained model. The control unit 314 executes various processes to control the functional units of the reference image generation server 103 and to control data transfer between the functional units.

[0040] 3C is a diagram illustrating an example of the functional configuration of the automatic photographing camera 101. Each functional unit of the automatic photographing camera 101 is realized by the CPU 212 loading a program stored in the ROM 213 into the RAM 214 and executing the program.

[0041] The automatic camera 101 includes a zoom control unit 321, a focus control unit 322, and a lens barrel control unit. 323, an image processing unit 324, an image analysis unit 325, an image recording unit 326, an audio processing unit 327, and a recording / playback unit 328. The automatic photographing camera 101 also includes a display control unit 329, an audio output unit 330, a photographing instruction acquisition unit 331, a feature acquisition unit 332, a reference image acquisition unit 333, a photographing start determination unit 334, a photographing control unit 335, a transmission / reception unit 336, and a control unit 337.

[0042] The zoom control unit 321 drives the zoom lens included in the lens barrel 218. The focus control unit 322 drives the lens included in the lens barrel 218. The lens barrel control unit 323 mechanically drives the lens barrel 218 in the tilt direction and pan direction. The shooting control unit 335 can change the shooting direction of the automatic camera 101 by driving the lens barrel 218 to rotate using the lens barrel control unit 323.

[0043] The image processing unit 324 A / D converts analog image data acquired from the image sensor serving as the input unit 217. The image processing unit 324 applies image processing such as distortion correction, white balance adjustment, and color interpolation processing to the A / D converted digital image data. The image processing unit 324 outputs the digital image data to which various image processes have been applied to the image analysis unit 325 and the image recording unit 326.

[0044] The image analysis unit 325 detects the subject by performing image analysis on the digital image data output by the image processing unit 324. The image analysis unit 325 also detects the facial expression and posture of the subject.

[0045] The image analysis unit 325 detects, for example, a person's face as a subject from an image obtained by subject detection image processing in the image processing unit 324 of a signal captured by the image sensor serving as the input unit 217. The image analysis unit 325 uses a predetermined pattern to determine a person's face, and can detect a portion in the image that matches the pattern as a person's face.

[0046] After detecting a person's face, the image analysis unit 325 can detect the person's facial expression. The image analysis unit 325 uses predetermined patterns to determine the person's facial expression, and can detect a facial expression pattern that has a higher degree of match with the detected person's face as the facial expression of that person.

[0047] Furthermore, after detecting the face of a person, the image analysis unit 325 can detect the posture of the person. The image analysis unit 325 can use a predetermined pattern to determine the posture of a person and detect a posture of a pattern that has a higher degree of agreement with the posture of the detected person as the posture of the person.

[0048] The image analysis unit 325 may detect the subject, the facial expression of the subject, the posture of the subject, and the like, using a learning model such as a neural network that has been trained by machine learning on image data related to the detection target.

[0049] The image recording unit 326 converts the digital image data output from the image processing unit 324 into a recording format such as JPEG format, and records it on the HDD 215 .

[0050] The audio processing unit 327 A / D converts an analog audio signal acquired from a microphone serving as the input unit 217. The audio processing unit 327 applies audio processing such as optimization processing to the A / D converted digital audio signal.

[0051] The image processing unit 324 reads out the image signal temporarily stored in the RAM 214 and encodes it to generate a compressed image signal. The audio processing unit 327 reads out the audio signal temporarily stored in the RAM 214 and encodes it to generate a compressed audio signal. do.

[0052] The recording and playback unit 328 records the compressed image signals generated by the image processing unit 324, the compressed audio signals generated by the audio processing unit 327, and other control data related to shooting. The recording and playback unit 328 may record the image signals generated by the image processing unit 324 and the audio signals generated by the audio processing unit 327 on the HDD 215 without compressing and encoding them.

[0053] Furthermore, the recording and reproducing unit 328 can read out compressed image signals, compressed audio signals, image signals, and audio signals recorded in the HDD 215. The recording and reproducing unit 328 reads out compressed image signals from the HDD 215 and transmits them to the image processing unit 324, and reads out compressed audio signals and transmits them to the audio processing unit 327. The image processing unit 324 decodes the compressed image signals received from the recording and reproducing unit 328 in a predetermined procedure and transmits the decoded signals to the display control unit 329. The audio processing unit 327 decodes the compressed audio signals received from the recording and reproducing unit 328 in a predetermined procedure and transmits the decoded signals to the audio output unit 330.

[0054] The display control unit 329 displays various types of information about the automatic camera 101. The display control unit 329 outputs, for example, the image signal transmitted by the image processing unit 324 or the image signal of the reference image acquired by the reference image acquisition unit 333 to the display unit 220. The audio output unit 330 outputs the audio signal received from the audio processing unit 327 or the audio signal of the reference image acquired by the reference image acquisition unit 333.

[0055] The shooting instruction acquisition unit 331 acquires, as shooting instructions, operation using the buttons on the input unit 217 or audio signals processed by the audio processing unit 327. The shooting instructions are words that express what kind of image the user wants the automatic camera 101 to capture. The words of the shooting instructions may be sentences. The shooting instructions include instructions regarding the subject to be photographed, the facial expression of the subject to be photographed, the posture of the subject to be photographed, the angle of view of the photograph, the composition, etc. For example, the shooting instructions are words such as "Take a picture if someone is smiling" or "Take a picture so that the sleeping child is in the center of the angle of view."

[0056] The feature acquisition unit 332 acquires the features of the image captured by the automatic camera 101 based on the shooting instructions acquired by the shooting instruction acquisition unit 331. The feature acquisition unit 332 creates a prompt that instructs the automatic camera 101 to estimate the features of the image to be captured based on the input shooting instructions and automatic camera information. The automatic camera information includes information on shooting control functions available to the automatic camera 101. Examples of the shooting control functions include a function for changing the shooting direction and a function for determining the facial expression of the subject. The automatic camera information also includes information such as the subject name of the subject registered as the subject to be photographed.

[0057] The feature acquisition unit 332 transmits the created prompt to the feature estimation server 102 via the transmission / reception unit 336. The feature acquisition unit 332 receives, via the transmission / reception unit 336, the features of the image to be captured that the feature estimation server 102 has estimated based on the prompt.

[0058] The reference image acquisition unit 333 acquires a reference image based on the features of the image acquired by the feature acquisition unit 332. That is, the reference image acquisition unit 333 acquires a reference image based on the words of the shooting instruction. The reference image acquisition unit 333 creates a prompt that instructs generation of a reference image based on the features of the image acquired by the feature acquisition unit 332 and a sample image. The reference image acquisition unit 333 transmits the created prompt to the reference image generation server 103 via the transmission / reception unit 336. The reference image acquisition unit 333 receives the reference image generated by the reference image generation server 103 based on the prompt via the transmission / reception unit 336.

[0059] The image data of the surroundings of the automatic camera 101 is acquired by the lens barrel control unit 323. Images captured by rotating in the pan direction or tilt direction are acquired by processing them with the image processing unit 324. Also, image data of a subject recorded in the HDD 215 is acquired by the reference image acquisition unit 333 searching, via the recording / playback unit 328, for image data tagged with the subject name included in the image features.

[0060] The shooting start determination unit 334 determines whether the reference image acquired by the reference image acquisition unit 333 matches the shooting instruction (whether it shows features estimated based on the words of the shooting instruction). The shooting start determination unit 334 causes the display unit 220 to display the reference image. The shooting start determination unit 334 may determine whether the reference image satisfies the conditions of the shooting instruction by asking the user whether the shooting instruction is satisfied. The shooting control unit 335 controls so that shooting is performed based on the reference image acquired by the reference image acquisition unit 333.

[0061] The transmitting / receiving unit 336 controls communication with the feature estimation server 102 and communication with the reference image generation server 103. For example, the transmitting / receiving unit 336 transmits a prompt created by the feature acquisition unit 332 to the feature estimation server 102, and transmits a prompt created by the reference image acquisition unit 333 to the reference image generation server 103. The transmitting / receiving unit 336 receives the image features transmitted from the feature estimation server 102 and the reference image transmitted from the reference image generation server 103.

[0062] The control unit 337 executes various processes to control each block of the automatic photographing camera 101 and to control data transfer between each block.

[0063] (Automatic camera operation) 6 is a diagram illustrating the operation of the automatic camera 101. A user 601 inputs verbal instructions to the automatic camera 101 to take a photograph (A1). The automatic camera 101 creates a prompt used to estimate the features of an image based on the input verbal instructions, and sends the created prompt to the feature estimation server 102 to instruct estimation of the features of the image to be captured (A2). The feature estimation server 102 estimates the features of the image to be captured based on the prompt received from the automatic camera 101 (A3). The feature estimation server 102 sends the estimated image features to the automatic camera 101 (A4).

[0064] The automatic camera 101 creates a prompt to be used for generating a reference image based on the features of the image received from the feature estimation server 102, and instructs the reference image generation server 103 to generate the reference image by transmitting the created prompt to the reference image generation server 103 (A5). The reference image generation server 103 generates a reference image based on the prompt received from the automatic camera 101 (A6). The reference image generation server 103 transmits the generated reference image to the automatic camera 101 (A7).

[0065] The automatic camera 101 displays the reference image received from the reference image generation server 103 on the display unit 220 to present to the user 601, and confirms with the user whether the reference image matches the shooting instructions (whether the reference image shows features estimated based on the words of the shooting instructions) (A8). The user 601 confirms whether the reference image displayed on the display unit 220 matches the shooting instructions input to the automatic camera 101 in A1, and inputs the confirmation result to the automatic camera 101 (A9).

[0066] The automatic camera 101 starts controlling shooting based on the reference image when it is confirmed by input from the user 601 that the reference image shows characteristics estimated based on the words of the shooting instruction. Before starting shooting control, the automatic camera 101 can align its understanding with the user 601 regarding what kind of image will be shot in response to the shooting instruction from the user 601.

[0067] (Shooting control processing) 7A to 7C, the photographing control process of the automatic photographing camera 101 will be described. Figures 7A to 7C show the processes of the automatic photographing camera 101, the feature estimation server 102, and the reference image generation server 103, respectively.

[0068] The processing of FIG. 7A is realized by the components shown in FIG. 3C when the CPU 212 of the automatic camera 101 loads a program stored in the ROM 213 into the RAM 214 and executes it. The shooting control processing shown in FIG. 7A is started, for example, by turning on the main power of the automatic camera 101. In the shooting control processing, the user can check the reference image to see what kind of photo will be taken in response to the shooting instruction. Furthermore, if the reference image shows features estimated based on the words of the shooting instruction, the automatic camera 101 controls the shooting so that the photo is taken based on the reference image.

[0069] In step S701, the photographing instruction acquisition unit 331 accepts a verbal photographing instruction via the input unit 217. If the input unit 217 is a microphone, the photographing instruction acquisition unit 331 acquires an audio signal from the microphone processed by the audio processing unit 327 as a photographing instruction. For example, the photographing instruction acquisition unit 331 recognizes the user's voice and accepts a photographing instruction in the form of words uttered by the user. If the input unit 217 is an operation unit (operation member) such as a button, the photographing instruction acquisition unit 331 may accept words input via the operation unit as a photographing instruction. The photographing instruction acquisition unit 331 may also accept words received from an external device as a photographing instruction.

[0070] In step S702, the feature acquisition unit 332 acquires the features of the image to be captured based on the shooting instruction acquired in step S701. The feature acquisition unit 332 instructs the feature estimation server 102 to estimate the features of the image based on the shooting instruction, and acquires the features of the image estimated by the feature estimation server 102 based on the words of the shooting instruction.

[0071] In step S703, the feature acquisition unit 332 determines whether or not an image matching the shooting instructions (an image showing features estimated based on the words of the shooting instructions) can be captured using the functions installed in the automatic camera 101. If the automatic camera 101 does not have the function for capturing an image matching the shooting instructions, the process proceeds to step S704. If the automatic camera 101 has the function for capturing an image matching the shooting instructions, the process proceeds to step S705.

[0072] In step S704, the display control unit 329 notifies the user that the automatic camera 101 does not have the function to capture an image that matches the capture instruction, and therefore will not capture an image based on the capture instruction. The display control unit 329 displays a message on the display unit 220 notifying the user that the automatic camera 101 will not capture an image based on the capture instruction.

[0073] In step S705, the reference image acquisition unit 333 acquires a sample image to be used to generate the reference image. The sample image is, for example, an image of the surroundings of the automatic camera 101, and may be an image captured by rotating the lens barrel 218.

[0074] In step S706, the reference image acquisition unit 333 acquires a reference image based on the features of the image acquired in step S702. The reference image acquisition unit 333 instructs the reference image generation server 103 to generate a reference image that indicates the features of the image, and can acquire the reference image generated by the reference image generation server 103. The reference image generation server 103 can receive the sample image acquired in step S705 from the automatic camera 101, and generate a reference image using the received sample image.

[0075] In step S707, the display control unit 329 displays the reference image acquired in step S706 as , and presents it to the user by displaying it on the display unit 220. The display control unit 329 may present the reference image, and may also output information about the features of the image acquired in step S702 to the display unit 220 to notify the user. Furthermore, the display control unit 329 may present the reference image to the user by transmitting the reference image to an external device via the transmission / reception unit 336 and displaying it on the display unit of the external device.

[0076] In step S708, the shooting start determination unit 334 determines whether the reference image matches the shooting instructions (whether it shows features estimated based on the words of the shooting instructions). The shooting start determination unit 334 may confirm with the user whether the reference image matches the shooting instructions. The user inputs whether the reference image displayed on the display unit 220 matches the shooting instructions via the input unit 217. The shooting start determination unit 334 may also communicate with an external device via the transmission / reception unit 336 to determine whether the reference image matches the shooting instructions. If the reference image matches the shooting instructions, the process proceeds to step S709. If the reference image does not match the shooting instructions, the process returns to step S701.

[0077] In step S709, the imaging control unit 335 controls imaging based on the reference image. Details of imaging control based on the reference image will be described later with reference to FIG.

[0078] In step S710, the control unit 337 determines whether or not to end the image capture. The control unit 337 determines to end the image capture when the main power of the automatic photographing camera 101 is turned off or when an instruction to end the image capture is input via the input unit 217. If it is determined to end the image capture, the processing shown in Fig. 7A ends. If it is determined not to end the image capture, the processing returns to step S701.

[0079] Fig. 7B is a flowchart illustrating the processing of the feature estimation server 102. The processing of Fig. 7B is realized by the respective units shown in Fig. 3A when the CPU 202 of the feature estimation server 102 loads a program stored in the ROM 203 into the RAM 204 and executes it.

[0080] In step S711, the transmitting / receiving unit 301 determines whether or not an instruction to estimate the characteristics of an image captured by the automatic camera 101 has been received. The transmitting / receiving unit 301 receives information about a capture instruction along with an instruction to estimate the characteristics of an image. The transmitting / receiving unit 301 receives, for example, a prompt including information about the capture instruction. The transmitting / receiving unit 301 may receive the capture instruction as an instruction to estimate the characteristics of an image. If an instruction to estimate the characteristics of an image has been received, the process proceeds to step S712. If an instruction to estimate the characteristics of an image has not been received, the process returns to step S711.

[0081] In step S712, the feature estimation unit 302 estimates the features of the image to be captured based on the capture instruction. In step S713, the transmission / reception unit 301 transmits the image features estimated in step S712 to the automatic camera 101.

[0082] Fig. 7C is a flowchart illustrating the processing of the reference image generation server 103. The processing of Fig. 7C is realized by the respective units shown in Fig. 3B when the CPU 202 of the reference image generation server 103 loads a program stored in the ROM 203 into the RAM 204 and executes it.

[0083] In step S721, the transmitting / receiving unit 311 determines whether or not an instruction to generate a reference image has been received from the automatic camera 101. The transmitting / receiving unit 311 receives the image features estimated by the feature estimation server 102 along with the instruction to generate a reference image. The transmitting / receiving unit 311 receives, for example, a prompt including the image features. The transmitting / receiving unit 311 may receive the image features as an instruction to generate a reference image. If an instruction to generate a reference image has been received, the process proceeds to step S722. If an instruction to generate a reference image has not been received, the process returns to step S721.

[0084] In step S722, the reference image generating unit 312 generates a reference image based on the image features. In step S723, the transmitting / receiving unit 311 transmits the reference image generated in step S722 to the automatic camera 101.

[0085] 7A to 7C, the automatic camera 101 is not limited to automatic shooting set in advance, but can also control shooting by receiving verbal shooting instructions from the user. The automatic camera 101 also acquires a reference image to show the user so that the user can check what kind of image will be captured based on the shooting instructions. Therefore, before the automatic camera 101 starts shooting, the user can check in advance whether the image the user intends to capture matches the image captured by the automatic camera 101 in accordance with the shooting instructions (whether the image has the characteristics according to the shooting instructions).

[0086] (Shooting control processing based on reference image) The photographing control process based on the reference image will be described with reference to Fig. 8. The process in Fig. 8 is realized by the components shown in Fig. 3C when the CPU 212 of the automatic photographing camera 101 loads a program stored in the ROM 213 into the RAM 214 and executes the program.

[0087] In step S801, the shooting control unit 335 searches for a subject. For example, the shooting control unit 335 changes the shooting direction of the automatic camera 101 by rotating the lens barrel 218 in the pan direction or tilt direction using the lens barrel control unit 323. The shooting control unit 335 searches for a subject that satisfies the shooting instructions by changing the shooting direction.

[0088] In step S802, the image analysis unit 325 calculates the similarity between the reference image and a live view image captured by the automatic camera 101 and displayed in real time on the display unit 220. The similarity between the reference image and the live view image is, for example, the similarity in the type of subject, the facial expression of the subject, the posture of the subject, and the position of the subject. The type of subject is a person, an animal, an object, etc.

[0089] The similarity in terms of subject type is set higher when the subject type matches between the reference image and the live-view image than when they do not match. The similarity in terms of subject expression is higher the more similar the subject's expression is between the reference image and the live-view image. The similarity in terms of subject posture is higher the more similar the subject's posture is between the reference image and the live-view image. The similarity in terms of subject position is higher the shorter the distance between the subject's position in the reference image and the subject's position in the live-view image.

[0090] In step S803, the photography control unit 335 determines whether the similarity calculated in step S802 is greater than a predetermined threshold. For example, the photography control unit 335 may determine whether the similarity regarding the type of subject, the facial expression of the subject, the posture of the subject, and the position of the subject is greater than a predetermined threshold. The photography control unit 335 may also determine whether the average value of each similarity is greater than a predetermined threshold. The predetermined threshold may be set or changed by the user. If the similarity is greater than the predetermined threshold, the process proceeds to step S804. If the similarity is equal to or less than the predetermined threshold, the process proceeds to step S805.

[0091] In step S804, since the similarity between the reference image and the live view image is greater than a predetermined threshold, the shooting control unit 335 performs control to shoot the subject. The image recording unit 326 records the shot image processed by the image processing unit 324 in the HDD 215.

[0092] In step S805, the photographing control unit 335 determines whether or not to end photographing. For example, if the photographing control unit 335 has failed (lost) in tracking the subject, the photographing control unit 335 determines not to end photographing. Furthermore, the photographing control unit 335 may determine not to end photographing if photographing has been performed a predetermined number of times based on the reference image. If it is determined not to end photographing, the photographing control unit 335 returns to step S801 and searches for another subject.

[0093] On the other hand, if an image that substantially matches the reference image is captured, the imaging control unit 335 determines to end imaging. The imaging control unit 335 may also determine to end imaging if an image that substantially matches the reference image is not captured for a predetermined time. If it is determined to end imaging, the process shown in FIG. 8 ends.

[0094] According to the first embodiment, the automatic camera 101 acquires a reference image based on the shooting instruction words and takes a photograph based on the reference image. By changing the shooting direction, the automatic camera 101 can search for and photograph a subject similar to the subject in the reference image. Furthermore, by presenting the reference image to the user, the automatic camera 101 can allow the user to confirm whether the reference image exhibits the characteristics estimated based on the shooting instruction words.

[0095] <Embodiment 2> In the first embodiment, the reference image generation server 103 generates a reference image based on the image features estimated by the feature estimation server 102. In contrast, in the second embodiment, the automatic camera 101 acquires a reference image from images previously captured to match the shooting instructions (images captured to show features estimated based on the words of the shooting instructions). Specifically, the automatic camera 101 accumulates images previously captured in a database based on the image features estimated by the feature estimation server 102, and searches the database to acquire a reference image corresponding to the image features estimated from the shooting instructions. In the second embodiment, the automatic shooting system includes a reference image search server 901 instead of the reference image generation server 103 of the first embodiment. Descriptions common to the first embodiment will be omitted.

[0096] (System Configuration) FIG. 9 is a diagram illustrating an example of the configuration of an automatic photography system according to the second embodiment. In the example of FIG. 9, the automatic photography system includes an automatic photography camera 101, a feature estimation server 102, and a reference image search server 901. The automatic photography camera 101, the feature estimation server 102, and the reference image search server 901 can communicate with each other via a network 104 such as the Internet. Note that part of the processing of the feature estimation server 102 and the reference image search server 901 may be executed by the automatic photography camera 101. Furthermore, the feature estimation server 102 and the reference image search server 901 may be configured as an integral part of the automatic photography camera 101. In other words, the functions of each server may be realized by the automatic photography camera 101.

[0097] The reference image search server 901 receives the image features estimated by the feature estimation server 102 from the automatic camera 101, and acquires a reference image that shows the features of the received image. The reference image search server 901 is an information processing device such as a personal computer (PC).

[0098] When the automatic camera 101 receives a shooting instruction from a user, it transmits the shooting instruction to the feature estimation server 102. When the automatic camera 101 receives the image features estimated by the feature estimation server 102, it transmits the image features to the reference image search server 901. When the automatic camera 101 receives a reference image from the reference image search server 901, it displays the reference image on the display unit 220 provided in the automatic camera 101. The automatic camera 101 may transmit the reference image to an external device provided with a display unit for display.

[0099] (Hardware configuration) FIG. 10 is a diagram illustrating an example of the hardware configuration of the reference image search server 901 included in the automatic photography system of FIG.

[0100] The reference image search server 901 includes a CPU 1002, a ROM 1003, a RAM 1004, a HDD 1005, a NIC 1006, an input unit 1007, and a display unit 1008. The components are connected to one another via a system bus 1001.

[0101] The CPU 1002 loads the programs stored in the ROM 1003 into the RAM 1004 and executes them to realize various functions of the reference image search server 901. The ROM 1003 stores various programs for realizing the processes executed by the reference image search server 901. The RAM 1004 temporarily stores data used by the various programs.

[0102] HDD 1005 stores combinations of previously captured images and corresponding image features. Fig. 11 is a diagram illustrating combinations of images and image features. In Fig. 11, previously captured image 1101 is an image of a smiling person. Image features 1102 corresponding to image 1101 include the features "person" and "smile."

[0103] The HDD 1005 stores a plurality of combinations of images and image features. By storing data of combinations of various image features and corresponding images, the HDD 1005 can provide reference images corresponding to various shooting instructions.

[0104] The NIC 1006 is connected to the network 104. The NIC 1006 transmits and receives data to and from external devices such as the automatic camera 101 via the network 104. The NIC 1006 includes a communication circuit capable of communicating using communication methods that comply with various standards for transmitting and receiving information via the Internet, dedicated lines, and the like.

[0105] The input unit 1007 includes input devices such as a keyboard and a mouse that accept input operations to the reference image search server 901. The display unit 1008 displays various types of information of the reference image search server 901.

[0106] (Functional configuration) 12 is a diagram illustrating an example of the functional configuration of the reference image search server 901. Each functional unit of the reference image search server 901 is realized by the CPU 1002 expanding a program stored in the ROM 1003 into the RAM 1004 and executing the program. The reference image search server 901 includes a transmission / reception unit 1201, a reference image search unit 1202, a storage unit 1203, and a control unit 1204.

[0107] The transmitting / receiving unit 1201 communicates with the automatic camera 101 to transmit and receive various data. For example, the transmitting / receiving unit 1201 receives a prompt transmitted from the automatic camera 101, and transmits to the automatic camera 101 the reference image search unit 1202 has searched for and acquired.

[0108] 13 is a diagram illustrating an example of a prompt that the reference image search server 901 receives from the automatic camera 101. The prompt that the automatic camera 101 gives to the reference image search server 901 includes a search instruction for a reference image and supplemental information to supplement the search instruction. The supplemental information is a feature to be included in the reference image to be searched. In the example of FIG. 13, the features to be included in the reference image are "person" and "smile."

[0109] The reference image search unit 1202 searches for a reference image based on a prompt regarding the search for a reference image. The reference image search unit 1202 searches for a reference image based on the characteristics of the images recorded in the HDD 1005 and The reference image search unit 1202 calculates the degree of association with the features contained in the reference image specified in the prompt. Of the images recorded on the HDD 1005, the reference image search unit 1202 retrieves the image with the highest degree of association between the features of the image and the features specified in the prompt as the reference image.

[0110] The relevance score is a score used to select an image from the images stored on the HDD 1005 that exhibits the features contained in the reference image specified in the prompt. The relevance score increases as an image has more features in common with the features specified in the prompt. The relevance score decreases as an image has fewer features in common with the features specified in the prompt.

[0111] The storage unit 1203 stores combinations of images and image features. The control unit 1204 executes various processes to control the functional units of the reference image search server 901 and to control data transfer between the functional units.

[0112] (Reference image search process) Fig. 14 is a flowchart illustrating a reference image search process performed by the reference image search server 901. The reference image search server 901 searches for a reference image to be presented to the user from images recorded on the HDD 1005 based on the characteristics of the image received from the automatic camera 101, and transmits the searched reference image to the automatic camera 101. The process of Fig. 14 is realized by the respective units shown in Fig. 12 when the CPU 1002 of the reference image search server 901 loads a program stored in the ROM 1003 into the RAM 1004 and executes the program.

[0113] In step S1401, the transmitting / receiving unit 1201 determines whether or not a reference image search instruction has been received from the automatic camera 101. The transmitting / receiving unit 1201 receives the image features estimated by the feature estimation server 102 along with the reference image search instruction. The transmitting / receiving unit 1201 may receive the image features as the reference image search instruction. If a reference image search instruction has been received, the process proceeds to step S1402. If a reference image search instruction has not been received, the process returns to step S1401.

[0114] In step S1402, the reference image search unit 1202 searches for a reference image based on the image features. In step S1403, the transmitting / receiving unit 1201 transmits the reference image searched for in step S1402 to the automatic camera 101.

[0115] In the above-described second embodiment, the reference image search server 901 searches for images recorded in the HDD 1005 based on the features of the images received from the automatic camera 101, and acquires reference images to be presented to the user. The reference image search server 901 transmits the acquired reference images to the automatic camera 101.

[0116] The present invention also includes cases where a software program that realizes the functions of the above-described embodiments is supplied to a system or device having a computer capable of executing the program, either directly from a recording medium or via wired or wireless communication, and the supplied program is executed. The program code itself supplied or installed on a computer to realize the functions of the above-described embodiments also realizes the present invention. In other words, the present invention includes a computer program for realizing the functions of the embodiments. The computer program may be in any form, such as object code, a program executed by an interpreter, or script data supplied to an OS, as long as it has the program's functions.

[0117] The recording medium for supplying the program may be, for example, a magnetic recording medium such as a hard disk or a magnetic tape, an optical / magneto-optical storage medium, or a non-volatile semiconductor memory. Also, as a method for supplying the program, a method for implementing the present invention on a server on a computer network may be used. The server may store a computer program for the purpose, and a client computer connected to the server may download the computer program.

[0118] The various controls described above may or may not be performed by a single piece of hardware (e.g., a processor or circuit). The entire device may be controlled by multiple pieces of hardware (e.g., multiple processors, multiple circuits, or a combination of one or more processors and one or more circuits) sharing the processing.

[0119] The above processor is a processor in a broad sense, and includes general-purpose processors and dedicated processors. General-purpose processors include, for example, CPUs (Central Processing Units), MPUs (Micro Processing Units), and DSPs (Digital Signal Processors). Dedicated processors include, for example, GPUs (Graphics Processing Units), ASICs (Application Specific Integrated Circuits), and PLDs (Programmable Logic Devices). Programmable logic devices include, for example, FPGAs (Field Programmable Gate Arrays) and CPLDs (Complex Programmable Logic Devices).

[0120] Although the embodiments of the present invention have been described in detail, the present invention is not limited to these specific embodiments, and various forms within the scope of the gist of the present invention are also included in the present invention. Furthermore, each of the above-described embodiments merely represents one embodiment of the present invention, and each embodiment can be combined as appropriate.

[0121] <Other embodiments> The present invention can also be realized by supplying a program that realizes one or more of the functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program, or by a circuit that realizes one or more of the functions.

[0122] The disclosure of this embodiment includes the following configuration, method, program, and medium. (Configuration 1) a reception means for receiving verbal instructions for taking photographs; an acquisition means for acquiring a reference image based on the instruction to take a photograph; notification means for presenting the reference image to a user; a control means for controlling the photographing based on the reference image; An imaging device comprising: (Configuration 2) The acquisition means estimates characteristics of the image to be captured based on the instruction to capture, and acquires the reference image showing the estimated characteristics. 2. The imaging device according to claim 1, (Configuration 3) The acquisition means generates the reference image based on the features. 3. The imaging device according to configuration 2. (Configuration 4) The notification means notifies the user that the image capturing device will not capture an image based on the image capturing instruction if the image capturing device does not have a function for capturing an image showing the characteristics estimated based on the image capturing instruction. 4. The imaging device according to any one of configurations 1 to 3. (Configuration 5) The acquisition means generates the reference image using an image captured by the imaging device. 5. The imaging device according to any one of configurations 1 to 4. (Configuration 6) The acquisition means generates the reference image using an image previously captured so as to exhibit the characteristics estimated based on the words of the image capture instruction. 6. The imaging device according to any one of configurations 1 to 5, wherein: (Configuration 7) The notification means presents the reference image and notifies the user of information about the feature estimated based on the words of the photographing instruction. 7. The imaging device according to any one of configurations 1 to 6, wherein: (Configuration 8) having an imaging means capable of changing the imaging direction, The control means controls the imaging means to search for a subject that satisfies the imaging instruction by changing the imaging direction of the imaging means, and to photograph the detected subject. 8. The imaging device according to any one of configurations 1 to 7, wherein: (Configuration 9) The notification means presents the reference image to the user by displaying the reference image on a display unit. 9. The imaging device according to any one of configurations 1 to 8. (Configuration 10) The notification means presents the reference image to the user by transmitting the reference image to an external device and displaying it. 10. The imaging device according to any one of configurations 1 to 9, wherein: (Configuration 11) The receiving means recognizes the user's voice and receives the photographing instruction in words uttered by the user. 11. The imaging device according to any one of configurations 1 to 10. (Configuration 12) The receiving means receives words received from an external device as the photographing instruction. 11. The imaging device according to any one of configurations 1 to 10. (Configuration 13) The receiving means receives words input via an operation unit as the photographing instruction. 11. The imaging device according to any one of configurations 1 to 10. (Configuration 14) The control means controls so as to perform shooting when the similarity between the reference image and the live view image is greater than a predetermined threshold. 14. The imaging device according to any one of configurations 1 to 13. (Configuration 15) The acquisition means acquires the reference image from among images previously captured so as to exhibit the characteristics estimated based on the words of the image capture instruction. 14. The imaging device according to any one of configurations 1 to 13. (Configuration 16) the notification means confirms with the user whether or not the reference image shows the feature estimated based on the words of the photographing instruction; The control means controls the image capturing to be performed based on the reference image when the user confirms that the reference image shows the characteristic. 16. The imaging device according to any one of configurations 1 to 15. (method) receiving verbal photography instructions; acquiring a reference image based on the instruction to take a photograph; presenting the reference image to a user; a step of controlling the photographing based on the reference image; 10. A method for controlling an imaging device, comprising: (program) A program for causing a computer to function as each means of the imaging device according to any one of configurations 1 to 16. (medium) 17. A computer-readable storage medium storing a program for causing a computer to function as each means of the imaging device according to any one of configurations 1 to 16. [Explanation of symbols]

[0123] 101: Imaging device (automatic camera), 212: CPU, 217: Input unit, 220: Display unit, 335: Imaging control unit

Claims

1. a reception means for receiving verbal instructions for taking photographs; an acquisition means for acquiring a reference image based on the instruction to take a photograph; notification means for presenting the reference image to a user; a control means for controlling the photographing based on the reference image; An imaging device comprising:

2. The acquisition means estimates characteristics of the image to be captured based on the instruction to capture, and acquires the reference image showing the estimated characteristics.

2. The imaging device according to claim 1.

3. The acquisition means generates the reference image based on the features.

3. The imaging device according to claim 2.

4. The notification means notifies the user that the image capturing device will not capture an image based on the image capturing instruction if the image capturing device does not have a function for capturing an image showing the characteristics estimated based on the image capturing instruction.

2. The imaging device according to claim 1.

5. The acquisition means generates the reference image using an image captured by the imaging device.

2. The imaging device according to claim 1.

6. The acquisition means generates the reference image using an image previously captured so as to exhibit the characteristics estimated based on the words of the image capture instruction.

2. The imaging device according to claim 1.

7. The notification means presents the reference image and notifies the user of information about the feature estimated based on the words of the photographing instruction.

2. The imaging device according to claim 1.

8. having an imaging means capable of changing the imaging direction, The control means controls the imaging means to search for a subject that satisfies the imaging instruction by changing the imaging direction of the imaging means, and to photograph the detected subject.

2. The imaging device according to claim 1.

9. The notification means presents the reference image to the user by displaying the reference image on a display unit.

2. The imaging device according to claim 1.

10. The notification means presents the reference image to the user by transmitting the reference image to an external device and displaying it.

2. The imaging device according to claim 1.

11. The receiving means recognizes the user's voice and receives the photographing instruction in words uttered by the user.

2. The imaging device according to claim 1.

12. The receiving means receives words received from an external device as the photographing instruction.

2. The imaging device according to claim 1.

13. The receiving means receives words input via an operation unit as the photographing instruction.

2. The imaging device according to claim 1.

14. The control means controls so as to perform shooting when the similarity between the reference image and the live view image is greater than a predetermined threshold.

2. The imaging device according to claim 1.

15. The acquisition means acquires the reference image from among images previously captured so as to exhibit the characteristics estimated based on the words of the image capture instruction.

2. The imaging device according to claim 1.

16. the notification means confirms with the user whether or not the reference image shows the feature estimated based on the words of the photographing instruction; The control means controls the image capturing to be performed based on the reference image when the user confirms that the reference image shows the characteristic.

2. The imaging device according to claim 1.

17. receiving verbal photography instructions; acquiring a reference image based on the instruction to take a photograph; presenting the reference image to a user; a step of controlling the photographing based on the reference image; 10. A method for controlling an imaging device, comprising:

18. A program for causing a computer to function as each of the means of the imaging device according to any one of claims 1 to 16.

19. A computer-readable storage medium storing a program for causing a computer to function as each of the means of the imaging device according to any one of claims 1 to 16.

Images