Imaging device, control method thereof, and program

Abstract

To provide an imaging apparatus capable of automatically capturing an image at a suitable timing predicted from the current image.SOLUTION: An imaging apparatus includes prediction means for generating a prediction image, in which an image at a second point in time future than a first point in time is predicted using an image at the first point in time output from imaging means, determination means for determining, with respect to the prediction image, a degree of suitability indicating the degree to which the prediction image is suitable as an image to be recorded, and control means for controlling the imaging means so as to capture an image when the time reaches the second point in time when the determined suitability is higher than a predetermined threshold.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to an imaging apparatus, a control method thereof, and a program. [Background technology]

[0002] In recent years, image prediction technology using machine learning has been actively developed. By inputting a video captured by a camera to an inference model trained on a large amount of video data of the subject's behavior scenes as training data, it is possible to predict the future image of the subject. Patent Document 1 discloses a technology in which an image captured by an in-vehicle camera is input to a trained convolutional neural network, and a dangerous area where a moving object may appear and cause a collision and its characteristics are estimated when a vehicle equipped with the camera continues to drive. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-162438 A Summary of the Invention [Problem to be solved by the invention]

[0004] By the way, it is expected that a shooting assist function with improved usability will be provided by applying a technology for predicting future images of a subject to the shooting assist function of a camera, a smartphone, etc. For example, if the shooting assist function of a camera or a smartphone automatically performs shooting at an appropriate shooting timing according to changes in the subject, it is considered that usability for a user will be improved. Although Patent Document 1 uses a technology for predicting future images, it does not consider using the technology in the shooting assist function.

[0005] The present invention has been made in view of the above-mentioned problems, and has as its object to realize a technique that makes it possible to automatically capture an image at an appropriate timing that is predicted from a current image. [Means for solving the problem]

[0006] In order to solve this problem, for example, an imaging device according to the present invention has the following configuration. A prediction means for generating a predicted image by predicting an image at a second time point, which is future than the first time point, using an image at a first time point output from an imaging means; a determination means for determining a suitability of the predicted image, the suitability indicating a degree to which the predicted image is suitable as an image to be recorded; and a control means for controlling the imaging means to capture an image when the time reaches the second time point when the determined suitability is higher than a predetermined threshold. death, The control means if the determined suitability is higher than the predetermined threshold, setting a shooting reservation for shooting an image at the second time point; after setting a first shooting reservation for capturing an image at the second time point based on the image at the first time point, determining whether the suitability of the predicted image generated using an image at a third time point that is later than the first time point is higher than the suitability of the predicted image generated using the image at the first time point; It is characterized by: Effect of the Invention

[0007] According to the present invention, it is possible to automatically capture an image at a suitable timing predicted from a current image. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of a camera as an example of an imaging device according to the present invention. [Diagram 2] FIG. 1 is a diagram showing an example of the configuration of an image prediction unit in the present embodiment. [Diagram 3] FIG. 2 is a diagram showing an example of the configuration of a preferred determination unit in the present embodiment; [Figure 4] FIG. 1 is a diagram showing the data flow from LV image generation to automatic shooting in this embodiment. [Diagram 5] FIG. 1 is a diagram showing the relationship between a predicted image generated in this embodiment and time. [Figure 6]FIG. 13 is a diagram showing an example of an imaging reservation management table according to the embodiment; [Figure 7] A flowchart showing a series of operations in automatic photography processing according to the present embodiment. [Figure 8] FIG. 13 is a diagram showing an example of a display screen for notification information according to the embodiment; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0010] In the following, as an example of an imaging device, a camera capable of automatically capturing images at suitable timing using a machine learning model will be described. However, this embodiment is not limited to a camera, and can be applied to any other device capable of automatically capturing images at suitable timing using a machine learning model.

[0011] <Camera configuration> An example of the functional configuration of a camera 100 as an example of an imaging device according to this embodiment will be described with reference to Fig. 1. One or more of the functional blocks shown in Fig. 1 may be realized by hardware such as an ASIC or a programmable logic array (PLA), or may be realized by a programmable processor such as a CPU or a GPU executing software. They may also be realized by a combination of software and hardware. Therefore, even when different functional blocks are described as operating subjects in the following description, they may be realized by the same hardware as the subject.

[0012] 101 is a control unit. The control unit 101 includes a processor such as one or more CPUs, and controls the operation of the entire camera 100 by reading and executing a program stored in the ROM 102 or the storage medium 108. The control unit 101 may further include a GPU. The control unit 101 also includes a hardware timer (also called a timer function) for making a shooting reservation. As will be described later, the control unit 101 determines whether a predicted image output from the image prediction unit 109 satisfies a predetermined condition based on the output from the suitability determination unit 110, and sets a shooting reservation at a future time of the predicted image.

[0013] Reference numeral 102 denotes a ROM (Read Only Memory), which is configured, for example, by a non-volatile semiconductor memory. The ROM 102 stores a control program executed by the control unit 101.

[0014] A random access memory (RAM) 103 is configured, for example, by a volatile semiconductor memory. The RAM 103 is used as a work memory for the control unit 101 to execute a program, or as a temporary storage area for various data.

[0015] Reference numeral 104 denotes an imaging unit, which is composed of, for example, an imaging lens, an aperture, a shutter, an imaging element such as a CMOS sensor, and an A / D converter. The imaging unit 104 converts an analog signal input via the imaging lens into digital data to obtain image data. The image data is stored in the RAM 103.

[0016] The imaging unit 104 generates a live view image (also called an LV image) captured by the camera and still image data for recording. The LV images are image data in a YUV format that are sequentially output from the imaging unit 104 at a predetermined frame rate during shooting operation and displayed on the display unit 106 in real time. The LV images are also input to an image prediction unit 109. The still image data for recording is high-resolution image data generated in response to a shooting instruction from a user, and is recorded in a storage medium 108 via a recording unit 107. Since the still image data for recording has a high resolution, it is compressed into, for example, a JPEG format.

[0017] An operation unit 105 receives operations from a user and notifies the control unit 101 of the input information. The operation unit 105 may be configured with at least one of a touch panel, a button, a switch, and a cross key, for example.

[0018] A display unit 106 is a processing unit that displays to a user information related to the state of the camera 100, a live view image generated by the imaging unit 104, image data recorded in a storage medium 108, and the like. The display unit 106 is configured by, for example, a liquid crystal panel or an LED.

[0019] Reference numeral 107 denotes a recording unit. The recording unit 107 controls reading and writing of data from and to a storage medium 108. The recording unit 107 controls initialization of the storage medium 108 and data transfer between the storage medium 108 and the RAM 103.

[0020] A storage medium 108 is a device having a large capacity storage area, and is configured, for example, by a memory card or a flash memory. The storage medium 108 may store a control program executed by the control unit 101.

[0021] 109 is an image prediction unit. The image prediction unit 109 configures an image prediction model. The image prediction unit 109 may be realized by a circuit or processor for executing the processing of the image prediction model, or may be realized by software for processing the image prediction model executed by the control unit 101. The image prediction unit 109 inputs an image and generates a predicted image of the image at a future time.

[0022] A specific configuration example of the image prediction unit 109 will be described with reference to FIG. 2(A). As shown in FIG. 2(A), the image prediction unit 109 includes a configuration in which a plurality of prediction units 202 are linked. When an image 201 is input, the image prediction unit 109 processes an image prediction model and outputs a plurality of predicted images 203. The prediction unit 202 generates and outputs a predicted image corresponding to a time next to an input image captured at a certain time. The predicted image is an image generated by predicting how a subject or background will be captured at a timing (second time) next to the time captured at a certain time (first time point) using an image captured at the certain time. In addition, the prediction unit 202 to which the predicted image is input further generates a predicted image at the next time. In this way, the image prediction unit 109 generates a plurality of predicted images in the same number as the number of linked prediction units 202.

[0023] FIG. 2(B) shows a specific configuration example of each prediction unit 202. The prediction unit 202 is, for example, configured by a neural network, and includes a convolutional network 204 that extracts feature information of an input image, and a recurrent neural network 205 that stores past states and infers the next state. The convolutional network 204 may be similar to a network that configures a CNN (Convolutional Neural Network), which is known as a deep learning technology for image recognition. The convolutional network 204 is a neural network that performs a convolution process that condenses feature points of an image using a filter, and a pooling process that reduces the amount of information of image data while maintaining information of important feature points. As shown in FIG. 2(B), the recurrent neural network 205 is a network that has a recurrent connection for state storage in a hidden layer (also called an intermediate layer) in the recurrent neural network 205. The recurrent neural network 205 can add features accompanied by time series changes by recurrent connection, and generates a future predicted image based on a past image. The recurrent neural network 205 may be, for example, a Long Short-Term Memory (LSTM), which is known as a recurrent neural network suitable for long-term memory. Any other configuration may be used as long as it is capable of generating a predicted image at the timing of the next time based on an image captured at a certain time. The neural network of the prediction unit 202 may be trained using training data in which, for example, an image at a first time point in a video is used as an input image, and an image at a second time point, which is the timing of the next time point in the video, is used as correct answer data.

[0024] 110 is a suitability determination unit. When an image is input, the suitability determination unit 110 determines the suitability of the image for shooting and recording. Here, suitability information is information indicating the degree to which the input image is suitable as an image to be recorded (also called a recorded image), that is, the degree to which the recorded image is preferred by the user. The suitability determination unit 110 may be realized by a circuit or processor for executing a process for determining the suitability of shooting and recording, or may be realized by software for the suitability determination process executed by the control unit 101. With reference to FIG. 3, a configuration example of the suitability determination unit 110 will be described. The suitability determination unit 110 may be configured using the above-mentioned CNN. The suitability determination unit 110 includes a convolution network 302 that extracts feature information of the input image 301, and a suitability determination network 303 in which a learning process is performed in advance using a large amount of images with high suitability as learning data. The configuration of the convolution network 302 is the same as that of the above-mentioned convolution network 205. The suitability determination network 303 is a network that configures a forward propagation type fully connected layer in the hidden layer, and outputs suitability information 304 in the output layer as a result of classifying suitability based on feature information. For example, in the case of figure skating photography, the learning data may include data in which a high suitability is assigned to an image in which the athlete reaches the peak of the jump and faces forward, and data in which a lower suitability is assigned to an image in which the athlete is starting to jump or does not face forward. The learning data is not limited to the example of figure skating, and may include images in which high or low suitability is assigned for various sports and various shooting scenes.

[0025] For example, the control unit 101 determines whether a predicted image is suitable as a recording image based on the suitability information output from the suitability determination unit 110. In addition, the control unit 101 sets a shooting reservation for a future time of the predicted image determined to be suitable as a recording image.

[0026] Reference numeral 111 denotes an internal bus that connects control signals exchanged among the above-mentioned processing units.

[0027] <Automatic photography processing overview> Next, an overview of the process from the generation of an LV image to the automatic shooting according to this embodiment (also simply referred to as automatic shooting process) will be described. The automatic shooting process according to this embodiment is assumed to be used as a shooting assist function, as an example. For example, consider a case where the automatic shooting process is applied to shooting figure skating. If an athlete who is about to jump is included in the LV video, it becomes possible to automatically shoot at the timing when the athlete reaches the peak of the jump and faces forward, that is, at a timing suitable for shooting.

[0028] An overview of the automatic shooting process will be described below with reference to Fig. 4. The imaging unit 104 generates LV images at regular time intervals while the imaging unit 104 is operating, and transmits the generated LV images to the display unit 106 and the image prediction unit 109. As an example in this embodiment, the LV images are generated as YUV data with a resolution of 800 x 600 at a speed of 20 frames per second (at 50 millisecond intervals). The display unit 106 displays the transmitted LV images in real time.

[0029] The image prediction unit 109 generates predicted images at a plurality of future times based on the input LV image, and stores the generated predicted images in a predicted image buffer 401. The predicted image buffer 401 is a memory area configured as part of the RAM 103 for temporarily storing predicted images.

[0030] In this embodiment, the image prediction unit 109 is, as an example, configured with 20 prediction units 202. The image prediction unit 109 generates predicted images up to 1 second later (50 milliseconds x 20) using LV images input at 50 millisecond intervals. For example, the relationship between the predicted images generated by the image prediction unit 109 and time is as shown in FIG. 5.

[0031] At time t0, the image prediction unit 109 predicts an LV image f t0 , and the predicted image f t1 ~f t20Then, at the next time t1, 50 milliseconds later, the image prediction unit 109 generates the LV image f t1 , and a predicted image f t2 ~f t21 Similarly, at subsequent times t2 and t3, 20 predicted images are generated.

[0032] Next, the control unit 101 inputs the predicted image stored in the predicted image buffer 401 to the suitability determination unit 110 and acquires suitability information from the suitability determination unit 110. The control unit 101 determines whether or not shooting and recording should be performed for the corresponding predicted image based on the suitability information acquired from the suitability determination unit 110, and when it is determined that shooting and recording should be performed, sets a shooting reservation according to the corresponding future time. Specifically, the control unit 101 executes a timer function so that shooting and recording by the imaging unit 104 is performed according to the corresponding future time, and records the predicted image and suitability information of the target of the shooting reservation in the shooting reservation management table. The shooting reservation management table is stored in the storage medium 108, for example, and is configured as a table shown in FIG. 6. The ID shown in FIG. 6 is a number for identifying each shooting reservation, and may also correspond to the shooting execution order. The predicted image ID shown in FIG. 6 is information for identifying the predicted image that is the target of the shooting reservation, and is expressed so that, for example, the corresponding time information can be known. The time information may be a relative time from the LV image, or may be an absolute time. The suitability shown in Fig. 6 is information indicating the suitability of a predicted image that is the subject of a shooting reservation, and is shown, for example, in a percentage (%) out of 100. The suitability may be other numerical values, such as decimal values ​​from 0 to 1, as long as it indicates the degree of suitability as a recording image. The example shown in Fig. 6 shows a state in which three shooting reservations have been set.

[0033] Note that the above-mentioned series of processes from image prediction to shooting reservation is executed while ensuring real-time performance so that the processes are completed within the LV image generation cycle (for example, 50 milliseconds).

[0034] Next, a series of operations related to the automatic shooting process will be described with reference to FIG. 7. This process is realized by the control unit 101 expanding and executing a program stored in the ROM 102 or the storage medium 108 in the working area of ​​the RAM 103, unless otherwise specified. This process is executed in a state in which the machine learning model (neural network) in the image prediction unit 109 and the suitability determination unit 110 has already learned. Furthermore, this process is started when the operation mode of the camera 100 is transitioned to the shooting mode by, for example, a user operation on the operation unit 105. Here, the shooting mode is one of the operation modes of the camera 100, and is an operation mode in which a user can shoot a still image at any time by pressing a shooting / recording button. For example, an operation mode in which the imaging function is not operated and a still image recorded on a memory card is reproduced and displayed on a liquid crystal monitor does not fall under this category.

[0035] In S701, the imaging unit 104 generates one frame of an LV image and outputs it to the image prediction unit 109. In S702, the image prediction unit 109 generates predicted images corresponding to a plurality of future times based on the LV image output from the imaging unit 104, and stores the generated predicted images in the predicted image buffer 401. When the image prediction unit 109 stores the generated images in the image buffer 401, it notifies the control unit 101. In S703, the control unit 101 takes out one predicted image from the predicted image buffer 401, inputs it to the suitability determination unit 110, and acquires corresponding suitability information from the suitability determination unit 110.

[0036] In S704, the control unit 101 uses the suitability information acquired from the suitability determination unit 110 to determine whether or not the shooting reservation should be executed. For example, when the suitability is greater than a predetermined threshold, the control unit 101 determines that the shooting reservation of the corresponding predicted image should be executed. The predetermined threshold may be, for example, 70%. When the suitability is greater than the predetermined threshold, the control unit 101 determines that the shooting reservation should be executed and proceeds to S705, and when the suitability is equal to or less than the predetermined threshold, the control unit 101 determines that the shooting reservation should not be executed and proceeds to S711.

[0037] In S705, the control unit 101 judges whether or not there is another shooting reservation at a time close to the future time judged to be reserved in S704, by referring to the shooting reservation management table in the storage medium 108. If the control unit 101 judges that a shooting reservation is set, the control unit 101 proceeds to S709, and if the control unit 101 judges that a shooting reservation is not set, the control unit 101 proceeds to S706. Here, the state in which another shooting reservation is already set at a close time means a state in which both shooting reservations cannot be made effective because the time interval between them is narrower than a predetermined time range, for example, shorter than the time required for shooting and recording processing for one image. In this embodiment, as an example, the time required for shooting and recording processing for one image is 200 milliseconds. In this case, the judgment of whether or not there is another shooting reservation set at a close time is a judgment of whether or not there is another shooting reservation included within 200 milliseconds before and after. In this way, by judging the overlap of shooting reservations, it is possible to avoid setting a shooting reservation that is difficult to execute.

[0038] In S706, the control unit 101 sets the image capture reservation determined to be reserved in S704. Specifically, a timer function is executed so that the image capture unit 104 performs image capture and recording at the corresponding future time, and information on the predicted image and suitability for the image capture reservation is added to the image capture reservation management table in the storage medium 108. In addition, in this step, the control unit 101 displays information notifying that an image capture reservation has been set in the camera 100 (automatic image capture is being performed) on the display unit 106 to alert the user to continue capturing the subject by the image capture unit 104. For example, the screen indicating that automatic image capture is in progress may be a screen 801 shown in FIG. 8. 801 is the entire screen, and 802 is notification information displayed by this step. The notification information 802 includes, for example, information indicating that an image capture reservation has been set (automatic image capture is being performed), and information indicating the period of automatic image capture and the number of images to be automatically captured. The notification information 802 may be configured using information in the image capture reservation management table. Notification information 802 shown in FIG. 8 shows an example in which information is displayed in text, but is not limited to text and may be displayed as an icon, or the notification information may be notified by voice.

[0039] In S707, the control unit 101 determines whether or not all of the predicted images stored in the predicted image buffer 401 in S702 have been extracted in S703. If the control unit 101 determines that all of the predicted images stored in the predicted image buffer 401 have been extracted, the control unit 101 proceeds to S708, and if the control unit 101 determines that all of the predicted images have not been extracted, the control unit 101 returns to S703.

[0040] In S708, the control unit 101 determines whether the operation mode of the camera 100 has been changed from the shooting mode to another mode by the user. If the operation mode has been changed to another mode, this flowchart ends. If the shooting mode is continued, the process returns to S701.

[0041] In S709, the control unit 101 performs processing in the case where a shooting reservation exists at a nearby time. Specifically, the control unit 101 refers to the shooting reservation management table and determines whether the suitability of the predicted image determined to be reserved in S704 is higher than the suitability of the predicted image detected to have been reserved for shooting at a nearby time in S705. If the control unit 101 determines that the suitability of the new predicted image determined to be reserved is higher than the suitability of the set predicted image, the control unit 101 proceeds to S710, and if not, the control unit 101 interrupts the processing of the shooting reservation determined to be reserved in S704 and proceeds to S707.

[0042] In S710, the control unit 101 cancels the already set shooting reservation detected in S705, and sets a new shooting reservation determined to be reserved in S704. Here, in order to cancel the shooting reservation, the control unit 101 stops the corresponding timer function and deletes the corresponding reservation information from the shooting reservation management table. The contents of the shooting reservation setting process are the same as the contents of the shooting reservation setting process described in S706. In this way, when there are adjacent predicted images, a shooting reservation of a more suitable predicted image is given priority, so that an image more useful to the user can be obtained by automatic shooting.

[0043] In S711, the control unit 101 determines whether or not a shooting reservation has already been set for the same future time as that determined in S704 for which the shooting reservation should not be executed. The control unit 101 refers to the shooting reservation management table in the storage medium 108, and if it determines that a shooting reservation has already been set for the same future time as that determined in S704, the control unit 101 proceeds to S712, otherwise proceeds to S707.

[0044] In S712, the control unit 101 cancels the already set shooting reservation detected in S711. The contents of the shooting reservation cancellation process are the same as the contents of the shooting reservation cancellation process described in S710. At this time, in this step, the control unit 101 updates the display of the notification information started in S706 in association with the cancellation of the shooting reservation. Alternatively, information notifying that the shooting reservation has been cancelled may be displayed on the display unit 106. When there is no shooting reservation (when all shooting reservations are cancelled), the display of the notification information started in S706 may be erased. When a future time that was determined to be suitable for shooting and recording in the image prediction of the previous LV image is determined to be unsuitable in the image prediction of the subsequent LV image by S711 and S712, the existing shooting reservation can be cancelled. After that, the control unit 101 executes S707 and S708 as described above to end a series of operations of the automatic shooting process.

[0045] In the above embodiment, the case where the number of the suitable determination units 110 configured in the camera 100 is one has been described as an example. However, the number of the suitable determination units 110 is not limited to this, and the camera 100 may be configured to be able to control a plurality of suitable determination units 110. Although the scale of the system configuration increases by configuring a plurality of suitable determination units 110, it becomes possible to simultaneously execute the suitable determination process for a plurality of predicted images in parallel, and it becomes possible to shorten the overall processing time. By using this method, it becomes possible to increase the number of predicted images output from the image prediction unit 109. This makes it possible to execute image prediction for a more distant future time.

[0046] In the above embodiment, the image prediction unit 109 and the suitability determination unit 110 are configured by a neural network. When using the image prediction unit 109 or the suitability determination unit 110 equipped with a neural network that requires a large amount of calculation, a mechanism for reducing the processing load may be further implemented. For example, after one shooting reservation is set by the control unit 101, the control unit 101 may not set a new shooting reservation until shooting recording by that shooting reservation is performed or until that shooting reservation is canceled. Then, the control unit 101 may only adjust the time of the shooting reservation that has already been set. For example, the control unit 101 may perform the suitability determination process by the suitability determination unit 110 only for predicted images at times close to the shooting reservation time, rather than for all predicted images generated by the image prediction unit 109. Here, the time close to the shooting reservation time may be, for example, a time included within 300 milliseconds before and after the shooting reservation time. In this way, by limiting the time to be the target of the shooting reservation, it is possible to reduce the processing load of the suitability determination unit 110 and the associated reduction in power consumption of the entire camera 100.

[0047] As described above, in this embodiment, a predicted image is generated by predicting an image at a second time point future than the first time point using an image at a first time point output from the imaging unit 104. Then, a suitability (degree to which the predicted image is suitable as an image to be recorded) of the generated predicted image is determined, and when the determined suitability is higher than a predetermined threshold, an image is captured when the time reaches the second time point. This makes it possible to automatically capture an image at a suitable timing predicted from a current image. In other words, it is possible to realize a system that uses an image prediction model to predict a future image suitable for capture from a current LV image and automatically captures the image. In addition, in this embodiment, after a capture reservation is set for a future time, if it is determined that the capture at that time is not suitable based on an LV image generated thereafter, the capture reservation is canceled. In this way, it is possible to adjust the future time for which a capture reservation has been set once so that it becomes the correct capture timing every time an LV image is generated, that is, as the set future time approaches the current time.

[0048] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0049] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0050] 101...control unit, 104...imaging unit, 105...operation unit, 106...display unit, 108...storage medium, 109...image prediction unit, 110...preferred determination unit

Claims

1. a prediction means for generating a predicted image by predicting an image at a second time point that is future than the first time point, using an image at a first time point output from an imaging means; a determination means for determining a suitability of the predicted image, the suitability indicating a degree to which the predicted image is suitable as an image to be recorded; and a control means for controlling the imaging means to capture an image when the time reaches the second time point when the determined suitability is higher than a predetermined threshold value, The control means if the determined suitability is higher than the predetermined threshold, setting a photography reservation for taking an image when the time reaches the second time point; An imaging device characterized in that, after setting a first shooting reservation for capturing an image at the second time point based on an image at the first time point, it is determined whether the desirability of the predicted image generated using an image at a third time point that is later than the first time point is higher than the desirability of the predicted image generated using the image at the first time point.

2. The imaging device according to claim 1 , wherein the prediction means is configured with a machine learning model that generates the predicted image corresponding to a time point in the future from the first time point based on the image at the first time point.

3. The imaging device according to claim 1 or 2, wherein the determining unit is configured with a machine learning model that determines the desirability of an input image.

4. 4. The imaging device according to claim 1, wherein the image at the first time point is a live view image sequentially output from the imaging means.

5. 2. The imaging apparatus according to claim 1, wherein the control means, in response to setting the photographing reservation, causes a display means to display information notifying that the photographing reservation has been set.

6. The imaging device of claim 1, characterized in that, when the desirability of the predicted image generated using the image at the third time point is higher than the desirability of the predicted image generated using the image at the first time point, the control means cancels the first shooting reservation and sets a second shooting reservation based on the image at the third time point.

7. The imaging device of claim 1 or 5, characterized in that after setting a first shooting reservation for capturing an image at the second time point based on an image at the first time point, the control means cancels the first shooting reservation if the desirability of the predicted image for the second time point or a predetermined time range from the second time point, generated using an image at a third time point that is after the first time point, is equal to or lower than the predetermined threshold value.

8. The imaging device according to claim 6 or 7, characterized in that, in response to canceling the first shooting reservation, the control means causes a display means to display information notifying that the shooting reservation has been canceled, or changes the information notifying that the shooting reservation has been set.

9. 8. The imaging device according to claim 7, wherein the predetermined time range is a time required for an image to be output from the imaging means and recorded in a storage medium.

10. the prediction means outputs predicted images at a plurality of time points in the future from the first time point; The imaging device described in any one of claims 1 and 5 to 9, characterized in that the control means controls the imaging means to capture an image when a predicted image point in time reaches a point in time where the desirability determined for the predicted images at the multiple points in time is higher than the predetermined threshold value.

11. The imaging device according to claim 10, characterized in that the determination means, in response to setting the shooting reservation, determines the desirability only for the predicted images for the multiple time points that are within a predetermined time range from the time point at which the shooting reservation is set, among the predicted images for the multiple time points.

12. 12. The imaging apparatus according to claim 1, further comprising a recording unit for recording the image captured at the second time point in a storage medium.

13. a prediction step of generating a predicted image by predicting an image at a second time point that is future than the first time point, using an image at a first time point output from an imaging means; a determination step of determining a suitability level of the predicted image, the suitability level indicating a degree to which the predicted image is suitable as an image to be recorded; and a control step of controlling the imaging means to capture an image when the time reaches the second time point when the determined suitability is higher than a predetermined threshold value, In the control step, if the determined suitability is higher than the predetermined threshold, setting a photography reservation for taking an image when the time reaches the second time point; A method for controlling an imaging device, characterized in that after setting a first shooting reservation for capturing an image at the second time point based on an image at the first time point, it is determined whether the desirability of the predicted image generated using an image at a third time point that is later than the first time point is higher than the desirability of the predicted image generated using the image at the first time point.

14. A program for causing a computer to function as each of the means of the imaging apparatus according to any one of claims 1 to 12.

Images