Imaging device, control method thereof, program, and storage medium

Abstract

To capture a good panning image.SOLUTION: An imaging device includes an imaging unit 101 for capturing an image of a moving body to generate an image, a calculation unit for calculating a mobile vector of the moving body from images of a plurality of frames captured by the imaging unit, a prediction unit for predicting, based on the mobile vector, a prediction mobile vector which is the vector of the moving body in a next frame, and a light exposure controller for controlling an exposure time for each pixel group of the imaging unit so that an exposure time of a second region which is a region including an end point of the prediction mobile vector is shorter than an exposure time of a first region which is a region including a start point of the prediction mobile vector.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an imaging apparatus, a control method thereof, a program, and a storage medium. [Background technology]

[0002] A technique for panning has been disclosed in the past (Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2012-142673 Summary of the Invention [Problem to be solved by the invention]

[0004] The problem to be solved by the present invention is to capture a good panning image. [Means for solving the problem]

[0005] In order to solve the above problem, an imaging device according to one aspect of the present invention includes an imaging unit that captures an image of a moving object to generate an image, a calculation unit that calculates a motion vector of the moving object based on a latest frame and a frame before the latest frame among a plurality of frames captured by the imaging unit, a prediction unit that predicts a predicted motion vector that is a vector representing the amount of movement and direction of movement of the moving object from the latest frame to a next frame a predetermined time after the latest frame based on the motion vector, and an exposure control unit that controls exposure time for each pixel group of the imaging unit so that exposure time for a second region that is a region including an end point of the predicted motion vector is shorter than exposure time for a first region that is a region including a start point of the predicted motion vector, information The area where there is a change in the moving object in the latest frame is defined as a first moving object area including the moving object in the latest frame. andThe method is characterized in that it determines a second moving object region including the moving object in the next frame by determining the first moving object region and moving the first moving object region based on the predicted motion vector, and determines the start point of the predicted motion vector from the first moving object region and the end point of the predicted motion vector from the second moving object region. [Effects of the Invention]

[0006] According to the present invention, a good panning image can be captured. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram showing an example of an imaging apparatus according to an embodiment of the present invention. [Figure 2] 10 is a flowchart showing an example of a control method for an imaging apparatus according to an embodiment of the present invention. [Figure 3] 1A and 1B are diagrams showing the relationship between a moving body, a photographing angle of view, and a photographed image according to an embodiment of the present invention. [Figure 4] 10A and 10B are examples of captured images according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. The embodiments described below are examples of means for realizing the present invention, and should be appropriately modified or changed depending on the configuration of the device to which the present invention is applied and various conditions. The present invention is not limited to the following embodiments. Furthermore, a configuration may be made by appropriately combining parts of each embodiment described below.

[0009] <Embodiment 1> First, an embodiment of the present invention will be described. Fig. 1 is a diagram showing an example of an imaging device 100 according to an embodiment of the present invention.

[0010] The imaging device 100 includes an imaging section 101, an imaging optical system 102, an imaging element 103, an image processing section 104, a memory 105, an imaging system control section 106, a display section 107, and an imaging device control section .

[0011] The imaging unit 101 includes an imaging optical system 102 and an imaging element 103 .

[0012] The imaging optical system 102 includes a plurality of lenses, a diaphragm, and optical filters such as an infrared cut filter.

[0013] The image sensor 103 includes a photoelectric conversion element, such as a CCD or CMOS, that can change the exposure conditions (exposure time and analog gain) for each pixel group. A pixel group may consist of a single pixel or multiple pixels, and the smallest area for which the exposure conditions can be changed is called an exposure area. The image sensor 103 obtains luminance information for each pixel group and can individually set the exposure time for each exposure area based on the luminance information. The image sensor 103 also calculates the exposure conditions for the next frame based on the luminance information for the previous frame for each exposure area. The exposure conditions are determined by the number of exposure time steps and the number of analog gain steps. Therefore, if the exposure time is increased by one step, an image can be captured with the same exposure amount by decreasing the analog gain by one step. In this way, the image sensor 103 adjusts the exposure amount by changing the analog gain for each pixel group, thereby correcting the brightness of the captured image. The image sensor 103 also has a brightness correction function using digital gain. The brightness of the output image can be corrected by correcting the difference in the number of exposure conditions using digital gain.

[0014] The image processing unit 104 performs predetermined image processing (for example, white balance adjustment processing, gamma correction processing, etc.) on the image generated by the image sensor 103, and then outputs the image to the memory 105. At this time, the image processing unit 104 outputs an image file in a predetermined format, such as JPEG, H.264, or H.265, or a video file to the memory 105. It is also possible to read data from the memory 105.

[0015] The image processing unit 104 also functions as a calculation unit that calculates a motion vector based on image information generated by the image sensor 103. The image processing unit 104 (calculation unit) calculates a motion vector for each exposure area that is the same as an exposure area in which the exposure time can be changed for at least each pixel group of the image sensor 103. The image processing unit 104 (calculation unit) also determines exposure areas where brightness has changed based on multiple pieces of image data after predetermined image processing, and calculates a motion vector from changes in the motion area. The motion vector may also be calculated using known methods such as block matching or optical flow. Information on the motion vector calculated by the image processing unit 104 (calculation unit) is transmitted to the imaging device control unit 108.

[0016] The motion vector is calculated by the image processing unit 104 (calculation unit) from multiple images captured by the image sensor 103, and also functions as a prediction unit that predicts a predicted motion vector, which is the motion vector for the next frame. The image processing unit 104 (prediction unit) predicts the direction and magnitude of the motion vector for the next frame from the motion vector calculated by the image processing unit 104 (calculation unit).

[0017] The memory 105 includes a volatile memory such as an SRAM or a DRAM, and a non-volatile memory such as a flash memory, and temporarily stores data output by the image processing unit 104 .

[0018] Based on the input signal, the imaging system control unit 106 controls the imaging unit 101. The imaging system control unit 106 has a function as an exposure control unit, and controls the exposure time for each pixel group of the image sensor 103 in the imaging unit 101.

[0019] The display unit 107 has a display made up of a liquid crystal or organic EL display, etc., and displays the image generated by the image processing unit 104. In addition to the image, the display unit 107 also displays superimposed shooting conditions such as the current exposure time, and the setting area. The display unit 107 has a touch panel, and accepts input from the user, such as specifying an area by dragging or tapping, or adjusting the exposure time.

[0020] The imaging device control unit 108 transmits and receives signals to the image processing unit 104, the imaging system control unit 106, and the display unit 107. The imaging device control unit 108 transmits the image processing content to be performed to the image processing unit 104. The imaging device control unit 108 acquires the direction and length of the movement vector calculated by the image processing unit 104. The imaging device control unit 108 sends a control signal regarding the exposure time for each region of the image sensor 103 to the imaging system control unit 106. The imaging device control unit 108 transmits images to be displayed on the display unit 107, as well as data for superimposing the shooting conditions and set regions. The imaging device control unit 108 reads information regarding adjustment of the region and exposure time specified by the user on the display unit 107.

[0021] The procedure of this embodiment will be described with reference to Fig. 2. Fig. 2 is a flowchart showing an example of a control method for an imaging device according to an embodiment of the present invention.

[0022] In S200, the first frame is photographed and the video signal is sent to the image processing unit 104. The image processing unit 104 stores the photographed result in the memory 105.

[0023] In S201, the second frame is photographed, and the video signal is sent to the image processing unit 104 in the same manner as the first frame. The image processing unit 104 stores the photographed result in the memory 105.

[0024] In S202, a moving object vector is calculated from the difference between the captured images of the first and second frames. The movement vector can be calculated from changes in brightness information. The movement vector calculation method is described in detail with reference to FIG. 3. The movement vector calculated from the image difference between the first and second frames is set as the movement vector for the second frame. A predicted movement vector corresponding to the movement vector for the third frame is predicted from the calculation result of the movement vector for the second frame. The direction of the predicted movement vector is the same as that of the movement vector for the second frame. The length of the predicted movement vector also varies depending on the frame interval (time) of the third frame. For example, if each frame is always captured at the same interval, the length of the predicted movement vector will also be the same. However, in the present invention, to capture a sense of speed, the third frame may be exposed (continuously) across multiple frames to lengthen the trajectory of the moving object. In addition, it is preferable to determine the frame interval of the third frame so that the moving object fits within the image in the next frame. Therefore, it is preferable to determine the interval of the third frame (the timing at which exposure of the third frame ends) based on the position and movement speed of the moving object in the image. That is, the length of the predicted motion vector is extended or shortened by the time magnification of the frame interval of the third frame relative to the frame interval of the second frame.

[0025] In S203, the exposure time is set to a value that results in a long exposure time (for example, 1 / 1 second) for the start point region that includes the position of the start point of the moving object, and a short exposure time (for example, 1 / 4000 second) for the end point region that includes the position of the end point. The start point position and end point position will be explained using Figure 3. Also, the short and long seconds are determined using relative values, not absolute values. However, it is preferable to set the start point region to a longer exposure time to better express blur. In other words, it is sufficient if the exposure time for the end point region is set shorter than the exposure time for the start point region. Details will be explained in Figure 3(D). It is preferable to set the end point region to a shorter exposure time to clearly capture the subject.

[0026] In S204, brightness is corrected. There are two main methods for correcting brightness. The first is to correct brightness at the time of shooting by setting an analog gain. The second is to correct brightness for the captured image using digital gain. These two methods may also be combined. Brightness correction using analog gain will now be described. The analog gain is set based on the exposure time set in S203. At this time, the number of analog gain steps is changed by the difference in the number of exposure time steps for each exposure area, correcting the exposure condition so that the number of steps is uniform across the image. Furthermore, exposure conditions may be set so that each exposure area has appropriate exposure based on the luminance information for each exposure area in the previous frame. This makes it possible to capture an image with an appropriate exposure amount even in scenes with differences in luminance. Brightness correction using digital gain will now be described. In image processing after capture, correction using a digital gain with the same number of steps as the difference in the number of exposure conditions for each exposure area can brighten bright areas and darken dark areas when viewed as a whole, resulting in a natural image.

[0027] In S205, an image is captured under the exposure conditions (exposure time and analog gain) determined in S203 and S204.

[0028] With reference to (A), (B), (C), and (D) of FIG. 3, the calculation method of the predicted moving object vector shown in S200 to S202 of FIG. 2 will be described. Furthermore, with reference to (D) of FIG. 3, the setting method of the exposure time shown in S203 of FIG. 2 will be described. FIG. 3 is a diagram showing the relationship between the moving object, the shooting angle of view, and the captured image in an embodiment according to the present invention. FIGS. 3(A), 3(B), and 3(C) all show different frames. For simplicity, the description will be made assuming that each frame shows a different frame. FIG. 3(D) shows the prediction of the moving object in the next frame of FIG. 3(C), predicted from FIGS. 3(A) to 3(C), and a method of setting the exposure time according to the prediction.

[0029] FIG. 3(A) is an explanatory diagram for clarifying the present invention. In FIG. 3(A), a moving object 301 is a subject moving at a constant (not significantly changing) speed in the direction of the arrow (dotted line) shown. In FIG. 3(A), the moving object 301 is not present within the shooting angle of view 302. Also, image 303 shows an image captured at the shooting angle of view 302, and illustrates a grid-shaped exposure area 304. The exposure area 304 is used to clarify changes in the image and the method for setting the shooting conditions. The shape and number of exposure areas 304 may be any shape or number as long as one exposure area is composed of at least a single pixel.

[0030] FIG. 3B shows a situation where a moving object 301 appears within the imaging angle of view 302, and corresponds to S200 in FIG. 2. The following description will be based on the image in FIG. 3 as the first frame. When the moving object 301 enters the imaging angle of view 302, the luminance information of the area where the moving object 301 is located changes. Based on this change in luminance information, the area where the moving object is located and its vector are calculated. In FIG. 3B, luminance information for each area is calculated, and an area where the luminance information changes between frames is determined to be the area where the moving object 301 is located (hereinafter referred to as a moving object area). The moving object area in FIG. 3B is indicated by a bold frame as moving object area 305B in image 303 in FIG. 3B. In addition, to reduce false detections due to noise, etc., it is desirable to set a threshold for the change in luminance information required to detect a moving object area, and to determine an area where the luminance information changes by a certain amount or more as a moving object area. A moving object area is preferably composed of one or more exposure areas. This allows the determination of the moving object region to be performed for each exposure region, thereby reducing the image processing load compared to processing for each pixel. The leading region 307 shown in Figure 3(B) will be explained in Figure 3(C), and is the region located in the moving object region in the direction of travel of the moving object.

[0031] Figure 3(C) shows the state of the next frame relative to Figure 3(B), and corresponds to S201 and S202 in Figure 2. In Figure 3(C), the moving object region is calculated in the same way as in Figure 3(B). The moving object region in Figure 3(C) is shown as moving object region 305C in a bold frame within image 303 in Figure 3(C).

[0032] At this time, a movement vector is calculated by comparing the shapes, including the positions and sizes, of moving object region 305B and moving object region 305C. Comparing moving object region 305B and moving object region 305C, it can be seen that the moving object region has expanded (moved) horizontally to the right by the width of one exposure region. Calculating the movement vector from this results in movement vector 306 in FIG. 3(C). The direction of movement vector 306 is the horizontal right direction from the direction of change in the moving object region, and the horizontal right direction is calculated as the traveling direction of moving object 301. The length of movement vector 306 is calculated as the width of one exposure region from the amount of change in the moving object region.

[0033] In addition, the exposure area 304 located at the leading end of the moving object area 305C relative to the movement vector (direction of travel) is designated as the leading area 307 and is shown as the shaded area in FIG. 3(C). Similarly, a leading area 307 is also shown within the moving object area 305B in FIG. 3(B). The shutter end time for the next frame is controlled so that this leading area 307 fits within the angle of view. Here, the leading area is five areas from the edge of the angle of view in the moving object's direction of travel. Furthermore, since the moving object moves only one area per frame, it is possible to set up to five frames to fit the moving object 301 within the angle of view. If the setting exceeds five frames, the moving object 301 will be cut off from the shooting angle of view, making it impossible to capture the desired image. Furthermore, if the setting is made to the very limit of five frames, the moving object 301 may be cut off depending on changes in movement speed or detection accuracy. Therefore, it is preferable to set the setting with a margin of at least one area. Here, shooting is performed at a frame interval of four frames. In this case, it is preferable to perform exposure across frames and capture one frame with a single readout. This is because if multiple readouts are combined, the image quality will be reduced due to readout noise. Figure 3(D) shows the predicted capture state of the next frame.

[0034] A method for setting the exposure time, which is one of the exposure conditions, will be described with reference to Fig. 3(D). Fig. 3(D) corresponds to S202, S203, S204, and S205 in Fig. 2. Fig. 3(D) also explains a method for setting the exposure time.

[0035] 3B and 3C, the boundary between the leading region of leading region 307 and the background region (region that is not a moving object region) is used as the detection point for the motion vector, and motion vector 306 is calculated. The predicted motion vector calculated in S202 of FIG. 2 at this time is shown as predicted motion vector 308 in FIG. 3D.

[0036] Furthermore, the moving object region predicted based on the predicted motion vector is 305D. At this time, moving object region 305D has a shape obtained by moving the shape of moving object region 305C in the direction of motion of the motion vector. Furthermore, since the direction opposite to the direction of motion of the motion vector cannot be predicted, it is preferable to determine moving object region 305C as a moving region as it is.

[0037] The exposure time is set based on the start and end positions of the moving object 301. The start point of the moving object 301 may be located anywhere in the moving object region 305C in FIG. 3(C). The end point of the moving object 301 may be located closer to the moving object 301's direction of travel than the start point position in the predicted moving object region 305D in FIG. 3(D). Here, the point in the moving object region 305C in the image of the second frame that is located furthest in the moving object's direction of travel is defined as start point position 309, and the point in the moving object region 305D predicted in the third frame that is located furthest in the moving object's direction of travel is defined as end point position 310. In the description of this embodiment, the length of the predicted motion vector and the distance between the start and end points are identical, but they do not necessarily have to be identical.

[0038] In Figure 3(D), the center of the start point position and the end point position are set as the boundary, and all exposure areas on the opposite side of the moving direction of the moving object 301 are set as the start point region, and all exposure areas in the moving direction of the moving object 301 are set as the end point region. At this time, in order to capture an image with a sense of speed, the start point region is set to a long exposure time, and the end point region is set to a short exposure time.

[0039] However, it is preferable that the exposure time be determined so that the end timing of the exposure time is the same for each area and the start timing of the exposure time is different for each area, which makes it possible to capture the moving object 301 at the exposure timing of the end area, which is set to a short second.

[0040] An image captured in this embodiment will be described with reference to FIG. 4. FIG. 4 is an example of a captured image according to an embodiment of the present invention. Captured image 400 corresponds to the image captured in S203 of FIG. 2 and FIG. 3(D). The image captured in the direction of travel of the moving object is clear and blur-free because it was captured at a short exposure time. The image captured in the opposite direction of travel is blurred and has a sense of speed because it was captured at a long exposure time. In this way, it is possible to capture an image of the same moving object in a single image that clearly captures the moving object while expressing a sense of speed through blur. Furthermore, by setting the start point position closer to the opposite side of the traveling direction or narrowing the range of the start point area, it is possible to capture the entire moving object clearly while retaining some blur. Furthermore, by arbitrarily setting the start point position and end point position or the range of the start point area and the range of the end point area, the user can freely change the blurred and clear areas.

[0041] As explained above, the end region is an image with a short exposure time, resulting in little blur of the moving object, while the start region is an image with a long exposure time, resulting in a large blur of the moving object.In other words, the end region captures the moving object clearly, while the start region is an image with a sense of speed and blur, resulting in a good panning image.

[0042] The present invention expresses a sense of speed through the blur of a moving object. In other words, while it is necessary to express the blur of a moving object in an image, it is not necessary to express the blur of the background. Therefore, for the background region, the exposure conditions may be determined based solely on brightness information, without setting the exposure conditions in S203 and S204 of FIG. 2. In this case, the exposure time is automatically set to ensure appropriate exposure for regions other than the moving object region. The exposure conditions for the start and end regions may also be manually set by the user in advance. However, the image processing unit 104 (prediction unit) automatically calculates the predicted motion vector and the start and end regions, and sets the user-specified exposure conditions for each exposure region of the start and end regions. It is also assumed that, when the maximum exposure time is determined, the user will capture the moving object within the desired angle of view. In this case, the imaging device 100 starts capturing (exposing) when the position of the moving object region predicted from the maximum exposure time becomes the desired position.

[0043] Furthermore, the start and end regions may each be composed of one or more exposure regions that include the start and end positions. It is preferable to set the exposure time for the region of the moving object predicted in Figure (D) that does not belong to either the start or end region to values ​​that gradually decrease in seconds as the region approaches the end region. This will smooth out any blurring in the captured image, resulting in a more natural-looking image.

[0044] In the embodiment, the method of setting the end region to a shorter time than the start region is shown, but the reverse is also possible. In this case, a clear moving object is captured in the end region, and blur occurs in the start region, so an image with a sense of speed, like reverse playback, can be captured.

[0045] It is preferable to use brightness information when setting the exposure time. By limiting the exposure time to the maximum in bright areas and the minimum in dark areas based on the brightness information in the image, it is possible to prevent overexposure and underexposure.

[0046] Furthermore, since the motion vector of the moving object is calculated, it is preferable to determine the exposure conditions of the predicted moving object region based on the exposure conditions of the moving object region before the movement. This allows the exposure conditions of the predicted moving object region to be set appropriately. For example, the exposure conditions of the leading region in Figure 3(D) are determined based on the exposure conditions and brightness information of the leading region in Figure 3(C). This is not limited to the leading region.

[0047] Although the detection point has been described as the boundary of the region on the leading side of the leading region, it can be set arbitrarily as long as it is within the moving object region.

[0048] In the explanation of Figure 2, the motion vector is calculated from the difference between two images, the first and second frames. Calculating the motion vector from multiple images improves the accuracy of the predicted motion vector. Specifically, capturing multiple frames of a moving object, calculating the motion vector between each frame, and averaging it improves the calculation accuracy of the predicted motion vector. Furthermore, rather than just averaging, the calculation accuracy of the predicted motion vector can be improved by extracting the difference in the length or direction of the motion vector and calculating the change in the acceleration or direction of the moving object.

[0049] If the amount of movement of the moving object is known in advance, the frame interval in S205 may be determined manually by the user.

[0050] The location of the moving object in the image varies depending on the user's preference. Therefore, it is preferable that the display unit 107 has an operation unit that allows the user to specify the end point position. The imaging device control unit 108 adjusts the frame time so that the end point of the predicted movement vector is at the specified position, depending on the end point position specified by the user. This makes it possible to fit the moving object in the image according to the user's preference.

[0051] In S200, for ease of explanation, the first frame is used in the description, but it is preferable to take consecutive frames and use the frame in which a moving object is detected as the first frame.

[0052] Here is some additional information about motion vectors. Motion vectors are calculated based on changes in pixel brightness information between frames. Therefore, even if the exposure conditions (exposure time and analog gain) for each region are different, the brightness information is corrected so that it is based on the same standard exposure conditions. For example, consider region X, which is the standard exposure condition, and region Y, which is set to an exposure condition with an exposure time one level higher. In this case, even if region Y is photographing a subject of the same brightness as region X, the pixel signal level will be one level higher. For this reason, a standard is set for the exposure conditions, and the brightness information is corrected to match the exposure conditions before calculation. In other words, the brightness information for region Y is calculated one level lower.

[0053] A supplementary explanation of how to adjust the focus is provided. It is preferable to adjust the focus to match the image captured by the user. However, to make image capture easier, an automatic focus adjustment method will be described. To perform the focus adjustment described in this embodiment, the imaging device 100 includes a distance calculation means, such as a distance sensor or dual-pixel phase-difference AF. There are two focus adjustment methods. The first is to adjust the focus to a moving object. The focus is adjusted based on distance information about the moving object area captured in the previous frame. This enables focus adjustment to match the moving object. The second is to adjust the focus to a predicted moving object area or its ground surface. Because the end area is where a moving object is clearly captured, it is preferable to focus on the end area. Therefore, focusing on the end area allows the focus to be adjusted to the area that you want to capture clearly. Furthermore, by focusing on the point with the shortest distance within the end area, it becomes easier to focus on the moving object and the ground surface, making it possible to capture the moving object itself more clearly.

[0054] A method for more easily capturing an image as intended by the user will be described. The image capturing device 100 has a reception unit in the display unit 107 that receives input for specifying a position or area by tapping or dragging on the captured image. The user can use the operation means to specify a specified appearance position, a specified movement vector, a specified end point position, and a specified start point position of a moving object.

[0055] The designated appearance position is the position where the moving object to be photographed appears. When the exposure area of ​​the designated appearance position is determined to be a moving object area, the image processing unit 104 (determination unit) performs the processing from calculation of the moving object vector onward. This makes it possible to reduce the number of times a moving object is photographed that the user does not expect.

[0056] The specified movement vector is a vector that specifies the direction or amount of movement of the moving object you want to photograph. If the degree of match between the specified movement vector and the movement vector exceeds a threshold, the system performs the calculation of the predicted movement vector and subsequent processes. This reduces the number of times you photograph a moving object that the user did not expect.

[0057] The specified end point and start point are positions that the user specifies to set the exposure time. If they are not included in the moving object area, the nearest points within the moving object area will be used as the specified end point and start point. By setting the specified end point, the user can photograph a moving object from any position within the image. By setting the specified start point, the position of the area where blur will occur can be specified.

[0058] Furthermore, the user can arbitrarily set how much of the exposure area each of the start point area and the end point area includes.

[0059] Although the present embodiment has been described with respect to a case where there is one moving object, there may be multiple moving objects. In this case, a predicted motion vector is calculated for each moving object, and exposure conditions are set. This makes the present embodiment applicable even when there are multiple moving objects captured at the same time.

[0060] The exposure conditions may be set based on the movement vector of the moving object. For example, if the movement vector is long, the movement speed is fast, so it is preferable to set the exposure time shorter. Setting it to a shorter time, especially in the end point region, will result in a clear image with less blur. Conversely, if the movement vector is short, the movement speed is slow, so it is preferable to set the exposure time longer. Setting it to a relatively long time within the range where blur does not occur, especially in the end point region, will allow the analog gain to be lowered and a low-noise image to be generated. However, in this case, the exposure time in the end point region must be set to a shorter time than the exposure time in the start point region.

[0061] <Other embodiments> The present invention can be realized by a process of reading and executing a program that realizes one or more functions of the above-described first embodiment. This program is supplied to a system or device via a network or a storage medium, and is read and executed by one or more processors in the computer of the system or device. The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. [Explanation of symbols]

[0062] 100 Imaging device 101 Imaging unit 102 Imaging optical system 103 Image sensor 104 Image processing section 105 Memory 106 Imaging system control unit 107 Display section 108 Imaging device control section

Claims

1. an imaging unit that captures an image of a moving object and generates an image; a calculation unit that calculates a motion vector of the moving object based on a latest frame and a frame before the latest frame among a plurality of frames of images captured by the imaging unit; a prediction unit that predicts, based on the motion vector, a predicted motion vector that is a vector representing the amount and direction of movement of the moving object from the latest frame to a next frame after a predetermined time has elapsed; an exposure control unit that controls an exposure time for each pixel group of the imaging unit so that an exposure time for a second region that is an area including an end point of the predicted motion vector is shorter than an exposure time for a first region that is an area including a start point of the predicted motion vector, The prediction unit determining a region in which luminance information changes among the plurality of frames as a first moving object region including the moving object in the latest frame; predicting a second moving object region including the moving object in the next frame by moving the first moving object region based on the predicted motion vector; An imaging apparatus comprising: an imaging device that determines a start point of the predicted motion vector from the first moving object region; and an imaging device that determines an end point of the predicted motion vector from the second moving object region.

2. 2. The imaging device according to claim 1, wherein the exposure control unit controls the exposure time for each pixel group of the imaging unit so that the exposure time of the region between the first region and the second region becomes gradually shorter from the first region to the second region.

3. 3. The imaging apparatus according to claim 1, further comprising an imaging system control unit that controls imaging by the imaging unit so that the entire moving object in the next frame is within the angle of view of the imaging unit.

4. a display unit that displays an image generated by the imaging unit; a receiving unit that receives user operations to specify a designated appearance position that specifies a position where the moving object will appear on the display unit, a designated movement vector that specifies a movement direction or a movement amount of the moving object, and a designated start position and a designated end position that specify a start point and an end point of the predicted movement vector; 4. The imaging device according to claim 1, further comprising:

5. a determination unit that determines whether an image of the moving object is detected on the display unit corresponding to the designated appearance position input by the reception unit; The imaging device according to claim 4 , wherein calculation of the predicted motion vector is started in accordance with a determination made by the determination unit.

6. 6. The imaging device according to claim 4, wherein the prediction unit predicts a predicted motion vector when the degree of coincidence between the designated motion vector input to the reception unit and the motion vector of the moving body calculated by the calculation unit exceeds a predetermined threshold.

7. The imaging device according to claim 1 , wherein the exposure control unit controls an exposure time based on the movement vector.

8. an imaging step of capturing an image of a moving object to generate an image; a calculation step of calculating a motion vector of the moving object based on a latest frame and a frame before the latest frame among the plurality of frame images captured in the imaging step; a prediction step of predicting, based on the motion vector, a predicted motion vector which is a vector representing the amount and direction of movement of the moving object from the latest frame to a next frame after a predetermined time has elapsed; an exposure control step of controlling an exposure time in the imaging step for each pixel group so that an exposure time of a second region, which is an area including an end point of the predicted motion vector, is shorter than an exposure time of a first region, which is an area including a start point of the predicted motion vector; In the prediction step, determining a region in which luminance information changes among the plurality of frames as a first moving object region including the moving object in the latest frame; predicting a second moving object region including the moving object in the next frame by moving the first moving object region based on the predicted motion vector; A control method for an imaging device, comprising determining a start point of the predicted motion vector from the first moving object region and determining an end point of the predicted motion vector from the second moving object region.

9. A program for causing a computer to execute the method for controlling an imaging apparatus according to claim 8.

10. A computer-readable storage medium storing the program according to claim 9.

Images