Information Processing Device, Information Processing Method, and Storage Medium

Abstract

The present invention provides an information processing device, an information processing method, and a storage medium. A control component performs tilt control and focus control. The tilt control is for tilting an imaging element with respect to a plane perpendicular to the optical axis of an imaging optical system about a rotation axis, and the focus control is for moving a focusing lens in the optical axis direction. A setting component sets an object region in a captured image obtained by an imaging unit. A determination component determines a correction amount of the position of the focusing lens based on a position corresponding to the position of the rotation axis in the image and the position of the object region in the image, and the focusing lens is moved based on the correction amount.

Description

Technical Field

[0001] The present invention relates to an information processing device, an information processing method, and a storage medium. Background Art

[0002] There is known a technique called the Scheimpflug principle, which adjusts the focal plane by rotating (performing tilt control) the imaging plane, so that the imaging plane is tilted toward a plane perpendicular to the optical axis of the imaging system for capturing an image of a subject, thereby changing the depth of field of the captured scene. According to this technique, reducing the depth of field enables an image in which parts other than the main subject are blurred to be obtained, and increasing the depth of field without reducing the aperture enables an image having a clear focus over the entire imaging area to be obtained while suppressing noise or blurring of the subject due to insufficient light quantity.

[0003] Japanese Patent Application Laid-Open No. 2008-205569 discloses an imaging device that captures a portrait in which parts other than the main subject are blurred by performing tilt control when focusing on the main subject. In addition, Japanese Patent Application Laid-Open No. 2018-056810 discloses the following technique: by adopting a configuration in which the imaging plane rotates about an arbitrary point as a rotation axis, the imaging plane rotates about a focus point specified in tilt control. Summary of the Invention

[0004] According to an embodiment of the present invention, an information processing device includes: a control component configured to perform tilt control and focus control, the tilt control being for tilting an imaging element relative to a plane perpendicular to the optical axis of an imaging optical system about a rotation axis, the focus control being for moving a focusing lens in a direction of the optical axis; a setting component configured to set an object area in a captured image obtained by an imaging unit; and a determination component configured to determine a correction amount of a position of the focusing lens based on a position corresponding to a position of the rotation axis in the image and a position of the object area in the image, wherein the control component moves the focusing lens based on the correction amount.

[0005] According to another embodiment of the present invention, an information processing method includes: performing tilt control and focus control, the tilt control being for tilting an imaging element relative to a plane perpendicular to the optical axis of an imaging optical system about a rotation axis, the focus control being for moving a focusing lens in a direction of the optical axis; setting an object area in a captured image obtained by an imaging unit; and determining a correction amount of a position of the focusing lens based on a position corresponding to a position of the rotation axis in the image and a position of the object area in the image, wherein the focusing lens is moved based on the correction amount.

[0006] A non - transitory computer - readable storage medium storing a program, which when executed by a computer causes the computer to perform the above - mentioned information - processing method.

[0007] The further features of the present invention will become clear from the following description of exemplary embodiments (with reference to the accompanying drawings). BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 is a block diagram showing an example of the functional configuration of an information - processing device according to the first embodiment;

[0009] Figure 2A and 2B are diagrams each showing an example of focus - position correction in tilt control according to the first embodiment;

[0010] Figures 3A to 3C are diagrams each showing an example of focus - position correction in tilt control according to the first embodiment;

[0011] Figure 4 is a diagram for explaining the calculation of the focus - position correction amount according to the first embodiment;

[0012] Figure 5 is a diagram showing an example of the relationship between the tilt angle and the focus - correction amount according to the first embodiment;

[0013] Figure 6 is a flowchart of a processing example of the information - processing method according to the first embodiment;

[0014] Figure 7 is a flowchart of a processing example of tilt control according to the first embodiment;

[0015] Figure 8 is a flowchart of a processing example including focus correction according to the first embodiment;

[0016] Figure 9 is a diagram showing an example of the relationship between the tilt angle and the focus - correction amount according to the second embodiment;

[0017] Figure 10 is a flowchart of a processing example including focus correction according to the second embodiment;

[0018] Figure 11 is a block diagram showing the functional configuration of a computer according to the third embodiment;

[0019] Figure 12 is a diagram showing an example of a captured image in which two subjects appear according to the third embodiment; and

[0020] Figure 13It is a flowchart of a processing example of the information processing method according to the third embodiment. Detailed implementation mode

[0021] In the technology disclosed in Japanese Patent Laid-Open No. 2008-205569, since the rotation axis for tilt control is fixed and focus adjustment is performed on the subject on the center line of the imaging element, when the subject is not on the center line, the focus will change and become blurred. On the other hand, in the technology disclosed in Japanese Patent Laid-Open No. 2018-056810, it is possible to focus on a specified focus, but a complex hardware configuration is required to drive an arbitrary point as the rotation axis.

[0022] Embodiments of the present invention can perform tilt control while maintaining focus on an arbitrary area in the imaging area using a simple hardware configuration.

[0023] Hereinafter, embodiments will be described in detail with reference to the drawings. Note that the following embodiments are not intended to limit the scope of the claimed invention. A plurality of features are described in the embodiments, but the invention is not limited to requiring all of these features, and a plurality of such features can be appropriately combined. In addition, in the drawings, the same reference numerals are given to the same or similar configurations, and redundant descriptions thereof are omitted.

[0024] [First Embodiment]

[0025] The information processing device according to the first embodiment includes an imaging element and an imaging optical system. In addition, the information processing device according to the present embodiment sets an object area in the image captured by the imaging unit, and corrects the focus position by calculating a correction amount of the focus position of the object area according to tilt control, which is performed to change the relative attitude between the imaging element and the imaging optical system.

[0026] Figure 1 An example of the functional configuration of the information processing device according to the embodiment is shown. The imaging optical system of the information processing device 100 according to the present embodiment forms a subject image by using the same processing as a general imaging optical system. Therefore, detailed descriptions of each process will be omitted. The information processing device 100 includes a zoom lens 101 that moves in the optical axis direction to change the focal length, a focus lens 102 that moves in the optical axis direction to adjust the focus position, and an aperture unit 103 for adjusting the light amount. The light passing through the imaging optical system forms a subject image as an optical image on the imaging element 106 via a band-pass filter (BPF) 104 and a color filter 105. The BPF 104 can move along the optical path of the imaging optical system.

[0027] The imaging element 106 performs photoelectric conversion on the subject image. After the analog electrical signal (imaging signal) output from the imaging element 106 undergoes gain adjustment by the AGC 107 and is converted into a digital signal by the AD converter 108, the converted signal is input to the signal processing unit 109. The signal processing unit 109 performs various image processes on the input digital signal to generate an image signal. Next, the signal processing unit 109 outputs the image signal to the monitoring device 111 connected to the information processing device 100 via wired or wireless communication through the communication unit 110. The monitoring device 111 outputs control signals such as commands to the area setting unit 112, tilt control unit 114, or focus control unit 117 in the information processing device 100 to cause each processing unit to perform the processes (described later).

[0028] The area setting unit 112 sets an object area to be focused on in the subject image on the imaging element based on an instruction from the communication unit 110. The object area is an area formed by one or more pixels, and for example, an area where a desired subject to be focused appears can be set as the object area. Details of the setting of the object area will be described later with reference to the flowchart of Figure 6 The evaluation value calculation unit 113 obtains the RGB pixel values or luminance values of the object area from the AD converter 108 or the signal processing unit 109. Next, the evaluation value calculation unit 113 calculates a focus evaluation value using autofocus (AF) control. In the present embodiment, it is assumed that the focus evaluation value is an evaluation value related to the contrast of a specific frequency. However, the present invention is not particularly limited thereto, and the focus evaluation value can be, for example, a value indicating distance information caused by a phase difference or the like.

[0029] The tilt control unit 114 indicates the target position of tilt control to the tilt drive unit 115 based on an instruction from the communication unit 110. In the present embodiment, the target position of tilt control is represented by a rotation angle with respect to the basic attitude of the imaging element 106 or the imaging optical system. Further, it is assumed that the tilt angle refers to the control amount (rotation angle) of tilt control from the basic attitude of the imaging element 106 to a given attitude. In the present embodiment, it is assumed that the basic attitude of the principal plane of the imaging optical system is parallel to the basic attitude of the imaging element 106 in the following description.

[0030] The tilt control according to the present embodiment is a control for tilting the focal plane set to be in focus by tilting the imaging surface with respect to a plane perpendicular to the optical axis for imaging the subject. In the present embodiment, tilt control is performed by rotating the imaging element 106 or the imaging optical system, so that it is possible to set the focus on the entire subject on the ground or on a plane such as a building. For example, tilt control is performed by a user operating an input device such as a button or a dial.

[0031] The tilt drive unit 115 tilts the imaging element 106 or the imaging optical system based on the target position indicated from the tilt control unit 114. Assume that the tilt drive unit 115 will tilt the imaging element 106, and this operation will be simply referred to as "tilt control" hereinafter. Additionally, in the present embodiment, it will be assumed that the tilt drive unit 115 will perform control by using the central axis at the center of the imaging element 106 as shown in FIGS. 2 or 3 (described later) as the rotation axis. However, the rotation axis is not limited to this position. Note that in this example, it will be assumed that the central axis of the imaging element is the axis connecting the midpoints of the two short sides of the imaging element. Note that in the following description, it is assumed that tilt control will be performed within the range of -90° to +90° with respect to the basic attitude.

[0032] Note that although tilt control will be described in the present embodiment by rotating the imaging element 106 as described above, tilt control can also be performed by rotating the imaging optical system. In this case, the relationship between the positions and postures of the imaging element 106 and the imaging optical system will be the same as the case where the imaging element 106 has rotated by the same rotation width around the rotation axis of the imaging optical system as the rotation center in the direction opposite to the rotation direction of the imaging optical system. Therefore, the focus correction value will also be calculated in the same manner as in this case by using the method described below.

[0033] In the case where tilt control is to be performed, the correction amount calculation unit 116 determines the correction amount (focus correction amount) of the focus correction to be performed by the focus control unit 117. The focus correction amount according to the present embodiment is the correction amount used for focus correction to maintain focus in the object area set by the area setting unit 112 based on tilt control. Although the focus correction amount will be calculated based on the control amount of tilt control herein, its detailed description will be given later with reference to Figure 4 The focus control unit 117 designates the focus position to the focus drive unit 118 based on the calculated focus correction amount or an instruction from the communication unit 110. The focus drive unit 118 controls the position of the focusing lens 102 based on the focus position indicated by the focus control unit 117. Additionally, in the case where the user has instructed to stop tilt control, the correction amount calculation unit 116 can recalculate the correction amount by setting the position corresponding to the stop instruction (the posture at the time point when the imaging element stops) as the target position.

[0034] [Tilt Control]

[0035] Hereinafter, reference will be made to Figure 2A and 2B as well as Figures 3A to 3C to describe the focus position correction corresponding to the tilt control performed by the information processing apparatus according to the present embodiment.Figure 2A This is a schematic diagram showing an example of the state before tilt control focusing on point A on the subject on the optical axis. In Figure 2A this example, the focus has been adjusted so that the focus will be at the subject distance L to point A. Additionally, since the principal plane of the optical system and the imaging element are arranged in parallel, the plane of the optical system and the imaging element will be parallel to the focal plane. Additionally, point A exists on the rotation axis of the focal plane (described later). Note that "set to focus" means photographing the subject within the depth of field determined according to the setting of the permissible circle of confusion of the imaging optical system. Although focus adjustment will be basically performed by AF in this embodiment, since it is a well-known process, its description will be omitted.

[0036] Figure 2B This is a schematic diagram showing an example of the state after tilt control around the central axis of the imaging element as the rotation axis. When tilt control is performed, the focal plane also rotates based on the Scheimpflug principle. Therefore, performing tilt control will enable focusing to be set over the entire range from the near distance to the far distance of the subject on a given plane. Note that at this time, the focal plane will rotate about a position corresponding to the tilt rotation axis (the rotation axis of the imaging element) as the axis. In this embodiment, the axis corresponding to the tilt rotation axis of the focal plane is the line provided on the focal plane at a position corresponding to the subject image on the tilt rotation axis of the imaging element. In Figure 2B this, the focal plane is rotated by tilt control so that the focal plane will coincide with the entire surface of the subject on this plane. Although the Scheimpflug principle is a well-known principle and its detailed description will be omitted, it is the following principle: when the principal plane of the optical system and the imaging surface of the imaging element intersect on a given single line, the focal plane will intersect on the same line as this line. Assuming f is the focal length, L is the subject distance, and α is the depression angle, the tilt angle b will be expressed as follows based on the Scheimpflug principle.

[0037]

[0038] Note that in the case where tilt control is to be performed in the direction opposite to the direction in the example shown in Figure 2A and 2B since the focal plane will also rotate in the opposite direction, the depth of field will become shallower. Therefore, it will be possible to make points other than point A on the subject more blurred.

[0039] Figure 3A This is a schematic diagram showing another example of the state before tilt control set to focus on point B not on the optical axis of the subject. In Figure 3AIn the example, the focus is adjusted so that the subject distance L set to focus on point B. In addition, since the principal plane of the imaging optical system and the imaging element are arranged in parallel, the plane of the imaging optical system and the imaging element are parallel to the focal plane. In addition, the rotation axis of the focal plane corresponding to the tilt control passes through Figure 2A and 2B point A, and point B falls outside the rotation axis. Since the focal plane will rotate about the position corresponding to the rotation axis of the imaging element 106 as the rotation axis when tilt control is performed in the same manner as in the example of Figure 2A and 2B , the point B of the subject that is not on the rotation axis of the focal plane will be blurred because it falls outside the focal plane.

[0040] Figure 3B is a schematic diagram showing an example of the state after tilt control is performed in the same manner as in Figure 2B . As described above, in this example, point B is out of focus, and it is preferable to perform focus correction to prevent blurring of this point B. Figure 3C is a schematic diagram of the case where such focus correction is performed by the information processing device 100 according to the present embodiment. Figure 3C The example of

[0041] shows that the state set to focus on point B is maintained by focus correction when the focal plane rotates. Figure 4 will be described later with reference to Figure 4 to describe an example of calculating the focus correction amount for correcting the focus position according to the tilt control. In

[0042] β = (tanα2 – tanα1) × k...(2)

[0043] In the above equation, the imaging element 106 is controlled by tilt control to rotate from the tilt angle α1 to the tilt angle α2, and k is the distance from the rotation axis on the imaging element 106 to the center pixel of the subject.

[0044] Figure 5This is a graph showing the relationship between the tilt angle and the focus correction amount β. In this example, the graph is not linear, and even when tilt control is performed by the same amount, the change amount of the focus correction amount varies according to the control amount for the basic posture. Since the tilt control of the imaging element 106 is performed within the range of -90° to +90° with respect to the basic posture, the focus correction amount corresponding to the control amount will increase as the absolute value of the control amount with respect to the basic posture increases.

[0045] Note that the calculation method of the focus correction amount is not particularly limited to Equation (2). For example, the correction amount calculation unit 116 can calculate a simple correction amount by dividing the focus correction amount β of Equation (2) by the sensitivity of the focusing lens, and can calculate a more accurate correction amount by solving a higher-order equation or polynomial corresponding to the sensitivity.

[0046] Hereinafter, reference will be made to Figure 6 Describe the processing to be performed by the information processing apparatus 100 according to the present embodiment. Figure 6 This is a flowchart showing an example of the main processing performed by the information processing apparatus 100.

[0047] In step S601, the area setting unit 112 sets an object area. In the present embodiment, the area setting unit 112 sets an arbitrary area designated by the user as the object area. In addition, the area setting unit 112 can automatically set the position detected by the function for detecting a specific object as the object area. For example, the area setting unit 112 can use the AF function to set the focus on the detected subject, and set the area indicating the subject as the object area.

[0048] In step S602, the focus driving unit 118 focuses on the object area. The focus driving unit 118 can automatically perform focusing by, for example, the AF function. In addition, for example, the focus driving unit 118 can also use the manual focusing (MF) function to perform focusing according to the manual input of the user who is viewing the captured image. In this case, the focus driving unit 118 can display auxiliary information for the user to view by magnifying the display of the object area, and present the degree of focus on the object area, and so on.

[0049] In step S603, the tilt control unit 114 determines whether the user has issued a tilt control instruction. If the user has issued a tilt control instruction, the process proceeds to step S604. Otherwise, it is confirmed again whether the user has issued a tilt control instruction.

[0050] In step S604, the correction amount calculation unit 116 calculates the distance from the rotation axis in the captured object image to the object area. In the present embodiment, the correction amount calculation unit 116 calculates the distance from the rotation axis to the central pixel of the object area. In step S605, the tilt control unit 114 obtains the target position of the tilt control set in the instruction confirmed in step S603. Only one point can be specified as the target position of the tilt control. For example, a tolerance range of -1° to +1° can be set, and a point falling within this range can be determined as the target position. In step S606, the correction amount calculation unit 116 determines whether the value of the distance from the rotation axis on the imaging element 106 to the object (the value of the distance from the position corresponding to the position of the rotation axis in the image to the position of the object area, or the value of the distance from the position corresponding to the position corresponding to the position of the rotation axis in the image to the position of the object area) is 0. If it is determined that this value is 0, when it is determined that no focus control is required because the focus on the object area will not shift even if tilt control is performed, the process proceeds to step S607. Otherwise, when it is determined that focus control is required, the process proceeds to step S608. Tilt control will be performed in step S607, and tilt control and focus control will be performed in step S608. After the processing in step S607 or step S608, the process ends.

[0051] Hereinafter, reference will be made separately to Figure 7 and 8 describe the processing of step S607 and the processing of step S608. Figure 7 is a flowchart showing an example of the detailed processing of the tilt control to be performed in step S607.

[0052] In step S701, the tilt drive unit 115 starts the tilt control toward the target position of the tilt control obtained in step S605. In step S702, the tilt drive unit 115 determines whether the tilt angle has reached the target position at this time point. If the tilt angle reaches the target position, the process ends. Otherwise, the process proceeds to step S703.

[0053] In step S703, the tilt control unit 114 determines whether the user has issued a stop instruction during the tilt drive operation. If a stop instruction has been issued, the process proceeds to step S704. Otherwise, the process proceeds to step S702. In step S704, the tilt drive unit 115 stops the tilt drive operation at the position corresponding to this stop instruction, and the process ends.

[0054] Figure 8This is a flowchart showing an example of the detailed processing of tilt control and focus control to be performed in step S608. In step S801, the tilt control unit 114 obtains the current tilt angle as the tilt start position. In step S802, the correction amount calculation unit 116 calculates the correction amount to be performed according to the tilt control and the focus target position corresponding to this correction amount. In the present embodiment, the correction amount is calculated by the above equation (2).

[0055] In step S803, the correction amount calculation unit 116 sets the focus drive speed for performing focus correction. In the present embodiment, the focus drive speed refers to the speed for controlling the position of the focusing lens to perform focus correction. The correction amount calculation unit 116 can set the focus drive speed according to the control speed of the tilt control drive operation determined in advance. In the case where the drive of the tilt control and the drive of the focus control are not synchronized, the effect of maintaining focus on the focus area during the tilt drive operation will be reduced. From this viewpoint, the correction amount calculation unit 116 according to the present embodiment will set the focus drive speed such that the focus drive time will be equal to the total drive time of the tilt control. That is, the correction amount calculation unit 116 will set the focus drive speed such that the correction of the focus position will be completed simultaneously with the completion of the tilt control. For example, the drive time of the tilt control is calculated by dividing the tilt drive amount (the difference between the tilt start position and the tilt target position) by the tilt drive speed (° / sec).

[0056] In the present embodiment, the tilt drive unit 115 performs tilt control at a predetermined speed. Since the focus correction amount corresponding to the tilt control amount increases as the absolute value of the tilt control amount with respect to the basic attitude increases, the correction amount calculation unit 116 can set the focus drive speed to increase as the absolute value of the tilt angle increases within the range of 0° to 90°. In addition, the correction amount calculation unit can set the focus drive speed during the tilt drive operation according to the value of the absolute value of the tilt angle before performing the tilt control.

[0057] In step S804, the tilt drive unit 115 starts the tilt drive toward the target position obtained in step S605, and the focus drive unit 118 starts controlling the position of the focusing lens toward the focus target position calculated in step S802. In step S805, the tilt drive unit 115 and the focus drive unit 118 determine whether the tilt angle and the focus position have reached their respective target positions. In the case where both the tilt angle and the focus position have reached their respective target positions, the process ends. Otherwise, the process proceeds to step S806.

[0058] In step S806, the tilt control unit 114 determines whether the user has issued a stop instruction during the tilt drive operation. If a stop instruction has been issued, the process proceeds to step S807. Otherwise, the process returns to step S805. In step S807, the tilt drive unit 115 stops the tilt drive operation at the position corresponding to the stop instruction.

[0059] In step S808, the focus control unit 117 recalculates the focus correction amount and the focus target position corresponding to the focus correction amount by setting the position corresponding to the tilt drive stop instruction as the target position. In step S809, the focus drive unit 118 controls the position of the focus lens according to the recalculated focus target position.

[0060] According to this processing, it will be possible to calculate the correction amount corresponding to the position of the target position based on the tilt control, and to correct the focus position by using this correction amount. Therefore, it will be possible to use a simple hardware configuration with a fixed tilt control rotation axis to change the depth of field by performing tilt control while maintaining focus on any area in the imaging region. In addition, by setting the drive time of the tilt control to be equal to the drive time of the focus control, it will be possible to maintain focus on the object area during one tilt drive operation.

[0061] [Second Embodiment]

[0062] The information processing apparatus according to the first embodiment sets the focus control such that the tilt control and the focus control will have equal drive times. That is, it is set such that one focus control operation will be performed for one tilt control operation. On the other hand, the information processing apparatus according to the present embodiment will repeatedly calculate the focus correction amount and correct the focus position a plurality of times during one tilt control operation. By performing the correction a plurality of times, it will be possible to control the focus position in the optimal state each time it is repeated. Note that the configuration and processing of the information processing apparatus according to the present embodiment are the same as those of the first embodiment Figure 1 and 6 except for the processing shown in step S608 ( Figure 8 processing), and redundant description thereof will be omitted.

[0063] Hereinafter, reference will be made to Figure 9 and 10 to describe the processing to be performed by the information processing apparatus according to the present embodiment. In the present embodiment, a predetermined tilt angle control amount S will be set as one step, and the focus position correction will be performed by recalculating the focus correction amount for each step. Figure 9 is by Figure 5An auxiliary line is drawn on the graph showing the relationship between the tilt angle and the focus correction amount β so that the graph obtained by referring to the focus correction amount for each tilt angle control amount S can be easily referenced.

[0064] In Figure 9 the example, reference numeral P1 indicates the start position of the tilt control, and reference numeral P2 indicates each target position of the tilt control, and the focus correction amount for each control amount S between the start position and each target position is shown. In the present embodiment, the focus correction amount as shown in Figure 9 is calculated for each control amount S, and the focus position is corrected each time. Note that although it is assumed herein that the focus drive speed during one focus correction operation will be constant, the focus drive speed can be finely adjusted as needed. Additionally, although in the present embodiment the respective focus correction amounts will be calculated according to Equation (2), the following format can also be adopted: storing table data or coefficients for obtaining the focus correction amounts, etc., and performing correction by referring to these table data or coefficients. Additionally, although the control amount S is set such that the tilt angle will reach the target position exactly from the start position by the drive operation for each step in the present embodiment, the present invention is not particularly limited thereto. For example, it can be configured such that: the focus correction amount is calculated for each control amount S until the control to be performed immediately before reaching the target position, and the focus correction amount for only the final control operation is calculated by setting the control amount from the tilt angle at this time to the target position.

[0065] Figure 10 is a flowchart showing an example of the detailed processing of the tilt control and the focus control to be performed by the information processing apparatus according to the present embodiment in step S608. In step S1001, the tilt control unit 114 obtains the current tilt angle as the tilt start position. In step S1002, the tilt control unit 114 sets the control amount S for one step of the tilt control. The user sets the value of the control amount S to an expected value. The focus maintenance effect during the tilt control increases as the value of the control amount decreases, and the time required for the tilt control decreases as the value of the control amount increases.

[0066] In step S1003, the correction amount calculation unit 116 calculates the focus correction amount corresponding to one step. In this example, the correction amount calculation unit 116 calculates the focus correction amount based on the tilt angle at this time point and the control amount S for one step. In step S1004, the tilt drive unit 115 and the focus drive unit 118 perform the tilt drive operation and the focus drive operation for one step. After both the tilt drive operation and the focus drive operation for one step are completed, the process proceeds to step S1005.

[0067] In step S1005, the tilt control unit 114 determines whether the tilt angle has reached the target position set in step S608. If it is determined that the tilt angle has reached the target position, the process ends. Otherwise, the process proceeds to step S1006. In step S1006, the tilt control unit 114 determines whether the user has issued a stop instruction during the tilt drive operation. If a stop instruction has been issued, the tilt drive unit 115 stops the tilt drive operation at the position corresponding to the stop instruction, and the process ends. Otherwise, it is determined that a tilt drive operation and a focus drive operation for the next step length are to be performed, and the process returns to step S1003.

[0068] According to this process, two or more focus correction operations can be performed for one tilt control operation. Therefore, by sequentially controlling to set the focus position in the optimal state each time the focus correction operation is repeated, the effect of maintaining focus on the object area can be improved.

[0069] [Third Embodiment]

[0070] The information processing apparatus according to the present embodiment automatically adjusts the focus position by performing tilt control and focus control so as to be focused on two object areas in the captured image. The information processing apparatus 100 can automatically adjust the focus position so as to be focused on two subjects at different distances from the information processing apparatus 100. Since the information processing apparatus 100 according to the present embodiment has the same configuration as that shown in the first embodiment and can perform the same processing, redundant descriptions will be omitted. Figure 1 The configuration is the same as that shown and the same processing can be performed, so redundant descriptions will be omitted.

[0071] The following will refer to Figure 12 and 13 to describe the processing performed by the information processing apparatus 100 according to the present embodiment. Figure 12 An example of an image captured by the information processing apparatus according to the present embodiment is shown. In this example, the subject 1201 is positioned at a distance closer to the information processing apparatus 100 than the subject 1202. When there is a difference in the distances to the two subjects, the depth of field may become insufficient and it may not be possible to be focused on the two subjects by performing a normal imaging operation. Therefore, the tilt control unit 114 according to the present embodiment will determine the control amount of the tilt control so as to be focused on both the subjects 1201 and 1202 by correcting the focus position. That is, the information processing apparatus according to the present embodiment will determine the tilt control amount and the focus correction amount such that the two subjects can be set on the focal plane by performing tilt control and focus control in the same manner as in the first embodiment.

[0072] The region setting unit 112 according to the present embodiment sets two object regions in the subject image on the imaging element in the same manner as in the first embodiment. Herein, the focus driving unit 118 may first be set to focus on one of the two object regions based on the focus evaluation results calculated by the evaluation value calculation unit 113 from each object region. The focus driving unit 118 may be set, for example, to focus on the object region with a higher priority level set for each object region, or may be set to focus on the object region selected based on the focus evaluation value before focusing, and the method for selecting the object region to be set to focus on is not particularly limited. The priority level set for each object region can be set by an input such as user selection. The priority level can also be set according to the type of the detected object (for example, set to give priority to focusing on a specific subject, etc.). The priority level can also be set according to a predetermined condition such as a condition for setting a higher priority level for the object region where a subject closer to the information processing device 100 appears. For the convenience of the following description, it is assumed that Figure 12 subjects 1201 and 1202 will be used as the subjects of the two object regions to be set by the region setting unit 112, and it is assumed that focusing will be preferentially set on subject 1201.

[0073] While maintaining focus on subject 1201, the tilt control unit 114 determines the target position of the tilt control that will cause focusing on subject 1202 by performing tilt control and focus control in the same manner as in the first embodiment. The tilt control unit 114 can detect, for example, the target position of the tilt control for setting focus on subject 1202 based on the focus evaluation value of the region on subject 1202 during tilt control (and accompanied by focus correction for maintaining focus on subject 1201). The tilt control unit 114 can also calculate a common tilt control position, where the focus lens position in the case of performing tilt control while maintaining focus on subject 1201 will be the same as the focus lens position in the case of performing tilt control while first setting focus on subject 1202 and then maintaining focus. In this case, the above common tilt control position can be calculated, for example, by using equation (2).

[0074] Note that when the distance k from the rotation axis in the subject image captured by the imaging sensor to the subject 1201 is zero, the tilt control will not change the focusing state on the subject 1201. Therefore, when k is zero, the correction amount calculation unit 116 can calculate the focus correction amount for any position of the subject 1202 to be tilt-controlled in the same manner as in the first embodiment without considering the focus on the subject 1201. Additionally, since it can be considered that when k approaches zero, the change in the focusing state on the subject 1201 caused by the tilt control will be minimized to the extent that it can be ignored, a predetermined range can be set to be approximately zero, and if the value of k falls within this range (for example, a range from -0.5 or greater to +0.5 or less), the value of k can be determined to be zero.

[0075] Figure 13 FIG. 4 is a flowchart showing an example of the processing performed by the information processing apparatus according to the present embodiment. In step S1301, the region setting unit 112 sets two object regions in the subject image on the imaging element. In step S1302, the region setting unit 112 sets one of the two object regions as the region to be preferentially focused on (hereinafter referred to as the focus region). In step S1303, the region setting unit 112 sets the region that was not set as the focus region in step S1302 as the region to be focused on by tilt control (hereinafter referred to as the tilt region).

[0076] In step S1304, the tilt control unit 114 adjusts the focus based on the focus evaluation value of the focus region. Here, the focus region is set to be focused on by the AF function.

[0077] In step S1305, the correction amount calculation unit 116 calculates the distance k from the rotation axis in the subject image captured by the imaging element to the focus region. When it is determined that k is 0, the process proceeds to step S1307. Otherwise, the process proceeds to step S1308.

[0078] In step S1307, the correction amount calculation unit 116 calculates the focus correction amount for the position to be tilt-controlled. The tilt control position can be determined based on, for example, an image or a user operation. After step S1307, the process proceeds to step S1309.

[0079] In step S1308, while maintaining the focus on the focus region, the tilt control unit 114 determines the tilt control target position by performing the same tilt control and focus control as in the first embodiment so that the tilt region is set to be focused on. After step S1308, the process proceeds to step S1309.

[0080] In step S1309, the tilt drive unit 115 and the focus drive unit 118 will perform drive operations based on the tilt control position determined in step S1307 or step S1308 and the focus lens position determined according to the tilt control position. Subsequently, the process ends.

[0081] According to this process, it is possible to simultaneously set the focus on two focused target subjects by performing tilt control and focus control. For example, by using the AF function, first set the focus on one subject, and then perform tilt control and focus control so that while maintaining the focus on the first subject, the focus position is set on the other subject. Therefore, it is possible to automatically set the focus on these two subjects at different distances from the imaging device.

[0082] [Fourth Embodiment]

[0083] In the above embodiments, for example, Figure 1 each processing unit shown, etc. is implemented by dedicated hardware. However, some or all of the processing units included in the information processing device 100 can be implemented by a computer. In this embodiment, at least a part of the processing according to the above respective embodiments is executed by a computer.

[0084] Figure 11 is a block diagram showing the basic configuration of a computer. In Figure 11 it, the processor 1110 is, for example, a CPU and controls the overall operation of the computer. The memory 1120 is, for example, a RAM and temporarily stores programs and data, etc. The computer-readable storage medium 1130 is, for example, a hard disk or a CD-ROM, etc., and stores programs and data, etc. for long-term storage. In this embodiment, each program stored in the storage medium 1130 and used to implement the functions of the corresponding unit is read out to the memory 1120. The functions of each unit are realized by causing the processor 1110 to operate according to the program on the memory 1120.

[0085] In Figure 11 it, the input interface 1140 is an interface for obtaining information from an external device. In addition, the output interface 1150 is an interface for outputting information to an external device. The bus 1160 connects the above units and enables data exchange.

[0086] Other Embodiments

[0087] Embodiments of the present invention can also be implemented by the following method, that is, by providing software (program) that executes the functions of the above embodiments to a system or device through a network or various storage media, and the computer or central processing unit (CPU), microprocessing unit (MPU) of the system or device reads and executes the program.

[0088] Although the present invention has been described with reference to exemplary embodiments, it should be understood that the present invention is not limited to the disclosed exemplary embodiments. The scope of the following claims should be given the broadest interpretation to cover all such modifications and equivalent structures and functions.

Claims

1. An information processing apparatus, comprising: A control unit for performing tilt control and focus control, the tilt control for tilting an imaging element about a rotation axis relative to a plane perpendicular to the optical axis of an imaging optical system, and the focus control for moving a focusing lens in the direction of the optical axis; A setting unit for setting an object area in a captured image obtained by an imaging unit; And A determination unit for determining a correction amount of the position of the focusing lens based on a position corresponding to the position of the rotation axis in the image and a position of the object area in the image, wherein the control unit moves the focusing lens based on the correction amount, and wherein, when a distance between the position of the rotation axis in the image and the position of the object area in the image is equal to zero, the control unit is configured to perform the tilt control and not perform the focus control, and when the distance between the position of the rotation axis in the image and the position of the object area in the image is not equal to zero, the control unit is configured to perform the tilt control and the focus control.

2. The information processing apparatus according to claim 1, wherein The determination unit determines the correction amount such that the correction amount of the position of the focusing lens increases as a distance from the rotation axis of the imaging element to a position corresponding to the object area increases.

3. The information processing apparatus according to claim 1, wherein The determination unit further determines the correction amount of the position of the focusing lens based on a rotation angle of rotation performed by the tilt control.

4. The information processing apparatus according to claim 3, wherein The determination unit determines the rotation angle of rotation based on a current rotation angle of one of the imaging element and the imaging optical system and a target rotation angle of the tilt control.

5. The information processing apparatus according to claim 3, wherein The determination unit determines the correction amount such that the correction amount of the position of the focusing lens increases as a rotation range of the rotation increases.

6. The information processing apparatus according to claim 1, wherein The control unit controls the position of the focusing lens at a speed corresponding to a control speed of rotation performed by the tilt control.

7. The information processing apparatus according to claim 6, wherein The control unit moves the focusing lens to complete the focus control simultaneously with the completion of the tilt control.

8. The information processing apparatus according to claim 6, wherein The control unit controls the position of the focusing lens at a speed corresponding to a rotation angle of one of the imaging element and the imaging optical system during the tilt control.

9. The information processing apparatus according to claim 8, wherein The control unit increases the speed of controlling the position of the focusing lens as an absolute value of a rotation angle of one of the imaging element and the imaging optical system during the tilt control increases within a range of 0° to 90°.

10. The information processing apparatus according to claim 1, wherein When the tilt control is to be stopped, the determination unit determines the correction amount based on a rotation angle of one of the imaging element and the imaging optical system at a time point at which the tilt control is to be stopped.

11. The information processing apparatus according to claim 1, wherein The determination unit determines no less than two correction amounts for an instruction for performing one tilt control operation, and the control unit adjusts the position of the focusing lens multiple times for the instruction for performing the one tilt control operation based on the no less than two correction amounts.

12. The information processing apparatus according to claim 11, wherein During the one tilt control operation, the control unit sequentially adjusts the position of the focusing lens based on each adjustment of the position of the focusing lens performed multiple times.

13. The information processing apparatus according to claim 1, wherein The object area is one of the area designated by the user and the area automatically detected as a detection object in the captured image.

14. The information processing apparatus according to claim 1, wherein The determination component determines a correction amount of the position of the focusing lens based on the shift of the focus on the object area performed according to the tilt control.

15. The information processing apparatus according to claim 1, further comprising a determination component configured to determine a control value for the tilt control such that focusing is achieved on both a first object area and a second object area by correction of the position of the focusing lens.

16. The information processing apparatus according to claim 15, further comprising: An adjustment component for adjusting the position of the focusing lens before performing the tilt control so as to be set in focus on the first object area, characterized in that the tilt control is performed while maintaining the state set in focus on the first object area.

17. The information processing apparatus according to claim 16, wherein The setting component further sets a priority level for each object area, and the priority level set for the first object area is higher than the priority level set for the second object area.

18. An information processing method, comprising: Performing tilt control and focus control, the tilt control for tilting the imaging element about a rotation axis with respect to a plane perpendicular to the optical axis of the imaging optical system, the focus control for moving the focusing lens in the direction of the optical axis; Setting an object area in the captured image obtained by the imaging unit; and Determining a correction amount of the position of the focusing lens based on the position corresponding to the position of the rotation axis in the image and the position of the object area in the image, characterized in that the focusing lens is moved based on the correction amount, when the distance between the position of the rotation axis in the image and the position of the object area in the image is equal to zero, the tilt control is performed and the focus control is not performed, and when the distance between the position of the rotation axis in the image and the position of the object area in the image is not equal to zero, the tilt control and the focus control are performed.

19. A non-transitory computer-readable storage medium storing a program, the program, when executed by a computer, causing the computer to perform the information processing method according to claim 18.

20. A computer program product, comprising a program which, when executed by a computer, causes the computer to perform the information processing method according to claim 18.

Images