Control apparatus, lens apparatus, image pickup apparatus, control method, and storage medium

Abstract

To provide a control device capable of easily obtaining an excellent image.SOLUTION: The control device is a control device used in a camera system including a lens device provided with an imaging optical system including a focus lens group, and an imaging device provided with an imaging element that acquires an image from an optical image formed by the imaging optical system, the control device including: a first acquisition unit that acquires a plurality of pieces of distortion aberration information used when distortion aberration correction processing by the imaging optical system is performed on the image; and a control device that performs, based on distance map information to a subject for each of a plurality of areas of the image, the image pickup apparatus includes a second acquisition unit configured to acquire ratio information about a ratio of an area of an object to a total area of an image, and a correction unit configured to perform correction processing using distortion information corresponding to the ratio information.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present invention relates to a control device, a lens device, an imaging device, a control method, and a program. [Background technology]

[0002] In an imaging optical system, when the focus lens group is moved to adjust the distance to a subject to be focused, fluctuations in the angle of view (focus breathing) may occur depending on the position of the focus lens group. Patent Document 1 discloses a configuration that performs breathing correction to convert the size of an image based on the imaging magnification at each position of the focus lens group.

[0003] Furthermore, when the focus lens group is moved, the distortion occurring in the imaging optical system fluctuates. Patent Document 2 discloses a configuration in which distortion is corrected for each divided area by using an image distribution (distance map information) of distance information to the subject. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP2007-142965Publication [Patent Document 2] Japanese Patent Application Laid-Open No. 2013-240022 Summary of the Invention [Problem to be solved by the invention]

[0005] Here, we will explain the problem "distortion variation during focusing," which is the problem that the present invention aims to solve. FIG. 9 shows an input image 91 obtained by capturing a distant subject 92 and a close-up subject 93, and the distortion aberration of each optical image (94, 95). The thick frame 96 indicates the enlarged range after breathing correction. Typically, the distance between the various subjects captured in a single frame of a captured video and the imaging device is not constant, and in-focus subjects and out-of-focus subjects exist within the same frame. The correction values ​​for the distortion aberration of the optical images of the focused and out-of-focus subjects are different, and correcting the distortion aberration of the optical image of the out-of-focus subject using the correction value for the distortion aberration of the optical image of the focused subject will result in some distortion remaining uncorrected.

[0006] In particular, when the difference in distortion is large, moving the focus lens group to adjust the subject distance will cause the amount of distortion remaining to fluctuate, resulting in focus distortion fluctuations, which cause the amount of distortion on the screen to fluctuate during video shooting. Figure 10 is a schematic diagram of focus distortion fluctuations. Figures 10(A) and 10(B) show distortion when focusing on a close-up subject and a distant subject, respectively. Focus distortion fluctuations are difficult to correct using breathing correction alone, and lead to a decrease in video quality.

[0007] Such distortion fluctuations during focusing can be suppressed by correcting not only the distortion aberration of the optical image of a focused subject but also the distortion aberration of the optical image of an out-of-focus subject, as in the configuration of Patent Document 2. However, Patent Document 2 does not assume processing of moving images, and the process of correcting distortion aberration corresponding to the subject distance for each area is complex and involves many steps, which can easily cause problems such as processing delays, making it unsuitable for correcting distortion fluctuations during focusing.

[0008] An object of the present invention is to provide a control device that can easily obtain a good image. [Means for solving the problem]

[0009] A control device according to one aspect of the present invention is a control device used in a camera system including a lens device having an imaging optical system including a focus lens group, and an imaging device having an image sensor that acquires an image from an optical image formed by the imaging optical system, and is characterized by having: a first acquisition means that acquires multiple pieces of distortion aberration information used when performing a correction process for distortion aberration caused by the imaging optical system on the image; a second acquisition means that acquires proportion information regarding the proportion of the area of ​​the subject to the total area of ​​the image based on distance map information to the subject for each of multiple areas of the image; and a correction means that performs a correction process using the distortion aberration information according to the proportion information. [Effects of the Invention]

[0010] According to the present invention, it is possible to provide a control device that can easily obtain a good image. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a schematic diagram of a camera system according to an embodiment of the present invention. [Figure 2] FIG. 10 is a diagram illustrating an example of distortion aberration information. [Figure 3] FIG. 10 is a diagram illustrating an example of magnification ratio information. [Figure 4] FIG. 10 is a diagram illustrating an example of subject area information. [Figure 5] FIG. 10 is a schematic diagram of correction of distortion fluctuation during focusing. [Figure 6] 10A and 10B are diagrams illustrating an example of a result of performing correction of distortion fluctuation during focusing. [Figure 7] 1 is a flowchart showing a correction process according to the first embodiment. [Figure 8] 10 is a flowchart showing a correction process according to a second embodiment. [Figure 9] 1A and 1B are schematic diagrams of a photographed scene, an input image, and distortion aberration (conventional example). [Figure 10] FIG. 10 is a schematic diagram of distortion fluctuation during focusing (conventional example). DETAILED DESCRIPTION OF THE INVENTION

[0012] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same reference numerals are used to designate the same components, and redundant explanations will be omitted.

[0013] 1 is a schematic diagram of a camera system according to an embodiment of the present invention. The camera system includes a lens device 100 and a camera (imaging device) 200. Examples of camera systems include digital video cameras, digital still cameras, broadcast cameras, and surveillance cameras. The lens device 100 may be configured to be detachable from the camera 200, or may be configured integrally with the camera 200.

[0014] The lens device 100 includes an imaging optical system 11, a focus position detection unit 13, a distortion correction information storage unit 14, and a breathing correction information storage unit 15. The imaging optical system 11 includes a focus lens group 12. As the focus lens group 12 moves, the in-focus subject distance changes within a driving range from close to infinity. A plurality of focus lens groups 12 may be provided, for example, as in a floating focus. The focus position detection unit 13 detects the in-focus position of the focus lens group 12. The distortion correction information storage unit 14 stores a plurality of pieces of distortion information that differ for each image height and are used when performing distortion correction processing by the imaging optical system 11 on an input image obtained by imaging. The distortion information is transmitted to the camera 200. The breathing correction information storage unit 15 stores magnification ratio information for enlarging the input image. The magnification ratio information is transmitted to the camera 200. This makes it possible to correct distortion and focus breathing of various lenses, including those with different types and manufacturing errors.

[0015] It is preferable that the magnification ratio information corresponds to the distortion aberration information. Specifically, it is preferable that the magnification ratio information and the distortion aberration information are set so that a corrected image is obtained by performing corresponding distortion aberration correction and enlargement processing together. Using appropriate magnification ratio information according to the distortion aberration information is preferable because it suppresses breathing correction residues.

[0016] It is also preferable to have two or more types of distortion information for each position of the focus lens group 12. This is preferable because it allows the user to select distortion correction values ​​for shooting conditions in which distortion variation during focusing is noticeable and those in which it is not. For example, if the distance to the subject being photographed is uniform, turning off distortion variation correction during focusing makes it possible to apply distortion correction that is optimal for the subject distance at which focus is achieved.

[0017] It is also preferable that the distortion information be switched to a different type when shooting a moving image, in which distortion fluctuations during focusing are noticeable, and when shooting a still image, in which distortion fluctuations during focusing are not as noticeable as during moving image shooting.

[0018] Furthermore, it is preferable that the distortion information includes information used when performing distortion correction processing on an optical image of an out-of-focus subject. The subject surface on which focus distortion variation correction is desired is not necessarily in focus, and it is preferable to perform distortion correction based on the information used when performing distortion correction processing on an optical image of an out-of-focus subject, because this can often minimize uncorrected distortion.

[0019] The camera 200 includes an image sensor 21 and a control unit (control device) 22. The image sensor 21 converts an optical image formed by the imaging optical system 11 into a digital signal. The control unit 22 includes a correction unit 23, a distance map information calculation unit 24, and an object area information calculation unit (second acquisition unit) 25. The correction unit 23 functions as a first acquisition unit that acquires multiple pieces of distortion information from the distortion correction information storage unit 14 and acquires magnification information from the breathing correction information storage unit 15. The correction unit 23 also performs distortion correction processing on the input image using distortion information corresponding to the object area information calculated by the object area information calculation unit 25 (described later). The correction unit 23 also performs breathing correction. The correction unit 23 may be configured as separate units that perform distortion correction processing and breathing correction. The distance map information calculation unit 24 acquires distance map information using defocus information acquired from the camera 200. The method of acquiring the distance map information can be replaced with any known method (for example, a TOF method, etc.) The subject area information calculation means 25 acquires ratio information (subject area information) regarding the ratio of the area of ​​the subject to the total area of ​​the input image based on the distance map information.

[0020] In this embodiment, the control unit 22 is provided in the camera 200, but it may be provided in the lens device 100, or may be configured as a control device separate from the camera system.

[0021] Furthermore, it is preferable that the distance map information is updated for each frame during video shooting, and that the subject area information is also updated. Because the subject shown during video shooting changes constantly, updating the subject area information for each frame makes it possible to apply distortion information appropriate for that frame.

[0022] Furthermore, it is preferable that correction means 23 determine the shape of distortion from the input image, and if the distortion of the input image is pincushion-shaped, change the magnification information in accordance with the magnification required to correct the pincushion-shaped distortion, thereby making it possible to appropriately correct distortion even in the case of pincushion-shaped distortion.

[0023] 2 is a diagram showing an example of distortion information stored in the distortion correction information storage unit 14, showing distortion information at an arbitrary zoom position and subject distance. The vertical axis represents image height, and the horizontal axis represents distortion information. A solid line 26 represents an example of distortion information for negative distortion (barrel distortion), and a dashed line 27 represents an example of distortion information for positive distortion (pincushion distortion). Based on the distortion information, the correction unit 23 performs distortion correction to reduce distortion.

[0024] 3 is a diagram showing an example of magnification ratio information stored in the breathing correction information storage means 15, showing magnification ratio information at an arbitrary zoom position. The vertical axis represents the magnification ratio information, and the horizontal axis represents the position of the focus lens group. The correction means 23 performs breathing correction based on the magnification ratio information to reduce breathing.

[0025] FIG. 4 shows an example of subject area information. FIGS. 4(A) and 4(B) show cases where a distant subject and a close-up subject occupy a large portion of the screen, respectively. Subject area information 42 and 44 are acquired for captured images 41 and 43. The subject area information 42 and 44 is information in the form of a matrix showing the area occupied by each subject distance on the screen. The subject distance can be replaced by other means as long as it can identify each subject shown on the screen, and for example, it may be a matrix of subject tags obtained by subject recognition and information showing the proportion.

[0026] FIG. 5 is a schematic diagram of correction of distortion variation during focusing. To suppress distortion variation during focusing, a distortion correction value is set so that distortion at a specified subject distance is corrected across the entire focus range. For example, as shown in FIG. 5, regardless of the position of the focus lens group 12 within its driving range, by correcting distortion of an infinitely far subject, it is possible to correct distortion of a distant subject, which occupies the majority of the area of ​​the output image, regardless of whether the subject is in focus or out of focus. Hereinafter, this correction will be referred to as correction of distortion variation during focusing.

[0027] Correction of distortion variation during focus is performed on the object plane for which residual distortion aberration correction is desired to be minimized. For example, the most noticeable distortion variation during focus is the distortion variation of the object occupying the largest area on the screen. Therefore, it is preferable to use distortion aberration information to correct this. On the other hand, although there is still some residual correction remaining on object planes other than the corrected object plane, distortion variation during focus is less noticeable on object planes occupying a small area on the screen, making it less likely to lead to a decrease in video quality. Also, depending on the object in a video shot with a wide-angle lens, distortion variation during focus may be more noticeable on a patterned object such as a building that occupies the second-largest area on the screen than on a distant object such as the sky that occupies the most of the screen. Therefore, distortion aberration information for the object plane to be corrected can be applied from object area information. As described above, by correcting the entire screen, including other objects, using distortion aberration information corresponding to the distortion aberration of the object plane that is most effective, distortion variation during focus can be corrected effectively and easily.

[0028] FIG. 6 shows an example of the results of performing distortion variation correction during focusing. Reference numerals 101 and 102 denote distortion information for an infinitely distant object and a close-up object, respectively, when the focus lens group 12 is positioned so as to focus on the close-up object. Reference numerals 103 and 104 denote correction residues when distortion correction (A) is performed on the distortion for the infinitely distant object and the close-up object using the distortion information 102 to correct the distortion for the close-up object, which is the in-focus plane. Reference numerals 105 and 106 denote correction residues when distortion correction (B) is performed on the distortion for the infinitely distant object and the close-up object using the distortion information 101 to correct the distortion for the infinitely distant object, which is the out-of-focus plane. Distortion correction (B) corresponds to correction of distortion variation during focusing, and by correcting distortion of the optical image of an out-of-focus object across the entire focus range, it becomes possible to accurately correct image point movement after focus breathing has been corrected.

[0029] As described above, according to the configuration of this embodiment, it is possible to easily obtain a good image.

[0030] It is preferable to perform distortion correction using distortion information so as to correct the distortion of the object that occupies the largest area of ​​the screen among the object area information. Since distortion fluctuation during focusing tends to be large in lenses with short focal lengths such as wide-angle lenses, objects at the same distance, such as landscapes, often occupy the largest area of ​​the screen. Therefore, it is preferable to correct the distortion corresponding to the object surface that occupies the largest area of ​​the screen across the entire screen. [Example]

[0031] FIG. 7 is a flowchart showing the correction process of this embodiment. This flow starts when the camera 200 is turned on. In step S71, the correction unit 23 acquires distortion information from the lens device 100. In step S72, the control unit 22 causes the camera 200 to acquire an image and acquires shooting information. In step S73, the correction unit 23 determines whether the video mode setting is ON. If the correction unit 23 determines that the video mode setting is ON, it executes the process of step S74. If the correction unit 23 determines that the video mode setting is not ON, it executes the process of step S77. In step S74, the distance map information calculation unit 24 acquires distance map information. In step S75, the subject area information calculation unit 25 acquires subject area information using the distance map information. In step S76, the correction unit 23 selects distortion correction data to be used when performing the process of correcting distortion of the optical image of the subject that occupies the largest proportion of the screen. In step S77, the correction means 23 selects distortion correction data for correcting distortion of the optical image on the focused object plane. In step S78, the correction means 23 executes distortion correction processing using the distortion information selected in step S76 or step S77. In step S79, the control unit 22 outputs an image. The output image may be stored on a recording medium, or may be displayed on a display device such as an internal or external output monitor. [Example]

[0032] FIG. 8 is a flowchart showing the correction process of this embodiment. This flow starts when the camera 200 is turned on. In step S81, the correction unit 23 acquires distortion information and magnification information from the lens device 100. In step S82, the control unit 22 causes the camera 200 to acquire an image and acquires shooting information. In step S83, the correction unit 23 determines whether the video mode setting is ON. If the correction unit 23 determines that the video mode setting is ON, it executes the process of step S84. If the correction unit 23 determines that the video mode setting is not ON, it executes the process of step S90. In step S84, the distance map information calculation unit 24 acquires distance map information. In step S85, the subject area information calculation unit 25 acquires subject area information using the distance map information. In step S86, the correction unit 23 selects distortion correction data to be used when performing the process of correcting distortion of the optical image of the subject that occupies the second largest proportion of the screen. In step S87, the correction means 23 performs distortion correction processing using the distortion information selected in step S86. In step S88, the correction means 23 determines whether the breathing correction mode setting is ON. If the correction means 23 determines that the breathing correction mode is ON, it performs processing in step S89, and if it determines that the breathing correction mode is not ON, it performs processing in step S92. In step S89, the correction means 23 performs breathing correction using magnification information. In step S90, it selects distortion correction data for correcting distortion of the optical image on the focused object plane. In step S91, the correction means 23 performs distortion correction processing using the distortion information selected in step S90. In step S92, the control unit 22 outputs the image. The output image may be saved on a recording medium or displayed on a display device such as an internal or external output monitor. [Other Examples] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0033] The disclosure of this embodiment includes the following configurations and methods. (Configuration 1) A control device used in a camera system including a lens device having an imaging optical system including a focus lens group, and an imaging device having an imaging element that acquires an image from an optical image formed by the imaging optical system, a first acquisition means for acquiring a plurality of pieces of distortion information used when performing a process of correcting distortion caused by the imaging optical system on the image; a second acquiring means for acquiring ratio information relating to the ratio of the area of ​​the subject to the total area of ​​the image based on distance map information to the subject for each of a plurality of areas of the image; and a correction unit that performs the correction process using distortion information corresponding to the ratio information. (Configuration 2) 2. The control device according to configuration 1, wherein the plurality of pieces of distortion aberration information include information used when performing a process of correcting distortion aberration of an optical image of an out-of-focus subject. (Configuration 3) 3. The control device according to configuration 1 or 2, wherein the correction means performs the correction process so as to correct distortion of the optical image of the subject having the largest proportion. (Configuration 4) the first acquisition means acquires enlargement ratio information used when enlarging the image after the correction process; 3. The control device according to configuration 1 or 2, wherein the correction means performs the enlargement process using the enlargement ratio information. (Configuration 5) the first acquisition means acquires a plurality of pieces of enlargement ratio information that are used when enlarging the image after the correction process, and each piece of enlargement ratio information corresponds to one of the plurality of pieces of distortion aberration information; The control device according to configuration 4, wherein the correction means performs the correction process using distortion aberration information corresponding to the ratio information, and then performs the enlargement process using enlargement ratio information corresponding to the distortion aberration information. (Configuration 6) The control device according to configuration 4 or 5, wherein the correction means changes the magnification ratio information when the shape of the distortion aberration of the image is pincushion-shaped, and performs the enlargement processing using the changed magnification ratio information. (Configuration 7) 7. The control device according to any one of configurations 4 to 6, wherein the correction means performs the enlargement process using the enlargement ratio information according to a change in the position of the focus lens group during video shooting. (Configuration 8) 8. The control device according to any one of configurations 1 to 7, wherein the first acquisition means acquires the plurality of pieces of distortion aberration information for each position of the focus lens group. (Configuration 9) 9. The control device according to any one of configurations 1 to 8, wherein distortion information used when shooting a moving image is different from distortion information used when shooting a still image. (Configuration 10) the distance map information is updated for each frame during video shooting; 10. The control device according to any one of configurations 1 to 9, wherein the second acquisition means acquires the ratio information in response to an update of the distance map information. (Configuration 11) 11. The control device according to any one of configurations 1 to 10, wherein each of the plurality of pieces of distortion aberration information is different for each image height. (Configuration 12) A control device according to any one of configurations 1 to 11; a lens device comprising the imaging optical system; (Configuration 13) A control device according to any one of configurations 1 to 11; An imaging device comprising the imaging element. (Configuration 14) A control device according to any one of configurations 1 to 11; The imaging optical system is characterized by having:

[0034] A camera system comprising the imaging element. (Method 1) 1. A control method used in a camera system including a lens device having an imaging optical system including a focus lens group, and an imaging device having an imaging element that acquires an image from an optical image formed by the imaging optical system, comprising: acquiring a plurality of pieces of distortion information used when performing a distortion correction process on the image by the imaging optical system; obtaining ratio information regarding a ratio of an area of ​​the object to a total area of ​​the image based on distance map information to the object for each of a plurality of areas of the image; and performing the correction process using distortion information corresponding to the ratio information. (Configuration 15) A program that causes a computer to execute the control method described in Method 1.

[0035] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. [Explanation of symbols]

[0036] 11 Imaging optical system 12 Focus lens group 21 Image sensor 22 Control unit (control device) 23 Correction means (first acquisition means, correction means) 25 Subject area information calculation means (second acquisition means) 100 Lens device 200 Camera (imaging device)

Claims

1. A control device used in a camera system including a lens device having an imaging optical system including a focus lens group, and an imaging device having an imaging element that acquires an image from an optical image formed by the imaging optical system, a first acquisition means for acquiring a plurality of pieces of distortion information used when performing a process of correcting distortion caused by the imaging optical system on the image; a second acquiring means for acquiring ratio information relating to a ratio of an area of ​​the subject to a total area of ​​the image based on distance map information to the subject for each of a plurality of areas of the image; and a correction unit that performs the correction process using distortion information corresponding to the ratio information.

2. 2. The control device according to claim 1, wherein the plurality of pieces of distortion information include information used when performing a process of correcting distortion of an optical image of an out-of-focus subject.

3. 3. The control device according to claim 1, wherein the correction means performs the correction process so as to correct distortion of the optical image of the subject having the largest proportion.

4. the first acquisition means acquires enlargement ratio information used when enlarging the image after the correction process; 3. The control device according to claim 1, wherein the correction means performs the enlargement process using the enlargement ratio information.

5. the first acquisition means acquires a plurality of pieces of enlargement ratio information that are used when enlarging the image after the correction process, and each piece of enlargement ratio information corresponds to one of the plurality of pieces of distortion aberration information; 5. The control device according to claim 4, wherein the correction means performs the correction process using distortion aberration information corresponding to the ratio information, and then performs the enlargement process using enlargement ratio information corresponding to the distortion aberration information.

6. 5. The control device according to claim 4, wherein the correction means changes the magnification ratio information when the shape of the distortion aberration of the image is pincushion-shaped, and performs the enlargement processing using the changed magnification ratio information.

7. 5. The control device according to claim 4, wherein the correction means performs the enlargement process using the enlargement ratio information corresponding to a change in the position of the focus lens group during video shooting.

8. 3. The control device according to claim 1, wherein the first acquisition means acquires the plurality of pieces of distortion information for each position of the focus lens group.

9. 3. The control device according to claim 1, wherein the distortion information used when capturing a moving image is different from the distortion information used when capturing a still image.

10. The distance map information is updated for each frame during video shooting, 3. The control device according to claim 1, wherein the second acquisition means acquires the ratio information in response to an update of the distance map information.

11. 3. The control device according to claim 1, wherein each of the plurality of pieces of distortion information is different for each image height.

12. The control device according to claim 1 or 2; a lens device comprising the imaging optical system;

13. The control device according to claim 1 or 2; An imaging device comprising the imaging element.

14. The control device according to claim 1 or 2; The imaging optical system is characterized by having: A camera system comprising the imaging element.

15. 1. A control method used in a camera system including a lens device having an imaging optical system including a focus lens group, and an imaging device having an imaging element that acquires an image from an optical image formed by the imaging optical system, comprising: acquiring a plurality of pieces of distortion information used when performing a distortion correction process on the image by the imaging optical system; obtaining ratio information regarding a ratio of an area of ​​the object to a total area of ​​the image based on distance map information to the object for each of a plurality of areas of the image; and performing the correction process using distortion information corresponding to the ratio information.

16. A program causing a computer to execute the control method according to claim 15.

Images