Processing device, imaging device, lens device, imaging system, image processing system, image processing device, processing method, and program

Abstract

To provide a processing device capable of processing a photographed image that has been corrected for enlargement or reduction with an appropriate angle of view.SOLUTION: A processing device is used in an imaging system that includes a lens device including an optical system and an imaging device that performs image enlargement / reduction correction on the basis of an optical image formed by the optical system. The processing device includes an acquisition unit that acquires an effective focal length that indicates the focal length when the optical system is in focus during imaging and a correction magnification that indicates the magnification when performing image enlargement / reduction correction, and a second acquisition unit that acquires a focal length for calculating the angle of view of the image after enlargement / reduction correction. When the effective focal length is feff, the correction magnification is β, and the focal length is feff', the processing device satisfies the equation: feff'=β×feff.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a processing device for correcting the enlargement or reduction of an image. [Background technology]

[0002] Conventionally, there is known a technique for enlarging or reducing a captured image according to the state (zoom or focus) of a lens device. Patent Document 1 discloses a configuration for correcting distortion caused by a lens device according to the amount of aberration, and for correcting the enlargement or reduction of a fluctuation in the angle of view accompanying the distortion correction. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-216365 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the configuration of Patent Document 1, the results of the zoom correction are not reflected in the focal length information output from the lens device, so when processing is performed using the focal length information and the zoom-corrected captured image, a mismatch in the angle of view occurs.

[0005] An object of the present invention is to provide a processing device that can process a captured image that has been corrected for enlargement or reduction with an appropriate angle of view. [Means for solving the problem]

[0006] A processing device according to one aspect of the present invention is a processing device used in an imaging system including a lens device including an optical system and an imaging device that performs enlargement / reduction correction of an image based on an optical image formed by the optical system, and includes an acquisition unit that acquires an effective focal length indicating the focal length when the optical system is in focus during imaging and a correction magnification indicating the magnification when performing enlargement / reduction correction of the image, and a second acquisition unit that acquires a focal length for calculating the angle of view of the image after the enlargement / reduction correction, wherein when the effective focal length is feff, the correction magnification is β, and the focal length is feff', feff'=β×feff The present invention is characterized in that the following formula is satisfied: [Effects of the Invention]

[0007] According to the present invention, it is possible to provide a processing device that can process a captured image that has been corrected for enlargement or reduction with an appropriate angle of view. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a configuration diagram of an image processing system according to an embodiment of the present invention. [Figure 2] 4 is a flowchart showing image processing according to the first embodiment. [Figure 3] FIG. 2 is a schematic diagram of image processing according to the first embodiment. [Figure 4] 10 is a flowchart showing image processing according to the second embodiment. [Figure 5] FIG. 10 is a schematic diagram of electronic image stabilization according to a second embodiment. [Figure 6] 10 is a flowchart showing image processing according to a third embodiment. [Figure 7] 10 is a flowchart showing image processing according to a fourth embodiment. [Figure 8] 13 is a flowchart showing image processing according to the fifth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same reference numerals are used to designate the same components, and redundant explanations will be omitted.

[0010] 1 is a diagram showing the configuration of an image processing system 1 according to an embodiment of the present invention. The image processing system 1 has an imaging system 2 and an image processing device 6. The imaging system 2 has a lens device (lens device) 3 and a camera device (imaging device) 4. In this embodiment, the lens device 3 is configured to be detachable from the camera device 4, but it may also be configured integrally with the camera device 4.

[0011] The lens device 3 has an optical system (not shown). The optical system includes at least one imaging lens and forms a subject image on the imaging surface of an imaging element. The optical system may include various optical elements such as a prism, or may include a zoom lens, a focus lens, etc.

[0012] The camera device 4 has an imaging element (not shown) that photoelectrically converts an optical image formed by an optical system. The imaging element is, for example, a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary MOS) image sensor, and outputs an image of a subject formed on an imaging surface.

[0013] The camera device 4 also includes a first acquisition unit 41, a second acquisition unit 42, and a transmission unit 43. The first acquisition unit 41 acquires an effective focal length indicating the focal length when the optical system is focused during shooting, and a correction magnification indicating the magnification when performing image scaling correction. The effective focal length is acquired from the lens device 3 if the lens device 3 is configured to be detachable from the camera device 4, or may be acquired from a storage unit (not shown) provided in the imaging system 2 if the lens device 3 is configured integrally with the camera device 4. The second acquisition unit 42 acquires a focal length for calculating the angle of view of the image after scaling correction. The transmission unit 43 transmits (outputs) the focal length for calculating the angle of view of the image to an external device (in this embodiment, the image processing device 6). In this embodiment, information from the transmission unit 43 is transmitted to the image processing device 6 via the cable 5 (by wired communication), but may also be transmitted by wireless communication.

[0014] In this embodiment, the first acquisition unit 41 and the second acquisition unit 42 are provided in the camera device 4, but the present invention is not limited to this. They may be provided in the lens device 3 or the image processing device 6, or in another processing device. When they are provided in the lens device 3 or the processing device, the transmission unit 43 may also be provided.

[0015] The image processing device 6 includes a PC and processing software. The PC includes an acquisition unit that acquires images captured by the imaging system 2 and shooting information, and a processing unit that performs synthesis processing and the like using the information acquired by the acquisition unit.

[0016] Image processing in each embodiment will be described below. (First embodiment) FIG. 2 is a flowchart showing image processing according to this embodiment. In step S11, the lens device 3 determines the zoom magnification and object distance for capturing a desired image. In step S12, the lens device 3 transmits the effective focal length feff, which indicates the focal length when in focus during image capture, to the camera device 4. The lens device 3 stores the effective focal length feff in a two-dimensional table for the zoom magnification and object distance. The effective focal length feff is calculated from the imaging relationship near the optical axis using Newton's formula. It is preferable that the effective focal length feff be calculated from the imaging relationship near the optical axis that is 20% or less of the image size. By using the imaging relationship near the optical axis, it is possible to calculate a focal length that is not affected by distortion and other aberrations that occur around the periphery of the image.

[0017] In step S13, the camera device 4 corrects distortion. The distortion correction performed in this step will now be described. FIG. 3 is a schematic diagram of image processing according to this embodiment. In FIG. 3, a positive distortion called pincushion distortion occurs. In FIG. 3(A), the dashed line indicates the pincushion distortion, and the solid line indicates an ideal state without aberration. When pincushion distortion is corrected, the peripheral portions of the image are reduced, resulting in missing data in the peripheral portions, as shown in FIG. 3(B). To eliminate the missing data, the corrected image is enlarged as shown in FIG. 3(C). In step S14, the camera device 4 determines (obtains) a correction magnification β, which is the magnification ratio used in the enlargement process. In step S15, the camera device 4 calculates (obtains) a corrected focal length feff′ (a focal length used to calculate the angle of view of the image after enlargement / reduction correction) using the following equation:

[0018] feff'=β×feff In step S16, the camera device 4 transmits (outputs) the corrected focal length and the corrected image.

[0019] In step S17, the image processing device 6 calculates the angle of view using the corrected focal length feff'. In step S18, the image processing device 6 performs image processing such as background synthesis and enlargement / reduction.

[0020] As described above, according to the configuration of this embodiment, even if zoom correction is performed by the camera device 4, the correction magnification β is reflected in the focal length, and it becomes possible to process the image at the correct angle of view after it is output from the camera device 4.

[0021] Here, a method for determining the correction magnification β in this embodiment will be described. When the change in distortion due to focusing from the infinity end to the close-up end is corrected by magnification processing by the camera device 4, the correction magnification β is the correction magnification βmax at the focus position where the amount of distortion in the positive direction is maximum among the distortion at the outermost image height over the entire focus range. The positive direction refers to distortion known as pincushion distortion, and as mentioned above, correcting pincushion distortion causes image loss due to the reduction of peripheral areas. Therefore, it is necessary to crop and enlarge the image. By using the correction magnification βmax at the focus position where the amount of distortion is maximum, it is possible to correct the change in the angle of view without image loss regardless of the focus position.

[0022] At a given zoom focal length, if the amount of distortion at the maximum image height is negative throughout the entire focus range, the correction magnification β is preferably 1. If the amount of distortion is negative, correcting it will not result in data loss, so the correction magnification β can be 1.0. At a given zoom focal length, if the amount of distortion at the maximum image height is positive anywhere throughout the focus range, the correction magnification β is preferably 1.0 or greater and 1.6 or less. If distortion is positive anywhere throughout the focus range, data loss will occur, requiring image cropping and enlargement, so the lower limit of the correction magnification β is 1.0. If the correction magnification β exceeds the upper limit, image degradation due to enlargement will occur, which is undesirable. The correction magnification β is more preferably 1.0 or greater and 1.4 or less, and even more preferably 1.0 or greater and 1.2 or less. (Second embodiment) Fig. 4 is a flowchart showing image processing of this embodiment. The basic flow of Fig. 4 is similar to the flow of Fig. 2. It differs from the first embodiment in that the correction processing resulting from the enlargement / reduction processing within the camera device 4 is electronic image stabilization. In this embodiment, only the configurations that are different from the first embodiment will be described, and a description of the common configurations will be omitted.

[0023] The processes in steps S21 and S22 are similar to those in steps S11 and S12, respectively, and therefore will not be described here.

[0024] In step S23, the camera device 4 performs electronic image stabilization, which corrects, by cropping an image, a shift in the shooting direction caused by vibration of the camera device 4. In step S24, the camera device 4 determines a correction magnification β, which is an enlargement magnification in the image enlargement process performed on the cropped image.

[0025] The processes in steps S25 to S28 are similar to those in steps S15 to S18, respectively, and therefore will not be described further.

[0026] According to the configuration of this embodiment, it is possible to prevent image loss regardless of the amount of image stabilization.

[0027] Here, a method for determining the correction magnification β in this embodiment will be described. Fig. 5 is a schematic diagram of electronic image stabilization in the second embodiment. In Fig. 5, changes in the angle of view caused by vibrations applied to the camera device 4 during shooting are corrected by cropping the image. The dashed line in Fig. 5 indicates the area of ​​the image to be cropped.

[0028] First, the maximum amount of image stabilization to be performed by electronic image stabilization is defined in advance. When the diagonal length of the image size is A and the maximum amount of image stabilization in the diagonal direction at the center of the screen is x, the correction magnification β is calculated using the following formula:

[0029] β=A / (A-2×x) It is preferable that the correction magnification β satisfies the following conditional expression.

[0030] 1.0≦β≦1.6 Since the correction in this embodiment is an enlargement correction, the lower limit is 1.0. Furthermore, if the correction magnification exceeds the upper limit, image degradation due to enlargement occurs, which is not preferable.

[0031] It is more preferable that the correction magnification β satisfies the following conditional expression.

[0032] 1.0≦β≦1.4 It is more preferable that the correction magnification β satisfies the following conditional expression:

[0033] 1.0≦β≦1.2 (Third embodiment) Fig. 6 is a flowchart showing image processing of this embodiment. The basic flow of Fig. 6 is similar to the flow of Fig. 2. It differs from the first embodiment in that the correction processing caused by the zoom processing in the camera device 4 is correction of angle of view change due to the focus position (correction of angle of view fluctuation due to focusing, breathing correction). In this embodiment, only the configuration different from the first embodiment will be described, and a description of the common configuration will be omitted.

[0034] The processes in steps S31 and S32 are similar to those in steps S11 and S12, respectively, and therefore will not be described here.

[0035] In step S33, the camera device 4 performs breathing correction. In step S34, the camera device 4 determines a correction magnification β, which is an enlargement magnification in the image enlargement process performed on the cut-out image.

[0036] The processes in steps S35 to S38 are similar to those in steps S15 to S18, respectively, and therefore will not be described further.

[0037] Here, a method for determining the correction magnification β in this embodiment will be described. When the maximum angle of view is θmax and the minimum angle of view is θmin in the entire focusing range from the close end to the infinity end, the correction magnification β is calculated by the following formula.

[0038] β=tanθmax / tanθmin By using the correction magnification β calculated by the above formula, it is possible to correct the change in the angle of view over the entire focus area without any loss of image.

[0039] It is preferable that the correction magnification β satisfies the following conditional expression.

[0040] 1.0≦β≦1.6 Since the correction in this embodiment is an enlargement correction, the lower limit is 1.0. Furthermore, if the correction magnification exceeds the upper limit, image degradation due to enlargement occurs, which is not preferable.

[0041] It is more preferable that the correction magnification β satisfies the following conditional expression.

[0042] 1.0≦β≦1.4 It is more preferable that the correction magnification β satisfies the following conditional expression:

[0043] 1.0≦β≦1.2 (Fourth embodiment) Fig. 7 is a flowchart showing image processing of this embodiment. The basic flow of Fig. 7 is similar to the flow of Fig. 2. It differs from the first embodiment in that the correction processing resulting from the enlargement / reduction processing within the camera device 4 is image cropping using electronic zoom. In this embodiment, only the configurations that are different from the first embodiment will be described, and a description of the common configurations will be omitted.

[0044] The processes in steps S41 and S42 are similar to those in steps S11 and S12, respectively, and therefore will not be described here.

[0045] In step S43, the camera device 4 performs electronic zooming. In step S44, the camera device 4 determines a correction magnification β, which is the magnification of the image.

[0046] The processes in steps S45 to S48 are similar to those in steps S15 to S18, respectively, and therefore will not be described further. (Fifth embodiment) Fig. 8 is a flowchart showing image processing in this embodiment. The basic flow in Fig. 8 is similar to the flow in Fig. 2. The difference from the first embodiment is that the enlargement / reduction process for aberration correction is performed by the image processing device 6, not the camera device 4. In this embodiment, only the configurations that are different from the first embodiment will be described, and a description of the common configurations will be omitted.

[0047] The processes in steps S51 and S52 are similar to those in steps S11 and S12, respectively, and therefore will not be described here.

[0048] In step S53, the camera device 4 outputs the effective focal length, distortion amount information, and the captured image to the image processing device 6.

[0049] In step S54, the image processing device 6 corrects distortion. In step S55, the image processing device 6 determines a correction magnification β, which is the enlargement magnification in the enlargement process. In step S56, the image processing device 6 calculates the corrected focal length. The processes of steps S57 and S58 are similar to the processes of steps S17 and S18, respectively, and therefore will not be described here.

[0050] In each embodiment, the effective focal length feff is transmitted from the lens device 3, but the present invention is not limited to this. The camera device 4 may hold the information, and the lens device 3 may transmit only drive amount information such as zoom and focus. Information for correcting distortion and the like may also be transmitted by the lens device 3, or may be held in advance by the camera device 4 and calculated from drive amount information such as zoom and focus from the lens device 3. [Other Examples] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0051] The disclosure of this embodiment includes the following configurations and methods. (Configuration 1) A processing device used in an imaging system including a lens device including an optical system and an imaging device that performs enlargement / reduction correction of an image based on an optical image formed by the optical system, an acquisition unit that acquires an effective focal length indicating a focal length when the optical system is in focus during photography and a correction magnification indicating a magnification when performing enlargement / reduction correction of the image; a second acquisition unit that acquires a focal length for calculating an angle of view of the image after the enlargement / reduction correction, When the effective focal length is feff, the correction magnification is β, and the focal length is feff', feff'=β×feff A processing device characterized in that the following formula is satisfied. (Configuration 2) 2. The processing device according to configuration 1, further comprising a transmitting unit that transmits the focal length to an external device. (Configuration 3) 3. The processing device according to configuration 1 or 2, wherein the enlargement / reduction correction is correction of distortion. (Configuration 4) The processing device according to configuration 3, wherein the correction magnification is a correction magnification at a focus position where the amount of distortion in the positive direction is maximum among distortion at the outermost image height in the entire focus range from the close end to the infinity end. (Configuration 5) 3. The processing device according to configuration 1 or 2, wherein the enlargement / reduction correction is an enlargement process using electronic image stabilization. (Configuration 6) In the electronic image stabilization, when the diagonal length of the image size is A and the maximum image stabilization amount in the diagonal direction at the center of the screen is x, the correction magnification is β=A / (A-2×x) 6. The processing device according to configuration 5, wherein the calculation is performed by the following formula: (Configuration 7) 3. The processing device according to configuration 1 or 2, wherein the enlargement / reduction correction is an enlargement / reduction process that corrects a variation in angle of view that accompanies focusing. (Configuration 8) In the angle of view fluctuation correction, when the maximum angle of view is θmax and the minimum angle of view is θmin in the entire focusing range from the close end to the infinity end, the correction magnification is β=tanθmax / tanθmin 8. The processing device according to configuration 7, wherein the calculation is performed by the following formula: (Configuration 9) 3. The processing device according to configuration 1 or 2, wherein the enlargement / reduction correction is an enlargement / reduction process accompanying image cropping using electronic zoom. (Configuration 10) 10. The processing device according to any one of configurations 1 to 9, wherein the effective focal length is calculated from an imaging relationship in the vicinity of the optical axis that is 20% or less of the image size. (Configuration 11) 1.0≦β≦1.6 11. The processing apparatus according to claim 1, wherein the following condition is satisfied: (Configuration 12) A processing device according to any one of configurations 1 to 10; an imaging element that photoelectrically converts an optical image formed by the optical system; (Configuration 13) A processing device according to any one of configurations 1 to 10; and an optical system. (Configuration 14) A processing device according to any one of configurations 1 to 10; An imaging element; an optical system; and an imaging system comprising: (Configuration 15) the imaging system according to configuration 14; An image processing system comprising: an image processing device that performs a synthesis process of a first image acquired from the imaging system and a second image different from the first image. (Configuration 16) An image processing device that performs image synthesis processing, a first acquisition unit that acquires an effective focal length indicating a focal length when the optical system is in focus during photography and a correction magnification indicating a magnification when performing image enlargement / reduction correction; a second acquisition unit that acquires a focal length for calculating an angle of view of the image after the enlargement / reduction correction, When the effective focal length is feff, the correction magnification is β, and the focal length is feff', feff'=β×feff 1. An image processing apparatus characterized in that the following conditional expression is satisfied: (Method 1) 1. A processing method used in an imaging system including a lens device including an optical system and an imaging device that performs image enlargement / reduction correction based on an optical image formed by the optical system, comprising: A step of acquiring an effective focal length indicating a focal length when the optical system is in focus during photography, and a correction magnification indicating a magnification when performing a zoom correction of the image; acquiring a focal length for calculating an angle of view of the image after the enlargement / reduction correction; When the effective focal length is feff, the correction magnification is β, and the focal length is feff', feff'=β×feff A processing method characterized by satisfying the following conditional expression. (Configuration 17) A program that causes a computer to execute the processing method described in Method 1.

[0052] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. [Explanation of symbols]

[0053] 2. Imaging system 3 Lens equipment (lens device) 4. Camera equipment (imaging devices) 41 First acquisition part 42 Second acquisition part

Claims

1. A processing device used in an imaging system including a lens device including an optical system and an imaging device that performs enlargement / reduction correction of an image based on an optical image formed by the optical system, an acquisition unit that acquires an effective focal length indicating a focal length when the optical system is in focus during photography and a correction magnification indicating a magnification when performing enlargement / reduction correction of the image; a second acquisition unit that acquires a focal length for calculating an angle of view of the image after the enlargement / reduction correction, When the effective focal length is feff, the correction magnification is β, and the focal length is feff', feff'=β×feff A processing device characterized in that the following formula is satisfied.

2. The processing device according to claim 1 , further comprising a transmitting unit that transmits the focal length to an external device.

3. 3. The processing device according to claim 1, wherein the enlargement / reduction correction is a correction of distortion.

4. 4. The processing device according to claim 3, wherein the correction magnification is a correction magnification at a focus position where the amount of distortion in the positive direction is maximum among distortions at the outermost image heights in the entire focus range from the closest end to the infinity end.

5. 3. The processing device according to claim 1, wherein the enlargement / reduction correction is an enlargement process using electronic image stabilization.

6. In the electronic image stabilization, when the diagonal length of the image size is A and the maximum image stabilization amount in the diagonal direction at the center of the screen is x, the correction magnification is β = A / (A-2 x) 6. The processing device according to claim 5, wherein the calculation is performed by the following formula:

7. 3. The processing device according to claim 1, wherein the enlargement / reduction correction is an enlargement / reduction process that corrects a variation in angle of view that occurs with focusing.

8. In the angle of view fluctuation correction, when the maximum angle of view is θmax and the minimum angle of view is θmin in the entire focusing range from the close end to the infinity end, the correction magnification is β=tanθmax / tanθmin 8. The processing device according to claim 7, wherein the calculation is performed by the following formula:

9. 3. The processing device according to claim 1, wherein the enlargement / reduction correction is an enlargement / reduction process accompanying image clipping using electronic zoom.

10. 3. The processing device according to claim 1, wherein the effective focal length is calculated from an imaging relationship in the vicinity of the optical axis that is 20% or less of an image size.

11. １．０≦β≦１．６ 3. The processing apparatus according to claim 1, wherein the following condition is satisfied:

12. The processing device according to claim 1 or 2; an imaging element that photoelectrically converts an optical image formed by the optical system;

13. The processing device according to claim 1 or 2; and an optical system.

14. The processing device according to claim 1 or 2; An imaging element; an optical system; and an imaging system comprising:

15. The imaging system according to claim 14; An image processing system comprising: an image processing device that performs a synthesis process of a first image acquired from the imaging system and a second image different from the first image.

16. An image processing device that performs image enlargement / reduction correction and synthesis processing, a first acquisition unit that acquires an effective focal length indicating a focal length when the optical system is in focus during photography and a correction magnification indicating a magnification when the enlargement / reduction correction is performed; a second acquisition unit that acquires a focal length for calculating an angle of view of the image after the enlargement / reduction correction, When the effective focal length is feff, the correction magnification is β, and the focal length is feff', feff'=β×feff 1. An image processing apparatus characterized in that the following conditional expression is satisfied:

17. 1. A processing method used in an imaging system including a lens device including an optical system and an imaging device that performs image enlargement / reduction correction based on an optical image formed by the optical system, comprising: A step of acquiring an effective focal length indicating a focal length when the optical system is in focus during photography, and a correction magnification indicating a magnification when performing a zoom correction on the image; acquiring a focal length for calculating an angle of view of the image after the enlargement / reduction correction; When the effective focal length is feff, the correction magnification is β, and the focal length is feff', feff'=β×feff A processing method characterized by satisfying the following conditional expression.

18. A program causing a computer to execute the processing method according to claim 17.

Images