Image processing device

Abstract

To provide a technique that enables optical correction processing to be suitably performed on a moving image regardless of whether the optical correction processing has been performed or not.SOLUTION: An image processing device according to the present invention includes first acquisition means for acquiring optical property information relating to the optical properties of a lens, second acquisition means for acquiring a moving image consisting of a plurality of frame images acquired through the lens, processing means for performing image processing on each frame image of the moving image on the basis of the optical property information, and output means for outputting the image after the image processing, the optical property information corresponding to the image, and image processing information relating to the image processing corresponding to the image in association with each other.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an image processing device, and more particularly to a technique for correcting moving images based on the optical characteristics of a lens. [Background technology]

[0002] Images captured by imaging devices suffer from image quality degradation due to the optical characteristics of the lens. For example, peripheral light falloff, distortion, and chromatic aberration of magnification occur. These image quality degradations can be reduced by image processing (optical correction processing) based on the optical characteristics of the lens. These degradations can be reduced (corrected) by the imaging device or in a post-process using a device other than the imaging device.

[0003] Since optical characteristics change depending on focal length, shooting distance, aperture value, etc., it is necessary to use the optical characteristics at the time of image capture in order to perform optical correction processing appropriately (to appropriately reduce image quality degradation caused by the optical characteristics of the lens through optical correction processing). Patent Document 1 discloses an image capture device with interchangeable lenses that records a captured image in association with correction data used in optical correction processing. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2014-23063 Summary of the Invention [Problem to be solved by the invention]

[0005] However, while conventional techniques can perform optical correction processing effectively on videos that have not been subjected to optical correction processing, they cannot perform optical correction processing effectively on videos that have been subjected to optical correction processing unique to the imaging device. Performing optical correction processing effectively means that the optical correction processing effectively reduces image quality degradation caused by the optical characteristics of the lens.

[0006] An object of the present invention is to provide a technique that enables optical correction processing to be suitably performed on a moving image regardless of whether optical correction processing has been performed or not. [Means for solving the problem]

[0007] A first aspect of the present invention is an image processing device characterized by having a first acquisition means for acquiring optical property information regarding the optical properties of a lens, a second acquisition means for acquiring a video consisting of multiple frame images acquired through the lens, a processing means for performing image processing on each frame image of the video based on the optical property information, and an output means for outputting the image after the image processing, the optical property information corresponding to the image, and image processing information regarding the image processing corresponding to the image in association with each other.

[0008] A second aspect of the present invention is an image processing device characterized by having an acquisition means for acquiring, for each frame of a video, an image after image processing based on the optical characteristics of a lens, optical characteristic information regarding the optical characteristics, and image processing information regarding the image processing, and a reprocessing means for canceling the image processing being performed on the image based on the image processing information, and performing image processing on the image after the cancellation based on the optical characteristic information.

[0009] A third aspect of the present invention is a method for acquiring an image of a subject, the method comprising: a first acquisition step of acquiring optical characteristic information relating to optical characteristics of a lens; and a second acquisition step of acquiring a moving image consisting of a plurality of frames of images acquired through the lens. 2 acquisition step, a processing step of performing image processing on the image of each frame of the video based on the optical property information, and an output step of outputting the image after the image processing, the optical property information corresponding to the image, and image processing information related to the image processing corresponding to the image in association with each other.

[0010] A fourth aspect of the present invention is an image processing method characterized by having an acquisition step of acquiring, for each frame of a video, an image after image processing based on the optical characteristics of a lens, optical characteristic information related to the optical characteristics, and image processing information related to the image processing, and a reprocessing step of canceling the image processing being performed on the image based on the image processing information, and performing image processing on the image after the cancellation based on the optical characteristic information.

[0011] A fifth aspect of the present invention is a program for causing a computer to function as each of the means of the image processing device.A sixth aspect of the present invention is a computer-readable storage medium storing a program for causing a computer to function as each of the means of the image processing device. [Effects of the Invention]

[0012] According to the present invention, it is possible to perform suitable optical correction processing on a moving image regardless of whether optical correction processing has been performed or not. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a block diagram of an imaging device according to a first embodiment. [Figure 2] 10 is a flowchart of a correction data determination unit according to the first embodiment. [Figure 3] FIG. 10 is a block diagram of an image processing system according to a second embodiment. [Figure 4] FIG. 10 is a schematic diagram of interpolation of optical property information according to the second embodiment. [Figure 5] 10 is a flowchart of the optical correction adjustment device according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0014] An embodiment of the present invention will be described.

[0015] When an imaging device performs optical correction processing (image processing that reduces image quality degradation caused by the optical characteristics of a lens based on the optical characteristics of the lens) on a video in real time, the timing at which an image is captured differs from the timing at which correction data used in the optical correction processing is obtained. As a result, excessive optical correction processing may be performed without using the optical characteristics of the lens at the time of capture. To address this, the amount of correction in the optical correction processing may be limited or the amount of correction may be changed using a predetermined time constant. Image quality degradation caused by optical characteristics includes, for example, peripheral shading, distortion, and chromatic aberration of magnification.

[0016] However, in the above example, the optical characteristics at the time of image capture are not used, and therefore image quality degradation caused by the optical characteristics is not sufficiently reduced. For this reason, there is a need to adjust the image in a post-process using a device other than the image capture device.

[0017] In a post-process, the timing difference can be adjusted to perform optical correction processing using the optical characteristics at the time of capture. However, while conventional technology can perform optical correction processing appropriately for videos that have not been subjected to optical correction processing, it cannot perform optical correction processing appropriately for videos that have been subjected to optical correction processing by the imaging device. This is because both information on the optical correction performed by the imaging device and the optical characteristics of the lens are required. Performing optical correction processing appropriately means that the optical correction processing appropriately reduces image quality degradation caused by the optical characteristics of the lens.

[0018] Therefore, in this embodiment, an image after optical correction processing (image after image processing), optical property information corresponding to the image, and optical correction information corresponding to the image are used. The optical property information corresponding to the image is information about the optical properties at the time of capturing the image, and indicates, for example, the optical properties. The optical correction information corresponding to the image is image processing information related to the optical correction processing performed on the image, and indicates, for example, the correction data (parameters) used in the optical correction processing. In this way, the image before the optical correction processing can be restored from the image after the optical correction processing and the optical correction information corresponding to the image. Then, by performing optical correction processing (reprocessing) on ​​the restored image based on the optical property information, an image can be obtained in which image quality degradation caused by the optical properties is appropriately reduced.

[0019] First Embodiment A first embodiment of the present invention will now be described. Fig. 1 is a block diagram showing the configuration of an image capture device 100, which is an example of an image processing device according to the first embodiment.

[0020] The lens 101 forms an optical image of a subject on the sensor 102. The sensor 102 converts the formed optical image into RAW image data by photoelectrically converting the optical image. The development unit 103 performs development processing on the RAW image data. As a result, image data after development processing is obtained. In the first embodiment, moving image data (moving image data consisting of images of multiple frames) is obtained. The optical correction unit 104 performs optical correction processing on the image of each frame of the moving image obtained by the development unit 103.

[0021] The microcomputer 107 controls the entire imaging device 100 .

[0022] The lens information acquisition unit 106 acquires lens information related to the state of the lens 101. For example, the lens information acquisition unit 106 acquires lens information related to the optical characteristics during image capture (when an optical image is formed from the lens 101 on the sensor 102). The lens information includes information such as the focal length, the subject distance, and the aperture value.

[0023] The microcomputer 107 acquires optical characteristic information regarding the optical characteristics of the lens 101 based on the lens information acquired by the lens information acquisition unit 106. The optical characteristic information indicates at least one of characteristics of peripheral light falloff, characteristics of chromatic aberration of magnification, and characteristics of distortion aberration.

[0024] The method for acquiring the optical characteristic information is not particularly limited. For example, a plurality of pieces of optical characteristic information corresponding to a plurality of pieces of lens information are stored in advance in the storage unit 108, and the microcomputer 107 reads out from the storage unit 108 the optical characteristic information corresponding to the lens information acquired by the lens information acquisition unit 106. If the optical characteristic information corresponding to the lens information acquired by the lens information acquisition unit 106 is not stored in the storage unit 108, the microcomputer 107 may read out from the storage unit 108 the optical characteristic information corresponding to the lens information that is closest to the acquired lens information. The microcomputer 107 may acquire the optical characteristic information corresponding to the lens information acquired by the lens information acquisition unit 106 by reading out the plurality of pieces of optical characteristic information from the storage unit 108 and performing an interpolation process using the plurality of pieces of optical characteristic information.

[0025] The correction data determination unit 109 determines correction data (correction characteristics) that are parameters used in optical correction processing based on the optical characteristic information acquired by the microcomputer 107. For example, the correction data determination unit 109 determines correction data for each frame so that changes in correction data between frames of a moving image are limited. The correction data determination unit 109 may determine correction data so that the correction data changes with a predetermined time constant. The correction data determination unit 109 may determine correction data so that the amount of change in correction data between frames of a moving image is limited to a threshold or less. These methods can prevent abrupt changes in the image due to abrupt changes in correction data. This can suppress the occurrence of changes and the occurrence of excessive optical correction processing due to the timing difference described above.

[0026] For example, the value of the correction data (correction value) may be determined by a predetermined ratio or difference from a target value (e.g., a correction value that eliminates image quality degradation indicated by optical characteristic information). If the target value is always used as the correction value, excessive optical correction processing can easily occur when the focal length, subject distance, aperture value, etc. are suddenly changed. If 80% of the target value is used as the correction value, excessive optical correction processing is less likely to occur even if the focal length, subject distance, aperture value, etc. are suddenly changed.

[0027] 2 is a flowchart showing the processing of the correction data determination unit 109. The processing of FIG. 2 is performed, for example, for each frame of a moving image.

[0028] In step S201, the correction data determination unit 109 determines whether the current timing is the timing to update the target value of the correction data. The target value is updated at a predetermined time interval of, for example, multiple frames. If it is determined that the timing is right, the process proceeds to step S202; otherwise, the process proceeds to step S204.

[0029] In step S202, the correction data determination unit 109 determines (updates) a target value according to the optical characteristic information acquired by the microcomputer 107. The target value is, for example, a correction value that eliminates the image quality degradation indicated by the optical characteristic information.

[0030] In step S203, the correction data determination unit 109 determines a current correction value (a correction value corresponding to the current frame) based on the previous correction value (a correction value determined for the frame immediately before the current frame) and the current target value. For example, the correction data determination unit 109 determines a value exactly midway between the previous correction value and the current target value as the current correction value. Note that in the first processing (first frame), the same value as the target value is determined as the correction value.

[0031] In step S204, the correction data determination unit 109 sets the current correction value determined in step S203 in the optical correction unit 104. The optical correction unit 104 performs optical correction processing using the set correction value.

[0032] In step S205, the correction data determination unit 109 stores the current correction value determined in step S203 as the previous correction value.

[0033] The occurrence of sudden changes in the correction value can be suppressed by performing the above-described processing in Fig. 2. The update frequency of the target value and the way in which the correction value approaches the target value are adjusted depending on, for example, the allowable change in the correction value (change in the correction value).

[0034] Returning to the explanation of FIG. 1, the image output unit 105 and the characteristic output unit 110 output the image (image data) after optical correction processing, the optical characteristic information corresponding to the image, and the optical correction information (correction data) corresponding to the image in a mutually associated manner. When optical correction processing is not performed, the image (image data) without optical correction processing and the optical characteristic information corresponding to the image are output in a mutually associated manner. These output data are, for example, input to an external device or recorded in the storage unit 108. The image output unit 105 outputs the image (image data) after optical correction processing. The characteristic output unit 110 outputs the optical characteristic information and optical correction information corresponding to the image output from the image output unit 105. The method of association is not particularly limited. For example, the image, optical characteristic information, and optical correction information may be stored in a single file. Common metadata (identification information for association) may be added to each of the image, optical characteristic information, and optical correction information, and they may be output as separate files. Furthermore, the image, optical characteristic information, and optical correction information may be output in a single system, or in multiple systems. For example, a separate file containing at least one of the optical characteristic information and the optical correction information may be output from an output unit other than the output unit that outputs the image file, with identification information for association added in the same manner as above.

[0035] Although the present invention has been described as being applied to an imaging device, the image processing device to which the present invention can be applied is not limited to the imaging device. For example, the present invention may also be applied to a personal computer, smartphone, tablet terminal, or the like connected to the imaging device. In this case, for example, the image processing device acquires an image (image data) before optical correction processing from the imaging device, and acquires optical characteristic information from the imaging device or a lens (a lens unit attached to the imaging device (interchangeable lens camera)). The image processing device then performs optical correction processing based on the acquired optical characteristic information. The optical correction information is acquired during the optical correction processing. The image processing device then outputs the image (image data) after optical correction processing, the optical characteristic information corresponding to the image, and the optical correction information corresponding to the image, in association with each other.

[0036] Second Embodiment A second embodiment of the present invention will now be described. Fig. 3 is a block diagram showing the configuration of an image processing system according to the second embodiment. The image processing system in Fig. 3 has an imaging device 300 and an optical correction adjustment device 311.

[0037] The imaging device 300 will now be described. A lens 301 forms an optical image of a subject on a sensor 302. The sensor 302 converts the formed optical image into RAW image data by photoelectrically converting the optical image. An optical correction unit 303 performs optical correction processing on the RAW image data obtained by the sensor 302. A development unit 304 performs development processing on the RAW image data after the optical correction processing. In this way, image data after development processing is obtained. In the second embodiment, video data (video data consisting of images of multiple frames) is obtained. The development processing includes interpolation processing of the RAW image data, gamma conversion, etc.

[0038] The microcomputer 307 controls the entire imaging device 300 .

[0039] The lens information acquisition unit 306 acquires lens information related to the state of the lens 301. For example, the lens information acquisition unit 306 acquires lens information related to the optical characteristics during image capture (when an optical image is formed from the lens 301 on the sensor 302). The lens information includes information such as the focal length, the subject distance, and the aperture value.

[0040] The microcomputer 307 acquires optical characteristic information regarding the optical characteristics of the lens 301 based on the lens information acquired by the lens information acquisition unit 306. The optical characteristic information indicates at least one of characteristics of peripheral light falloff, characteristics of chromatic aberration of magnification, and characteristics of distortion aberration.

[0041] The method for acquiring the optical characteristic information is not particularly limited. For example, a plurality of pieces of optical characteristic information corresponding to a plurality of pieces of lens information are stored in advance in the storage unit 308, and the microcomputer 307 reads out from the storage unit 308 the optical characteristic information corresponding to the lens information acquired by the lens information acquisition unit 306. If the optical characteristic information corresponding to the lens information acquired by the lens information acquisition unit 306 is not stored in the storage unit 308, the microcomputer 307 may read out from the storage unit 308 the optical characteristic information corresponding to the lens information that is closest to the acquired lens information. The microcomputer 307 may acquire the optical characteristic information corresponding to the lens information acquired by the lens information acquisition unit 306 by reading out the plurality of pieces of optical characteristic information from the storage unit 308 and performing interpolation processing using the plurality of pieces of optical characteristic information.

[0042] An example of acquiring optical characteristic information indicating characteristics of vignetting will be described. is a characteristic in which the amount of light (brightness) changes according to the image height h from the center (optical axis) of the lens 301. The characteristic value (e.g., amount of light) of vignetting corresponding to the image height h is represented as V(h), and the characteristic value V(h) corresponding to the focal length z, the subject distance f, and the aperture value i is represented as V(h)(z, f, i). Of the multiple characteristic values ​​V(h) stored in the storage unit 308, eight characteristic values ​​V(h) corresponding to eight coordinates near the coordinate (z, f, i) that combines the focal length, the subject distance, and the aperture value are represented as follows: V(h)(z1,f1,i1) V(h)(z2,f1,i1) V(h)(z1,f2,i1) V(h)(z2,f2,i1) V(h)(z1,f1,i2) V(h)(z2,f1,i2) V(h)(z1,f2,i2) V(h)(z2,f2,i3)

[0043] The focal length z divides the distance between focal lengths z1 and z2 in the ratio nz:1-nz, the subject distance f divides the distance between subject distances f1 and f2 in the ratio nf:1-nf, and the aperture value i divides the distance between aperture values ​​i1 and i2 in the ratio ni:1-ni. The value of the point that divides the distance between points A and B in the ratio n:1-n is represented as Interp(A,B,n).

[0044] In this case, V(h)(z,f,i) can be calculated by the following formula: Figure 4 is a schematic diagram showing the following formula. V(h)(z,f,i) =Interp[ Interp{ Interp(V(z1,f1,i1),V(z2,f1,i1),nz), Interp(V(z1,f2,i1),V(z2,f2,i1),nz), nf}, Interp{ Interp(V(z1,f1,i2),V(z2,f1,i2),nz), Interp(V(z1,f2,i2),V(z2,f2,i2),nz), nf}, ni]

[0045] The correction data determination unit 309 determines correction data (correction characteristics) that are parameters used in the optical correction process based on the optical characteristic information acquired by the microcomputer 307.

[0046] The image output unit 305 and the characteristic output unit 310 output an image (image data) after optical correction processing and gamma conversion, optical characteristic information corresponding to the image, optical correction information corresponding to the image, and gamma conversion information corresponding to the image, all of which are associated with one another. The image output unit 305 outputs the image (image data) after optical correction processing and gamma conversion. The characteristic output unit 310 outputs the optical characteristic information, optical correction information, and gamma conversion information corresponding to the image output from the image output unit 305. The gamma conversion information corresponding to the image is image processing information related to the gamma conversion performed on the image, and indicates, for example, gamma characteristics (characteristics of gamma conversion). Note that the image processing performed after the optical correction processing does not have to be gamma conversion.

[0047] In the second embodiment, gamma conversion is performed after optical correction processing. Therefore, it is not possible to restore an image before optical correction processing in the imaging device 300 (an image that has not been subjected to optical correction processing or gamma conversion) using only the image and optical correction information output from the imaging device 300. Therefore, in the second embodiment, gamma conversion information is also output from the imaging device 300. By doing so, it is possible to cancel the gamma conversion performed on the image from the image and gamma conversion information output from the imaging device 300 and restore an image that has been subjected to optical correction processing but not gamma conversion. It is possible to cancel the optical correction processing performed on the image from the restored image before gamma conversion and the optical correction information and restore an image before optical correction processing in the imaging device 300 (an image that has not been subjected to optical correction processing or gamma conversion). Then, by performing optical correction processing (reprocessing) on ​​these canceled images based on optical characteristic information, it is possible to obtain an image in which image quality degradation caused by optical characteristics is appropriately reduced.

[0048] The optical correction adjustment device 311 will now be described. The optical correction adjustment device 311 is an image processing device that acquires an image output from the imaging device 300 and information associated with the image, and adjusts the optical correction processing applied to the image according to the user's preferences. The optical correction adjustment device 311 is, for example, a personal computer, a smartphone, or a tablet terminal. The optical correction adjustment device 311 may be a part of the imaging device 300.

[0049] The acquisition unit 312 acquires the image and information associated with the image output from the imaging device 300. The adjustment unit 313 adjusts the optical correction processing applied to the image acquired by the acquisition unit 312.

[0050] 5 is a flowchart showing the processing of the optical correction adjustment device 311. Here, it is assumed that the optical correction adjustment device 311 has already acquired the image output from the imaging device 300 and the information associated with it for each frame of the moving image.

[0051] In step S501, the adjustment unit 313 determines whether or not to adjust the optical correction process. If adjustment is to be performed, the process proceeds to step S502; if not, the process in FIG. 5 ends.

[0052] The processing in steps S502 to S506 is performed for each frame of the moving image.

[0053] In step S502, the adjustment unit 313 performs an inverse conversion of the gamma conversion performed by the imaging device 300, based on the gamma conversion information associated with the target image, which is the image of the frame to be processed. This cancels the gamma conversion performed on the target image.

[0054] In step S503, the adjustment unit 313 determines whether or not optical correction processing has been performed on the target image. If optical correction processing has been performed, the process proceeds to step S504; if not, the process proceeds to step S505. The method of determination is not particularly limited. A flag indicating whether or not optical correction processing has been performed may be associated with the target image, and whether or not optical correction processing has been performed may be determined based on the flag. Whether or not optical correction processing has been performed may also be determined based on the presence or absence of optical correction information associated with the target image.

[0055] In step S504, the adjustment unit 313 performs the reverse process of the optical correction process performed by the imaging device 300, based on the optical correction information associated with the target image. This cancels the optical correction process performed on the target image.

[0056] In step S505, the adjustment unit 313 performs the adjustment based on the optical identification information associated with the target image. Based on this, optical correction processing (reprocessing) is performed. If the target image has not been subjected to optical correction processing, optical correction processing is performed on the image obtained in step S502, and if the target image has been subjected to optical correction processing, optical correction processing is performed on the image obtained in step S504. The optical correction processing in step S505 may result in an image in which image quality degradation caused by optical characteristics has been eliminated, or may result in an image in which image quality degradation has been reduced to a degree that is not excessive.

[0057] In step S506, based on the gamma conversion information associated with the target image, the adjustment unit 313 performs the same gamma conversion as that performed by the image capture device 300. This makes it possible to obtain an image that reproduces the image obtained by the image capture device 300 (the image capture device 300 in a state in which the optical correction processing performed by the optical correction unit 303 has been changed).

[0058] Note that, because gamma conversion is performed after optical correction processing is performed in the imaging device 300, the optical correction adjustment device 311 performs the inverse conversion of the gamma conversion (step S502) and the gamma conversion (step S506), but this is not limited to this. For example, as in the first embodiment, if gamma conversion is performed before optical correction processing is performed in the imaging device, steps S502 and S506 are unnecessary. Because gamma characteristics vary depending on the imaging device, it is preferable to use the gamma conversion information described above in steps S502 and S506. However, if the gamma conversion to be performed in the imaging device is predetermined, predetermined inverse conversion and gamma conversion may be performed in steps S502 and S506 without using the gamma conversion information.

[0059] The various controls described above may or may not be performed by a single piece of hardware (e.g., a processor or circuit). The entire device may be controlled by multiple pieces of hardware (e.g., multiple processors, multiple circuits, or a combination of one or more processors and one or more circuits) sharing the processing.

[0060] The processor is a processor in the broad sense, and includes general-purpose processors and dedicated processors. General-purpose processors include, for example, CPUs (Central Processing Units), MPUs (Micro Processing Units), and DSPs (Digital Signal Processors). Dedicated processors include, for example, GPUs (Graphics Processing Units), ASICs (Application Specific Integrated Circuits), and PLDs (Programmable Logic Devices). PLDs include, for example, FPGAs (Field Programmable Gate Arrays). Array), CPLD (Complex Programmable Logic Device), etc.

[0061] Although the embodiments of the present invention have been described in detail, the present invention is not limited to these specific embodiments, and various forms within the scope of the gist of the present invention are also included in the present invention. Furthermore, each of the above-described embodiments merely represents one embodiment of the present invention, and each embodiment can be combined as appropriate.

[0062] <Other embodiments> The present invention can also be realized by a process in which a program that realizes one or more functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in the computer of the system or device read and execute the program, or by a circuit that realizes one or more functions.

[0063] The disclosure of this embodiment includes the following configuration, method, program, and medium. (Configuration 1) a first acquisition means for acquiring optical characteristic information relating to the optical characteristics of the lens; a second acquisition means for acquiring a moving image made up of a plurality of frames of images acquired through the lens; a processing means for performing image processing on an image of each frame of the moving image based on the optical characteristic information; an output means for outputting the image after the image processing, the optical property information corresponding to the image, and image processing information related to the image processing corresponding to the image in association with each other; 1. An image processing device comprising: (Configuration 2) The processing means determines the image processing parameters for each frame so that changes in the image processing parameters between frames of the moving image are limited. 2. The image processing device according to configuration 1, (Configuration 3) The processing means determines the parameter so that the parameter changes with a predetermined time constant. 3. The image processing device according to configuration 2. (Configuration 4) The processing means determines the parameter so that the amount of change in the parameter between frames of the video is limited to a threshold or less. 4. The image processing device according to configuration 2 or 3. (Configuration 5) The optical characteristic information indicates characteristics of peripheral light falloff. 5. The image processing device according to any one of configurations 1 to 4. (Configuration 6) The optical characteristic information indicates characteristics of chromatic aberration of magnification. 6. The image processing device according to any one of configurations 1 to 5. (Configuration 7) The optical characteristic information indicates the characteristics of distortion. 7. The image processing device according to any one of configurations 1 to 6. (Configuration 8) The image processing information indicates parameters of the image processing. 8. The image processing device according to any one of configurations 1 to 7, wherein: (Configuration 9) a second processing means for performing a second image processing on an image of each frame of the moving image after the image processing; and The output means outputs the image after the image processing and the second image processing, the optical property information corresponding to the image, the image processing information corresponding to the image, and second image processing information related to the second image processing corresponding to the image, in association with each other. 9. The image processing device according to any one of configurations 1 to 8. (Configuration 10) The second image processing information indicates a gamma characteristic. 10. The image processing device according to configuration 9, (Configuration 11) The output means records the processed image, the optical property information corresponding to the processed image, and the image processing information corresponding to the processed image in association with each other. 11. The image processing device according to any one of configurations 1 to 10. (Configuration 12) If the image processing is not performed, the output means outputs the image data that has not been subjected to the image processing. The image and the optical characteristic information corresponding to the image are output in association with each other. 12. The image processing device according to any one of configurations 1 to 11. (Configuration 13) an acquisition means for acquiring, for each frame of the video, an image after image processing based on the optical characteristics of the lens, optical characteristic information relating to the optical characteristics, and image processing information relating to the image processing; a re-processing means for canceling the image processing being performed on the image based on the image processing information, and performing image processing on the image after the cancellation based on the optical characteristic information; 1. An image processing device comprising: (Configuration 14) When the image acquired by the acquisition means is an image obtained by performing second image processing after performing the image processing based on the optical characteristics, the acquisition means further acquires second image processing information related to the second image processing, The reprocessing means cancels the image processing being performed on the image after canceling the second image processing being performed on the image. 14. The image processing device according to configuration 13. (Method 1) a first acquisition step of acquiring optical characteristic information relating to optical characteristics of the lens; a second acquisition step of acquiring a moving image consisting of a plurality of frames of images acquired through the lens; a processing step of performing image processing on an image of each frame of the moving image based on the optical characteristic information; an output step of outputting the image after the image processing, the optical property information corresponding to the image, and image processing information related to the image processing corresponding to the image in association with each other; An image processing method comprising: (Method 2) an acquisition step of acquiring, for each frame of the video, an image after image processing based on the optical characteristics of the lens, optical characteristic information related to the optical characteristics, and image processing information related to the image processing; a re-processing step of canceling the image processing being performed on the image based on the image processing information, and performing image processing on the image after the cancellation based on the optical characteristic information; An image processing method comprising: (program) A program for causing a computer to function as each means of the image processing device according to any one of configurations 1 to 14. (medium) 15. A computer-readable storage medium storing a program for causing a computer to function as each means of the image processing device according to any one of configurations 1 to 14. [Explanation of symbols]

[0064] 100, 300: Imaging device 102, 302: Sensor 104,303: Optical correction unit 107,307: Microcomputer 105, 305: Image output section 110, 310: Characteristics output section 311: Optical correction adjustment device 312: Acquisition section 313: Adjustment section

Claims

1. a first acquisition means for acquiring optical characteristic information relating to the optical characteristics of the lens; a second acquisition means for acquiring a moving image made up of a plurality of frames of images acquired through the lens; a processing means for performing image processing on an image of each frame of the moving image based on the optical characteristic information; an output means for outputting the image after the image processing, the optical property information corresponding to the image, and image processing information related to the image processing corresponding to the image in association with each other; 1. An image processing device comprising:

2. The processing means determines the image processing parameters for each frame so that changes in the image processing parameters between frames of the moving image are limited.

2. The image processing device according to claim 1, wherein:

3. The processing means determines the parameter so that the parameter changes with a predetermined time constant.

3. The image processing device according to claim 2.

4. The processing means determines the parameter so that the amount of change in the parameter between frames of the video is limited to a threshold or less.

3. The image processing device according to claim 2.

5. The optical characteristic information indicates characteristics of peripheral light falloff.

2. The image processing device according to claim 1, wherein:

6. The optical characteristic information indicates characteristics of chromatic aberration of magnification.

2. The image processing device according to claim 1, wherein:

7. The optical characteristic information indicates the characteristics of distortion.

2. The image processing device according to claim 1, wherein:

8. The image processing information indicates parameters of the image processing.

2. The image processing device according to claim 1, wherein:

9. a second processing means for performing a second image processing on an image of each frame of the moving image after the image processing; and The output means outputs the image after the image processing and the second image processing, the optical property information corresponding to the image, the image processing information corresponding to the image, and second image processing information related to the second image processing corresponding to the image, in association with each other.

2. The image processing device according to claim 1, wherein:

10. The second image processing information indicates a gamma characteristic.

10. The image processing device according to claim 9,

11. The output means records the processed image, the optical property information corresponding to the processed image, and the image processing information corresponding to the processed image in association with each other.

2. The image processing device according to claim 1, wherein:

12. If the image processing is not performed, the output means outputs the image that has not been subjected to the image processing and the optical characteristic information corresponding to the image in association with each other.

2. The image processing device according to claim 1, wherein:

13. an acquisition means for acquiring, for each frame of the video, an image after image processing based on the optical characteristics of the lens, optical characteristic information relating to the optical characteristics, and image processing information relating to the image processing; a re-processing means for canceling the image processing being performed on the image based on the image processing information, and performing image processing on the image after the cancellation based on the optical characteristic information; 1. An image processing device comprising:

14. When the image acquired by the acquisition means is an image obtained by performing second image processing after performing the image processing based on the optical characteristics, the acquisition means further acquires second image processing information related to the second image processing, The reprocessing means cancels the image processing being performed on the image after canceling the second image processing being performed on the image.

14. The image processing device according to claim 13.

15. a first acquisition step of acquiring optical characteristic information relating to optical characteristics of the lens; a second acquisition step of acquiring a moving image consisting of a plurality of frames of images acquired through the lens; a processing step of performing image processing on an image of each frame of the moving image based on the optical characteristic information; an output step of outputting the image after the image processing, the optical characteristic information corresponding to the image, and image processing information related to the image processing corresponding to the image in association with each other; An image processing method comprising:

16. an acquisition step of acquiring, for each frame of the video, an image after image processing based on the optical characteristics of the lens, optical characteristic information related to the optical characteristics, and image processing information related to the image processing; a re-processing step of canceling the image processing being performed on the image based on the image processing information, and performing image processing on the image after the cancellation based on the optical characteristic information; An image processing method comprising:

17. A program for causing a computer to function as each of the means of the image processing device according to any one of claims 1 to 14.

18. A computer-readable storage medium storing a program for causing a computer to function as each of the means of the image processing device according to any one of claims 1 to 14.

Images