Image processing device, image processing method and computer-readable medium

Abstract

An image processing apparatus according to the invention includes a first acquisition unit configured to acquire optical property information regarding an optical property of a lens, a second acquisition unit configured to acquire a moving image including images of a plurality of frames acquired via the lens, a processing unit configured to perform image processing on an image of each frame of the moving image based on the optical property information, and an output unit configured to output an image after image processing, the optical property information corresponding to the image, and image processing information corresponding to the image regarding image processing in association with each other.

Description

BACKGROUND OF THE INVENTIONField of the invention

[0001] The present invention relates to an image processing apparatus, and more particularly to a technique for correcting a moving image based on an optical characteristic of a lens. Description of related technology

[0002] In an image obtained by an imaging device, image quality deterioration occurs due to an optical characteristic of a lens. For example, peripheral light falloff, distortion aberration, chromatic magnification aberration, and the like occur. Such image quality deterioration can be reduced by image processing (optical correction processing) based on the optical characteristic of the lens, and is reduced (corrected) by the imaging device, or is reduced in a subsequent process using a device other than the imaging device.

[0003] Since the optical property changes depending on a focal length, a photographing distance, an aperture value, and the like, it is necessary to use the optical property at the time of imaging to appropriately perform optical correction processing (in order to reduce image quality deterioration caused by the optical property of the lens without over- or under-processing the optical correction processing). JP 2014-23063 A discloses an interchangeable lens imaging device that records a captured image and correction data used for optical correction processing in association with each other.

[0004] Although the optical correction processing can be properly performed on a moving image in which optical correction processing has not been previously performed, the optical correction processing cannot be properly performed on a moving image in which specific optical correction processing has already been performed by an imaging device. The term "properly performing optical correction processing" means reducing the image quality deterioration caused by the optical properties of the lens without excessive or insufficient optical correction processing. SUMMARY OF THE INVENTION

[0005] The present invention provides a technique capable of appropriately performing optical correction processing on a moving image regardless of whether the optical correction processing has already been performed or not.

[0006] According to its first aspect, the present invention provides an image processing apparatus comprising a first acquisition unit configured to acquire optical property information regarding an optical property of a lens, a second acquisition unit configured to acquire a moving image including images of a plurality of frames acquired via the lens, a processing unit configured to perform first image processing on an image of each frame of the moving image based on the optical property information, and an output unit configured to output an image after the first image processing, the optical property information corresponding to the image, and first image processing information regarding the first image processing corresponding to the image in association with each other.

[0007] According to its second aspect, the present invention provides an image processing apparatus comprising an acquisition unit configured to acquire an image after first image processing based on an optical property of a lens, optical property information regarding the optical property, and first image processing information regarding the first image processing for each frame of a moving image, and a reprocessing unit configured to cancel the first image processing performed on the image based on the first image processing information and perform second image processing on an image after the cancellation based on the optical property information.

[0008] According to its third aspect, the present invention provides an image processing method comprising a first acquisition step of acquiring optical property information regarding an optical property of a lens, a second acquisition step of acquiring a moving image including images of a plurality of frames acquired via the lens, a processing step of performing image processing on an image of each frame of the moving image based on the optical property information, and an output step of outputting an image after the image processing, the optical property information corresponding to the image, and image processing information regarding the image processing corresponding to the image in association with each other.

[0009] According to its fourth aspect, the present invention provides an image processing method comprising an acquisition step of acquiring an image after first image processing based on an optical property of a lens, optical property information regarding the optical property and first image processing information regarding the first image processing for each frame of a moving image, and a reprocessing step of canceling the first image processing performed on the image based on the first image processing information and performing second image processing on an image after the cancellation based on the optical property information.

[0010] According to its fifth aspect, the present invention provides a computer-readable medium storing a program for causing a computer to operate as each unit of the image processing apparatus described above.

[0011] Further features of the present invention will become apparent from the following description of embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 shows a block diagram of an imaging device according to a first embodiment, Fig. 2 shows a flowchart of a correction data determination unit according to the first embodiment, Fig. 3 shows a block diagram of an image processing system according to a second embodiment, Fig. 4 shows a schematic representation of an interpolation of optical property information according to the second embodiment, and Fig. 5 shows a flowchart of an optical correction adjusting device according to the second embodiment. DESCRIPTION OF THE EMBODIMENTS

[0012] Embodiments of the present invention are described. Each embodiment of the present invention described below may be implemented alone or as a combination of a plurality of the embodiments or their features, where necessary, or where combining elements or features from individual embodiments into a single embodiment is advantageous.

[0013] In a case where optical correction processing (image processing for reducing image quality deterioration caused by an optical property of a lens based on the optical property) is performed by an imaging device on a moving image in real time, a timing at which an image is obtained by imaging differs from a timing at which correction data used for optical correction processing is obtained. For this reason, excessive optical correction processing may be performed without using the optical property of the lens at the time of imaging. Therefore, a correction amount of the optical correction processing may be limited, or the correction amount may be changed with a predetermined time constant.The image quality deterioration caused by the optical property is, for example, peripheral light falloff, distortion aberration or chromatic magnification aberration.

[0014] However, since the above example does not utilize the optical property at the time of imaging, the image quality degradation caused by the optical property is not sufficiently reduced. Therefore, it is necessary to adjust an image in a subsequent process using a device different from the imaging device.

[0015] In the subsequent process, optical correction processing can be performed using the optical characteristic at the time of imaging by adjusting the timing difference. Although the prior art can properly perform optical correction processing on a moving image for which optical correction processing has not been performed, optical correction processing cannot properly perform optical correction processing on a moving image for which optical correction processing has been performed by the imaging device. This is because both information of optical correction performed by the imaging device and the optical characteristic of the lens are required.The term “to appropriately perform optical correction processing” means to reduce image quality deterioration caused by the optical characteristic of the lens without too much or too little optical correction processing.

[0016] Therefore, in the present embodiment, an image after optical correction processing (image after image processing), optical property information corresponding to the image, and optical correction information corresponding to the image are used. The optical property information corresponding to the image is information regarding an optical property at the time of capturing the image and indicates, for example, the optical property. The optical correction information corresponding to the image is image processing information regarding the optical correction processing performed on the image and indicates, for example, correction data (parameters) used for the optical correction processing. In this way, the image before the optical correction processing can be restored from the image after the optical correction processing and the optical correction information corresponding to the image.Then, by performing optical correction processing (reprocessing) based on the optical property information on the restored image, it is possible to obtain an image in which image quality deterioration caused by the optical property has been reduced without over- or under-reduction. First embodiment

[0017] A first embodiment of the present invention will be described below. Fig. 1 is a block diagram showing a configuration of an imaging device 100 which is an example of an image processing device according to the first embodiment.

[0018] A lens 101 forms an optical image of an object on a sensor 102. The sensor 102 performs photoelectric conversion on the formed optical image to convert the optical image into RAW image data. A developing unit 103 performs development processing on the RAW image data. As a result, image data after the development processing is obtained. In the first embodiment, moving image data (data of a moving image including images of a plurality of frames) is obtained. An optical correction unit 104 performs optical correction processing on an image of each frame of the moving image obtained by the developing unit 103.

[0019] A microcomputer 107 controls the entire imaging device 100.

[0020] A lens information acquisition unit 106 acquires lens information regarding a state of the lens 101. For example, the lens information acquisition unit 106 acquires lens information regarding an optical property at the time of imaging (when an optical image is formed from the lens 101 to the sensor 102). The lens information includes information such as a focal length, a subject distance, and an aperture value.

[0021] The microcomputer 107 acquires optical property information regarding the optical property of the lens 101 based on the lens information acquired by the lens information acquisition unit 106. The optical property information indicates at least one of a peripheral light falloff property, a chromatic magnification aberration property, and a distortion aberration property.

[0022] A method for acquiring the optical property information is not particularly limited. For example, a plurality of pieces of optical property information corresponding to a plurality of pieces of lens information are stored in advance in a storage unit 108, and the microcomputer 107 reads the optical property information corresponding to the lens information acquired by the lens information acquisition unit 106 from the storage unit 108. If the optical property information corresponding to the lens information acquired by the lens information acquisition unit 106 is not stored in the storage unit 108, the microcomputer 107 may read optical property information corresponding to the lens information closest to the lens information from the storage unit 108.The microcomputer 107 can acquire the optical property information corresponding to the lens information acquired by the lens information acquisition unit 106 by reading a plurality of pieces of optical property information from the storage unit 108 and performing interpolation processing using the plurality of pieces of optical property information.

[0023] Based on the optical characteristic information acquired by the microcomputer 107, a correction data determination unit 109 determines correction data (a correction characteristic) representing a parameter used for optical correction processing. The correction data determination unit 109 determines the correction data, for example, for each frame such that a change in the correction data between frames of the moving image is restricted. The correction data determination unit 109 may determine the correction data such that the correction data changes at a predetermined time constant. The correction data determination unit 109 may determine the correction data such that an amount of change in the correction data between frames of the moving image is restricted to a threshold value or less.With these methods, it is possible to suppress the occurrence of a rapid change in the image due to a rapid change in the correction data, and to suppress the occurrence of excessive optical correction processing due to the time difference as described above.

[0024] A value of the correction data (correction value) can be determined, for example, by a predetermined ratio or a predetermined difference from a target value (for example, a correction value specified by the optical characteristic information for eliminating image quality deterioration). If the target value is always used as the correction value, excessive optical correction processing is likely to occur when the focal length, object distance, aperture value, and the like are changed rapidly. If 80% of the target value is used as the correction value, excessive optical correction processing is less likely to occur even when the focal length, object distance, aperture value, and the like are changed rapidly.

[0025] Fig. 2 shows a flowchart of a processing of the correction data determination unit 109. The processing in Fig. 2 is performed, for example, for each frame of the moving image.

[0026] In step S201, the correction data determination unit 109 determines whether the current time is a time to update the target value of the correction data. For example, the target value is updated at predetermined time intervals for a plurality of frames. If it is determined that the current time is the update time, the process proceeds to step S202; otherwise, the process proceeds to step S204.

[0027] In step S202, the correction data determination unit 109 determines (updates) the target value according to the optical property information acquired by the microcomputer 107. The target value is, for example, a correction value for eliminating image quality deterioration, which is indicated by the optical property information.

[0028] In step S203, the correction data determination unit 109 determines a current correction value (a correction value corresponding to a current frame) based on a previous correction value (a correction value determined for a previous frame of the current frame) and a current target value. The correction data determination unit 109 determines, for example, a median value between the previous correction value and the current target value as the current correction value. In the first processing (the first frame), the same value as the target value is determined as the correction value.

[0029] In step S204, the correction data determination unit 109 sets the current correction value determined in step S203 to the optical correction unit 104. The optical correction unit 104 performs optical correction processing using the set correction value.

[0030] In step S205, the correction data determination unit 109 stores the current correction value determined in step S203 as the previous correction value.

[0031] By performing the processing described above, Fig. 2, it is possible to suppress the occurrence of a rapid change in the correction value. The update frequency of the setpoint, how the correction value approaches the setpoint, and the like are adjusted, for example, according to a permissible correction value change (change in the correction value).

[0032] The description returns to Fig. 1. An image (image data) after optical correction processing, optical characteristic information corresponding to the image, and optical correction information (correction data) corresponding to the image are output by the image output unit 105 and a characteristic output unit 110 in association with each other. When the optical correction processing has not been performed, an image (image data) for which the optical correction processing has not been performed and optical characteristic information corresponding to the image are output in association with each other. The output is input to an external device or recorded in the storage unit 108, for example. The image output unit 105 outputs the image (image data) after optical correction processing. The characteristic output unit 110 outputs optical characteristic information and optical correction information corresponding to the image output from the image output unit 105.The connection method is not particularly limited. For example, the image, optical property information, and optical correction information can be stored in a single file. Common metadata (identification information for the connection) can be added to each of the image, optical property information, and optical correction information, and these can be output as separate files. The image, optical property information, and optical correction information can be output by one system or can be output by multiple systems.For example, an output unit different from the output unit that outputs the image file may output another file containing the optical property information and / or the optical correction information after assigning the identification information for the compound similarly as described above.

[0033] Note that although the example in which the present invention is applied to an imaging device has been described, the image processing device to which the present invention can be applied is not limited to the imaging device. For example, the present invention can be applied to an image processing device such as a personal computer, a smartphone, a tablet device, or the like connected to an imaging device. In such a case, for example, the image processing device acquires the image (image data) before the optical correction processing from the imaging device and acquires the optical characteristic information from the imaging device or the lens (the lens unit attached to the imaging device (interchangeable lens camera)).The image processing device then performs optical correction processing based on the acquired optical property information. The optical correction information is acquired in the process of performing the optical correction processing. Afterward, the image processing device outputs an image (image data) after the optical correction processing, optical property information corresponding to the image, and optical correction information corresponding to the image in conjunction with each other. Second embodiment

[0034] A second embodiment of the invention will be described below. Fig. 3 shows a block diagram of the structure of an image processing system according to the second embodiment. The image processing system in Fig. 3 includes an imaging device 300 and an optical correction adjustment device 311.

[0035] The imaging device 300 will be described. A lens 301 forms an optical image of an object on a sensor 302. The sensor 302 performs photoelectric conversion on the formed optical image to convert the optical image into RAW image data. An optical correction unit 303 performs optical correction processing on the RAW image data obtained by the sensor 302. A development unit 304 performs development processing on the RAW image data after the optical correction processing. As a result, image data after the development processing is obtained. In the second embodiment, moving image data (data of a moving image containing images of a plurality of frames) is obtained. The development processing includes interpolation processing of the RAW image data, gamma conversion, and the like.

[0036] A microcomputer 307 controls the entire imaging device 300.

[0037] A lens information acquisition unit 306 acquires lens information regarding a state of the lens 301. For example, the lens information acquisition unit 306 acquires lens information regarding an optical property at the time of imaging (when an optical image is formed from the lens 301 to the sensor 302). The lens information includes information such as a focal length, a subject distance, and an aperture value.

[0038] The microcomputer 307 acquires optical property information regarding an optical property of the lens 301 based on the lens information acquired by the lens information acquisition unit 306. The optical property information indicates a peripheral light falloff property, a chromatic magnification aberration property, and a distortion aberration property.

[0039] A method for acquiring the optical property information is not particularly limited. For example, a plurality of pieces of optical property information corresponding to a plurality of pieces of lens information are stored in advance in a storage unit 308, and the microcomputer 307 reads the optical property information corresponding to the lens information acquired by the lens information acquisition unit 306 from the storage unit 308. If the optical property information corresponding to the lens information acquired by the lens information acquisition unit 306 is not stored in the storage unit 308, the microcomputer 307 may read the optical property information closest to the lens information from the storage unit 308.The microcomputer 307 can acquire the optical property information corresponding to the lens information acquired by the lens information acquisition unit 306 by reading a plurality of pieces of optical property information from the storage unit 308 and performing interpolation processing using the plurality of pieces of optical property information.

[0040] An example of acquiring optical characteristic information indicating the peripheral light falloff characteristic will be described. The peripheral light falloff characteristic is a characteristic in which a light quantity (brightness) changes according to an image height h from the center (optical axis) of the lens. A characteristic value (e.g., a light quantity) of the peripheral light falloff corresponding to the image height h is represented as V(h), and a characteristic value V(h) corresponding to the focal length z, the object distance f, and the aperture value i is represented as V(h)(z, f, i). Among a plurality of characteristic values ​​V(h) stored in the storage unit 308, eight characteristic values ​​V(h) corresponding to eight coordinates near the coordinates (z, f, i) obtained by combining the focal length, the object distance, and the aperture value are represented as follows. V(h)(z1,f1,i1) V(h)(z2,f1,i1) V(h)(z1,f2,i1) V(h)(z2,f2,i1) V(h)(z1,f1,i2) V(h)(z2,f1,i2) V(h)(z1,f2,i2) V(h)(z2,f2,i3)

[0041] It is assumed that the focal length z internally divides a space between the focal length z1 and the focal length z2 into nz : 1-nz, the object distance f internally divides a space between the object distance f1 and the object distance f2 into nf : 1-nf, and the aperture value i internally divides a space between the aperture value i1 and the aperture value i2 into ni : 1-ni. Furthermore, a value of a point that internally divides a space between a point A and a point B into n : 1-n is represented as Interp(A, B, n).

[0042] In this case, V(h)(z, f, i) can be calculated by the following formula. Fig. 4 shows a schematic diagram illustrating the following formula. V(h)(z,f,i) =Interp[ Interp{ Interp(V(z1,f1,i1),V(z2,f1,i1),nz), Interp(V(z1,f2,i1),V(z2,f2,i1),nz), nf}, Interp{ Interp(V(z1,f1,i2),V(z2,f1,i2),nz), Interp(V(z1,f2,i2),V(z2,f2,i2),nz), nf}, ni]

[0043] Based on the optical characteristic information acquired by the microcomputer 307, a correction data determining unit 309 determines correction data (a correction characteristic) representing a parameter used for the optical correction processing.

[0044] An image (image data) after optical correction processing and gamma conversion, optical characteristic information corresponding to the image, optical correction information corresponding to the image, and gamma conversion information corresponding to the image are output by the image output unit 305 and a characteristic output unit 310 in conjunction with each other. The image output unit 305 outputs the image (image data) after optical correction processing and gamma conversion. The characteristic output unit 310 outputs the optical characteristic information, the optical correction information, and the gamma conversion information corresponding to the image output from the image output unit 305. The gamma conversion information corresponding to the image is image processing information regarding gamma conversion performed on the image and indicates, for example, a gamma characteristic (characteristic of gamma conversion).It is noted that the image processing performed after the optical correction processing cannot be gamma conversion.

[0045] In the second embodiment, gamma conversion is performed after the optical correction processing. Therefore, it is not possible to restore the image before the optical correction processing in the imaging device 300 (the image to which neither the optical correction processing nor the gamma conversion was performed) from only the image output from the imaging device 300 and the optical correction information. Therefore, in the second embodiment, the gamma conversion information is also output from the imaging device 300. As a result, it is possible to eliminate the gamma conversion performed on the image from the image output from the imaging device 300 and the gamma conversion information, and restore the image as an image to which the optical correction processing was performed and the gamma conversion was not performed.From the restored image before gamma conversion and the optical correction information, it is possible to remove the optical correction processing performed on the image and restore the image as an image before the optical correction processing in the imaging device 300 (the image to which neither the optical correction processing nor the gamma conversion was performed). By performing optical correction processing (reprocessing) based on the optical characteristic information on the image after removal, it is possible to obtain an image in which the image quality deterioration caused by the optical characteristic is reduced without excessive or insufficient.

[0046] The optical correction adjustment device 311 will now be described. The optical correction adjustment device 311 is an image processing device that acquires an image output from the imaging device 300 and information associated with the image, and adjusts optical correction processing performed on the image according to a user preference. The optical correction adjustment device 311 is, for example, a personal computer, a smartphone, a tablet device, or the like. The optical correction adjustment device 311 may be a part of the imaging device 300.

[0047] An acquisition unit 312 acquires an image output from the imaging device 300 and information associated with the image. An adjustment unit 313 adjusts the optical correction processing performed on the image acquired by the acquisition unit 312.

[0048] Fig. 5 shows a flowchart of processing of the optical correction adjusting device 311. Here, it is assumed that the optical correction adjusting device 311 has already acquired the image output from the imaging device 300 and the information associated with the image for each frame of the moving image.

[0049] In step S501, the adjustment unit 313 determines whether or not to perform adjustment of the optical correction processing. If the adjustment is performed, the process proceeds to step S502, and otherwise, the processing ends. Fig. 5.

[0050] The processing in steps S502 to S506 is performed for each frame of the moving image.

[0051] In step S502, the adjustment unit 313 performs an inverse conversion of the gamma conversion performed by the imaging device 300 based on the gamma conversion information associated with the target image, which is the image of the frame to be processed. As a result, the gamma conversion performed on the target image is canceled.

[0052] In step S503, the adjustment unit 313 determines whether or not the optical correction processing has already been performed on the target image (by the imaging device 300). If the optical correction processing has already been performed, the process proceeds to step S504; otherwise, the process proceeds to step S505. The determination method is not particularly limited. A flag indicating whether or not the optical correction processing has been performed is associated with the target image, and according to the flag, it can be determined whether or not the optical correction processing has been performed. Whether or not the optical correction processing has been performed can be determined according to the presence or absence of the optical correction information associated with the target image.

[0053] In step S504, the adjustment unit 313 performs inverse processing of the optical correction processing performed by the imaging device 300 based on the optical correction information associated with the target image. As a result, the optical correction processing performed on the target image is canceled.

[0054] In step S505, the adjustment unit 313 performs optical correction processing (reprocessing) based on the optical property information associated with the target image. In a case where the optical correction processing by the imaging device 300 has not already been performed on the target image, the optical correction processing is performed on the image obtained in step S502, and in a case where the optical correction processing by the imaging device 300 has already been performed on the target image, the optical correction processing is performed on the image obtained in step S504.Through the optical correction processing in step S505, an image in which image quality deterioration caused by an optical property has been eliminated, or an image in which image quality deterioration is reduced to an extent that the image quality deterioration does not become excessive can be obtained.

[0055] In step S506, the adjustment unit 313 performs the same gamma conversion as the gamma conversion performed by the imaging device 300 based on the gamma conversion information associated with the target image. As a result, it is possible to obtain an image obtained by reproducing an image obtained by the imaging device 300 (the imaging device 300 in a state where the optical correction processing performed by the optical correction unit 303 is changed).

[0056] Note that, since gamma conversion is performed after the optical correction processing is performed in the imaging device 300, the inverse conversion (step S502) of the gamma conversion and the gamma conversion (step S506) are performed in the optical correction adjusting device 311, but the present invention is not limited thereto. For example, in a case where gamma conversion is performed before the optical correction processing is performed in the imaging device as in the first embodiment, steps S502 and S506 are not required. Since the gamma characteristic varies depending on the imaging device, it is preferable that the gamma conversion information described above in steps S502 and S506 be used.However, in a case where the gamma conversion to be performed by the imaging device is determined in advance, predetermined inverse conversion and gamma conversion may be performed in steps S502 and S506 without using the gamma conversion information.

[0057] It should be noted that the various control types described above may represent processing performed by a piece of hardware (e.g., a processor or a circuit), or may represent other processing. The processing may be distributed among a plurality of pieces of hardware (e.g., a plurality of processors, a plurality of circuits, or a combination of one or more processors and one or more circuits), thereby executing control of the entire device.

[0058] The above processor is also a processor in the broad sense and includes general-purpose processors and dedicated processors. Examples of general-purpose processors include a central processing unit (CPU), a microprocessing unit (MPU), a digital signal processor (DSP), etc. Examples of dedicated processors include a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), a programmable logic device (PLD), etc. Examples of PLDs include a field-programmable gate array (SPGA), a complex programmable logic device (CPLD), etc.

[0059] The above-described embodiment (including modification examples) is merely an example. Any structure obtained by appropriately modifying or changing configurations of the embodiment within the scope of the subject matter of the present invention also belongs to the present invention. The present invention also includes other configurations obtained by appropriately combining various features of the embodiment.

[0060] According to the present invention, it is possible to perform appropriate optical correction processing on a moving image regardless of whether the optical correction processing has already been performed or not. Further examples

[0061] Embodiments of the present invention may also be implemented by a computer of a system or apparatus that retrieves and executes computer-executable instructions (e.g., one or more programs) recorded on a storage medium (which may also be more fully referred to as a "non-transitory computer-readable storage medium") for performing the functions of one or more of the embodiments described above, and / or that includes one or more circuits (e.g., an application-specific integrated circuit (ASIC)) for performing the functions of one or more of the embodiments described above, and by a method,which is carried out by the computer of the system or device, for example, by reading and executing the computer-executable instructions from the storage medium to perform the functions of one or more of the above-described embodiments and / or controlling the one or more circuits to perform the functions of one or more of the above-described embodiments. The computer may comprise one or more processors (e.g., central processing unit (CPU),Microprocessing Unit (MPU)) and may include a network of separate computers or separate processors for reading and executing the computer-executable instructions. The computer-executable instructions may be provided to the computer, for example, from a network or the storage medium. The storage medium may include, for example, a hard disk and / or random access memory (RAM) and / or read-only memory (ROM) and / or distributed computing system memory and / or an optical disk (such as a compact disk (CD), digital versatile disk (DVD), or Blu-ray Disk (BD)™) and / or a flash memory device and / or a memory card, and the like.

[0062] Although the present invention has been described with reference to exemplary embodiments, it should be understood that the invention is not limited to the disclosed embodiments. The scope of the following claims is intended to be accorded the broadest interpretation to encompass such modifications and equivalent structures and functions. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] JP 2014-23063 A

[0003]

Claims

[1] Image processing device with a first acquisition unit configured to acquire optical property information regarding an optical property of a lens, a second acquisition unit configured to acquire a moving image containing images of a plurality of frames acquired via the lens, a processing unit configured to perform a first image processing on an image of each frame of the moving image based on the optical property information, and an output unit configured to output an image after the first image processing, the optical property information corresponding to the image, and the first image processing information corresponding to the image with respect to the first image processing in conjunction with one another. [2] The image processing apparatus according to claim 1, wherein the processing unit determines a parameter of the first image processing for each frame such that a change in the parameter of the first image processing is restricted between frames of the moving image. [3] The image processing apparatus according to claim 2, wherein the processing unit determines the parameter such that the parameter changes with a predetermined time constant. [4] The image processing apparatus according to claim 2 or 3, wherein the processing unit determines the parameter such that an amount of change of the parameter between frames of the moving image is limited to a threshold value or less. [5] The image processing apparatus according to any one of claims 1 to 4, wherein the optical characteristic information indicates a characteristic of a peripheral light falloff. [6] The image processing apparatus according to any one of claims 1 to 5, wherein the optical characteristic information indicates a characteristic of a chromatic magnification aberration. [7] The image processing apparatus according to any one of claims 1 to 6, wherein the optical characteristic information indicates a characteristic of a distortion aberration. [8] The image processing apparatus according to any one of claims 1 to 7, wherein the first image processing information indicates a parameter of the first image processing. [9] Image processing apparatus according to one of claims 1 to 8, further comprising a second processing unit configured to perform a second image processing on an image of each frame of the moving image after the first image processing, wherein the output unit outputs an image after the first image processing and the second image processing, the optical property information corresponding to the image, the first image processing information corresponding to the image, and the second image processing information corresponding to the image with respect to the second image processing in association with each other. [10] The image processing apparatus according to claim 9, wherein the second image processing information indicates a gamma characteristic. [11] The image processing apparatus according to any one of claims 1 to 10, wherein the output unit records an image after the first image processing, the optical property information corresponding to the image, and the first image processing information corresponding to the image in association with each other. [12] The image processing apparatus according to any one of claims 1 to 11, wherein, in a case where the processing unit does not perform first image processing on the image of each frame of the moving image based on the optical characteristic information, the output unit outputs an image on which first image processing is not performed and the optical characteristic information corresponding to the image in association with each other. [13] Image processing device with an acquisition unit configured to acquire an image after a first image processing based on an optical property of a lens, optical property information regarding the optical property, and first image processing information regarding the first image processing for each frame of a moving image, and a reprocessing unit configured to cancel the first image processing performed on the image based on the first image processing information and perform a second image processing on an image after the cancellation based on the optical property information. [14] Image processing apparatus according to claim 13, wherein in a case where the image acquired by the acquisition unit is an image obtained by performing a third image processing after performing the first image processing based on the optical property, the acquisition unit further acquires third image processing information regarding the third image processing, and the reprocessing unit cancels the first image processing performed on the image after canceling the third image processing performed on the image. [15] Image processing methods with a first acquisition step of obtaining optical property information regarding an optical property of a lens, a second acquisition step of acquiring a moving image containing images of a plurality of frames acquired via the lens, a processing step of performing image processing on an image of each frame of the moving image based on the optical property information, and an output step of outputting an image after the image processing, the optical property information corresponding to the image, and image processing information corresponding to the image with respect to the image processing in association with each other. [16] Image processing methods with an acquisition step of acquiring an image after a first image processing based on an optical property of a lens, optical property information regarding the optical property, and first image processing information regarding the first image processing for each frame of a moving image, and a reprocessing step of canceling the first image processing performed on the image based on the first image processing information and performing a second image processing on an image after the cancellation based on the optical property information. [17] A computer-readable medium storing a program for causing a computer to operate as each unit of the image processing apparatus according to any one of claims 1 to 14.

Images