Image correction device, image correction method, and image correction program
The image correction device addresses the challenge of varying shooting distances by calculating optical characteristic data and performing precise defocus correction, enhancing image clarity through adaptive techniques.
Patent Information
- Application Number
- JP2023073959
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-04-28
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2043-04-28
AI Technical Summary
Existing image correction techniques face challenges in achieving high accuracy defocus correction, particularly in imaging scenes with varying shooting distances, due to the large amount of two-dimensional data calculation required for wavefront aberration and the need for discrete optical characteristic data calculations.
An image correction device that includes an optical property calculation unit to calculate optical characteristic data based on defocus amount and aberration definition functions, and a defocus correction unit to perform precise defocus correction using this data, allowing for accurate defocus correction even in scenes with varying shooting distances.
Enables highly precise defocus correction of captured images, improving accuracy compared to previous methods by utilizing discrete optical characteristic data calculations and adaptive defocus correction techniques.
Smart Images

Figure 2024158611000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to techniques for correcting images for defocus. [Background technology]
[0002] When an object is photographed by an imaging optical system, the light emitted from one point does not converge to one point but has a small spread due to the effects of aberration and diffraction that occur inside the imaging optical system, defocusing during imaging, etc. This distribution with a small spread is called the PSF. PSF is an abbreviation for Point Spread Function.
[0003] Due to the influence of such an imaging optical system, image degradation based on the PSF shape occurs in the captured image, resulting in a decrease in resolution.
[0004] Techniques have been proposed for correcting such image degradation through image processing. The degradation of the captured image due to defocusing differs depending on the shooting distance from the subject and the image height. Therefore, there is a method to estimate the PSF for each shooting condition and perform image correction by inverse calculation using the PSF and the captured image. [Prior art documents] [Patent documents]
[0005] [Patent Document 1] Patent No. 6537228 Summary of the Invention [Problem to be solved by the invention]
[0006] Patent Document 1 discloses a technology in which second optical characteristic data, which is optical characteristic data at the time of defocus, is generated by adding defocus characteristic data to first optical characteristic data that has been stored in advance, and image correction processing is performed using this second optical characteristic data.
[0007] When the first optical characteristic data, the defocus characteristic data, and the second optical characteristic data are a two-dimensional profile of wavefront aberration, the amount of calculation required for two-dimensional data calculation becomes large because the first optical characteristic data needs to be sufficiently large compared to the optical characteristic data when defocused. In addition, the optical characteristic data when defocused must be calculated discretely according to the shooting distance and image height, which poses a problem in terms of defocus correction accuracy, particularly in imaging scenes with variations in shooting distance.
[0008] There are various imaging scenes in which the shooting distance varies, for example, when a scene with depth is imaged, or when an image is captured from above by pointing the imaging device at a target rather than directly below the imaging device.
[0009] An object of the present disclosure is to make it possible to perform defocus correction on a captured image with high accuracy. [Means for solving the problem]
[0010] The image correction device according to the present disclosure corrects a captured image obtained by photographing a subject using an imaging device. The image correction device includes: an optical characteristic calculation unit that calculates optical characteristic data indicating a calculation result of a point spread function of the optical system based on defocus amount data indicating a defocus amount for the object and an aberration definition function; and a defocus correction unit that performs defocus correction on the captured image using the optical characteristic data. Effect of the Invention
[0011] According to the present disclosure, it is possible to perform defocus correction on a captured image with high accuracy. [Brief description of the drawings]
[0012] [Figure 1]FIG. 1 is a configuration diagram of an image correction device 100 according to a first embodiment. [Diagram 2] FIG. 1 is a functional configuration diagram of an image correction device 100 according to a first embodiment. [Diagram 3] 3 is a flowchart of an image correction method according to the first embodiment. [Figure 4] 11 is a flowchart of step S120 in the first embodiment. [Diagram 5] 1 shows an example of a functional configuration of an image correction device 100 according to a first embodiment. [Figure 6] FIG. 2 is a diagram illustrating the positional relationship between a subject and an optical system according to the first embodiment. [Figure 7] FIG. 11 is a functional configuration diagram of an image correction device 100 according to a second embodiment. [Figure 8] 11 is a flowchart of an image correction method according to the second embodiment. [Figure 9] FIG. 11 is a functional configuration diagram of an image correction device 100 according to a third embodiment. [Figure 10] 13 is a flowchart of an image correction method according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0013] In the embodiments and drawings, the same or corresponding elements are denoted by the same reference numerals. Descriptions of elements denoted by the same reference numerals as those of the elements described above will be omitted or simplified as appropriate. Arrows in the drawings mainly indicate data flow or processing flow.
[0014] Embodiment 1 The image correction device 100 will be described with reference to FIGS.
[0015] ***Configuration Description*** The configuration of an image correction device 100 will be described with reference to FIG. The image correction device 100 is a computer including hardware such as a processor 101, a memory 102, an auxiliary storage device 103, and an input / output interface 104. These pieces of hardware are connected to each other via signal lines.
[0016] The processor 101 is an IC that performs arithmetic processing and controls other hardware. For example, the processor 101 is a CPU. IC is an abbreviation for Integrated Circuit. CPU is an abbreviation for Central Processing Unit.
[0017] The memory 102 is a volatile or non-volatile storage device. The memory 102 is also called a primary storage device or a main memory. For example, the memory 102 is a RAM. Data stored in the memory 102 is saved in the secondary storage device 103 as necessary. RAM is an abbreviation for Random Access Memory.
[0018] The auxiliary storage device 103 is a non-volatile storage device. For example, the auxiliary storage device 103 is a ROM, a HDD, a flash memory, or a combination of these. Data stored in the auxiliary storage device 103 is loaded into the memory 102 as needed. ROM is an abbreviation for Read Only Memory. HDD is an abbreviation for Hard Disk Drive.
[0019] The input / output interface 104 is a port to which an input device and an output device are connected. For example, the input / output interface 104 is a USB terminal, the input device is a keyboard and a mouse, and the output device is a display. Input and output of the image correction device 100 is performed using the input / output interface 104. USB is an abbreviation for Universal Serial Bus.
[0020] The image correction device 100 includes elements such as an optical characteristic calculation unit 111 and a defocus correction unit 112. These elements are realized by software.
[0021] The auxiliary storage device 103 stores an image correction program for causing a computer to function as an optical characteristic calculation unit 111 and a defocus correction unit 112. The image correction program is loaded into the memory 102 and executed by the processor 101. The auxiliary storage device 103 further stores an OS. At least a part of the OS is loaded into the memory 102 and executed by the processor 101. The processor 101 executes an image correction program while executing the OS. OS is an abbreviation for Operating System.
[0022] Input and output data for the image correction program are stored in the storage unit 120 . The auxiliary storage device 103 functions as the storage unit 120. However, a storage device such as the memory 102, a register in the processor 101, or a cache memory in the processor 101 may function as the storage unit 120 instead of the auxiliary storage device 103 or together with the auxiliary storage device 103.
[0023] The image correction program can be recorded (stored) in a computer-readable manner on a non-volatile recording medium such as an optical disk or a flash memory.
[0024] FIG. 2 shows the functional configuration of the image correction device 100. The captured image 131, the corrected image 132, the defocus amount data 121, the aberration definition function 122, and the optical characteristic data 123 will be described later.
[0025] ***Explanation of Operation*** The operation procedure of image correction device 100 corresponds to an image correction method, and also corresponds to a processing procedure according to an image correction program.
[0026] The image correction method will be described with reference to FIG. In step S 110 , the user inputs the captured image 131 and the defocus amount data 121 to the image correction device 100 . The defocus correction unit 112 receives the input captured image 131 . The optical characteristic calculation unit 111 receives the input defocus amount data 121 .
[0027] The captured image 131 is an image obtained by photographing a subject using an imaging device. Unless otherwise specified, the term "imaging device" refers to the imaging device used to obtain the captured image 131. For example, the imaging device is mounted on an air vehicle, and the captured image 131 is obtained by photographing from the air vehicle. Examples of the air vehicle are an aircraft, a drone, or an artificial satellite. A drone is an unmanned aerial vehicle (UAV).
[0028] The defocus amount data 121 is data that indicates the defocus amount for the subject.
[0029] In step S 120 , the optical characteristic calculation unit 111 calculates the optical characteristic data 123 using the defocus amount data 121 and the aberration definition function 122 .
[0030] The optical characteristic data 123 is data that indicates the calculation result of the point spread function of the optical system. Unless otherwise specified, the "optical system" refers to the optical system of the imaging device.
[0031] The procedure of step S120 will be described with reference to FIG. In step S121, the optical characteristic calculation unit 111 uses the defocus amount data 121 to calculate the wavefront aberration W(ξ, η) according to the aberration definition function func. The aberration definition function func is a function that defines the defocus characteristic of the wavefront aberration, and is, for example, a function that models the defocus as the radius of curvature of the wavefront. In another example, it is a function that models the defocus as a wavefront whose radius of curvature varies depending on the direction, that is, a function that also includes astigmatism in the model.
[0032] The wavefront aberration W(ξ, η) is calculated by calculating equation (1). "ε Z" is the amount of defocus. "λ" is the wavelength of each channel image (for example, a red image) included in the captured image. (ξ, η) indicates a position in the pupil plane coordinate system.
[0033]
number
[0034] In step S122, the optical characteristic calculation unit 111 calculates a pupil function P(ξ, η) using the wavefront aberration W(ξ, η).
[0035] The pupil function P(ξ, η) is expressed by the formula (2). "t" is the aperture mask. The aperture mask spatially indicates the area where the aperture of the optical system exists and is defined by 0 and 1. "Exp" means the exponential function.
[0036]
number
[0037] In step S123, the optical characteristic calculation unit 111 calculates the point spread function PSF(x, y) of the optical system using the pupil function P(ξ, η). The point spread function PSF(x, y) of the optical system is expressed by equation (3). (x, y) indicates a position in the image plane coordinate system. "f" is the focal length of the optical system. "F" stands for Fourier transform.
[0038]
number
[0039] The optical characteristic data 123 is data indicating the calculation result of the point spread function PSF(x, y) of the optical system.
[0040] Returning to FIG. 3, the description will continue from step S130. In step S130, the defocus correction unit 112 performs defocus correction on the captured image 131 using the optical characteristic data 123 to generate a corrected image 132. The corrected image 132 is the captured image 131 after defocus correction.
[0041] In the defocus correction, a deconvolution operation is performed using the optical characteristic data 123 as a filter kernel. There are various methods for defocus correction. For example, Wiener Deconvolution is a method to estimate the true image using the PSF, noise components, and the S / N ratio of the true signal components. The S / N ratio means the signal-to-noise ratio.
[0042] In step S140, the defocus correction unit 112 outputs the corrected image 132. For example, the defocus correction unit 112 displays the corrected image 132 on a display.
[0043] ***Addition to the first embodiment*** The defocus amount data 121 will be explained in more detail. The defocus amount for a subject varies depending on the distance at the time of shooting. Therefore, it is desirable to calculate the defocus amount data 121 discretely depending on the distance at the time of shooting. When the defocus amount data 121 is calculated discretely, the optical characteristic data 123 is also calculated discretely depending on the distance at the time of shooting, just like the defocus amount data 121.
[0044] FIG. 5 shows an example of the functional configuration of the image correction device 100. The image correction device 100 may further include an element called a defocus amount calculation unit 119. The image correction program further causes the computer to function as a defocus amount calculation unit 119 .
[0045] The defocus amount calculation unit 119 calculates the defocus amount data 121 . The defocus amount data 121 can be calculated based on the imaging condition data or the captured image 131 .
[0046] For example, the defocus amount data 121 is calculated as follows. A user inputs imaging condition data to the image correction device 100 . The imaging condition data indicates the conditions (imaging conditions) when an image of a subject is captured. The imaging conditions are composed of the focal length of the optical system, the distance between the subject (ground surface) and the sensor of the optical system, and information on the focal plane, or a combination of these. For example, the imaging condition data indicates the orientation of the optical system, the positional relationship between the subject and the optical system, etc. The defocus amount calculation unit 119 receives the input imaging condition data, and calculates the defocus amount data 121 based on the received imaging condition data.
[0047] A method for calculating the defocus amount data 121 will be described with reference to FIG. The solid lines indicate elements when the light source from the subject is not focused on the sensor surface (defocused). The dashed lines indicate elements when the light source from the subject is focused on the sensor surface (focused). Note that there may be areas on the sensor surface where the light source is focused and areas where it is not. Therefore, there may be times when the sensor surface is in focus and when it is defocused at the same time. The object plane is the surface on which the object exists, and the shape of the object plane is known. For example, the object plane is the ground surface. The object distance is the distance between the object and the sensor of the optical system. The defocus condition refers to a case where the position at which the light source image is formed from the subject differs from the sensor surface. First, the defocus amount calculation unit 119 calculates the distance between the subject and the sensor of the optical system and the focal plane of the subject based on the orientation of the optical system and the positional relationship between the subject and the optical system. Then, the defocus amount calculation unit 119 calculates the defocus amount for the subject based on the focal length of the optical system, the distance between the subject and the sensor of the optical system, and the focal plane of the subject. Data indicating the calculated defocus amount is defocus amount data 121.
[0048] For example, the defocus amount calculation unit 119 analyzes the captured image 131 to calculate the defocus amount data 121. The defocus amount data 121 is calculated as follows. First, the defocus amount calculation unit 119 detects the edge of a structure reflected in the captured image 131. The structure may be a subject or an object other than a subject. It is preferable that a subject having a known shape is detected as a structure. Next, the defocus amount calculation unit 119 obtains an LSF from a pixel value profile (image feature amount) around the detected edge. The pixel value profile indicates the distribution of pixel values in the normal direction of the edge. The LSF is a line spread function (LSF). It is an abbreviation for Function. Then, the defocus amount calculation unit 119 calculates the defocus amount for the object using the LSF. The defocus amount is calculated based on the steepness of pixel values. The steepness is calculated using, for example, a Fourier transform. Data indicating the calculated defocus amount is defocus amount data 121.
[0049] ***Advantages of the First Embodiment*** An object of the first embodiment is to perform defocus correction with high accuracy. In particular, an object of the first embodiment is to perform defocus correction with high accuracy in an imaging scene with variation in shooting distance. According to the first embodiment, it is possible to perform defocus correction with high accuracy. The defocus correction according to the first embodiment is more accurate than a method of calculating a PSF using (prestored) optical characteristic data in an imaging scene with variation in shooting distance.
[0050] *** Summary of the first embodiment *** The features of the first embodiment are described below. The reference numerals of elements corresponding to the elements in parentheses are shown in parentheses. The first embodiment relates to information processing capable of performing highly accurate correction for fluctuations in defocus (out-of-focus) during imaging. The information processing device (100) includes an aberration definition function storage unit (120), a PSF calculation unit (111), and a PSF correction unit (112). The aberration definition function storage unit (120) stores an aberration definition function (122). The PSF calculation unit (111) generates second optical characteristic data (123) based on the aberration definition function (122) and the defocus amount (121). The PSF correction unit (112) performs a restoration process on the captured image (131) based on the second optical characteristic data (123).
[0051] Embodiment 2 The manner in which the aberration definition function 122 is calculated will be described below with reference to FIGS. 7 and 8, mainly with respect to the points that differ from the first embodiment.
[0052] ***Configuration Description*** The configuration of the image correction device 100 will be described with reference to FIG. The image correction device 100 further includes elements such as a sensor data calculation unit 113 and an aberration definition function calculation unit 114. The image correction program further causes the computer to function as a sensor data calculation unit 113 and an aberration definition function calculation unit 114 .
[0053] ***Explanation of Operation*** The image correction method will be described with reference to FIG. In step S 210 , the user inputs the captured image 131 , the defocus amount data 121 and the reference image 133 to the image correction device 100 . The sensor data calculation unit 113 receives the input reference image 133 .
[0054] The reference image 133 is an image obtained by photographing using an imaging device. For example, the reference image 133 is an image in which a light source image is captured, and is obtained by photographing a point light source using an imaging device. The imaging equipment used is the same model as the imaging equipment used to obtain the captured image 131. For example, the imaging equipment used to obtain the captured image 131 is used to obtain the reference image 133.
[0055] In step S 220 , the sensor data calculation unit 113 calculates the sensor data 124 based on the reference image 133 . The sensor data 124 represents the point spread function of the optical system and the wavefront aberration of the optical system.
[0056] First, the sensor data calculation unit 113 calculates a point spread function PSF of the optical system based on a point light source image (reference image 133) given for calibration. Then, the sensor data calculation unit 113 calculates the wavefront aberration W(ξ, η) of the optical system based on the relationship of equations (2), (3), and (4) of the first embodiment using the point spread function PSF of the optical system.
[0057] In step S230, the aberration definition function calculation unit 114 calculates the aberration definition function 122 based on the sensor data .
[0058] Steps S240 to S260 are the same as steps S120 to S140 in the first embodiment.
[0059] ***Effects of the second embodiment*** According to the second embodiment, it is possible to maintain the accuracy of defocus correction by using the sensor data of the actual measurement value corresponding to the aging deterioration and misalignment of the optical system. The misalignment occurs, for example, when a large impact is applied to the optical system.
[0060] ***Additional information to the second embodiment*** The image correction device 100 may include a defocus amount calculation section 119 to calculate the defocus amount data 121 .
[0061] Embodiment 3 A mode for obtaining an optimal defocus amount will be described below, mainly with respect to the differences from the first embodiment, with reference to FIGS.
[0062] ***Configuration Description*** The configuration of the image correction device 100 will be described with reference to FIG. The image correction device 100 further includes elements such as a cost calculation unit 115 and a defocus amount update unit 116. The image correction program further causes the computer to function as a cost calculation unit 115 and a defocus amount update unit 116.
[0063] ***Explanation of Operation*** The image correction method will be described with reference to FIG. In step S 310 , the captured image 131 and the defocus amount data 121 are input to the image correction device 100 . Step S310 is the same as step S110 in the first embodiment.
[0064] In step S320 , the optical characteristic calculation unit 111 calculates the optical characteristic data 123 using the defocus amount data 121 and the aberration definition function 122 . Step S320 is the same as step S120 in the first embodiment.
[0065] In step S330, the defocus correction unit 112 performs defocus correction on the captured image 131 using the optical characteristic data 123 to generate a provisional image . The provisional image 134 is the captured image 131 after defocus correction. The method for performing the defocus correction is the same as the method in step S130 in the first embodiment.
[0066] In step S340 , the cost calculation unit 115 calculates the cost data 125 using the provisional image 134 , the captured image 131 , and the optical characteristic data 123 . The cost data 125 indicates a cost that represents the degree of mismatch of the optical characteristic data 123 with respect to the captured image 131 .
[0067] For example, the cost data 125 is calculated as follows: First, the cost calculation unit 115 performs a blurring operation on the provisional image 134 using the optical characteristic data 123. The blurring operation is an operation of convolving the optical characteristic data 123 with the provisional image 134. In other words, the cost calculation unit 115 defocuses the provisional image 134 using the optical characteristic data 123. Then, the cost calculation unit 115 compares the provisional image 134 after the blur calculation with the captured image 131, and calculates the amount of difference between the captured image 131 and the provisional image 134 after the blur calculation. For example, the amount of difference is a simple difference between pixel values. Data indicating the calculated amount of difference as a cost becomes the cost data 125.
[0068] The cost data 125 may be calculated as follows. The cost calculation unit 115 divides each of the captured image 131 and the provisional image 134 after the blur calculation into a plurality of image regions, and calculates the cost for each defocus amount for each image region. Data indicating the calculated cost (cost volume) becomes cost data 125. By obtaining the cost volume, it is possible to estimate the defocus amount that minimizes the cost (optimum defocus amount).
[0069] In step S350, the defocus amount update unit 116 determines whether to update the defocus amount data 121. If the optimal defocus amount is not obtained, the defocus amount data 121 is updated.
[0070] For example, the defocus amount update unit 116 determines whether the cost indicated in the cost data 125 satisfies a cost condition. If the cost condition is not satisfied, the defocus amount data 121 is updated.
[0071] For example, the defocus amount update unit 116 compares the cost indicated in the cost data 125 with a threshold value. If the cost is greater than the threshold value, the defocus amount data 121 is updated.
[0072] If the defocus amount data 121 is to be updated, the process proceeds to step S360.
[0073] In step S 360 , the defocus amount update unit 116 updates the defocus amount data 121 based on the cost data 125 . The defocus amount data 121 is updated by an optimization method so as to reduce the cost. The updated defocus amount data 121 is referred to as defocus amount data 126 .
[0074] After step S360, the process proceeds to step S320. In step S 320 , the optical characteristic calculation unit 111 calculates new optical characteristic data 123 using the defocus amount data 126 and the aberration definition function 122 . In step S330, the defocus correction unit 112 performs defocus correction on the captured image 131 using the new optical characteristic data 123 to generate a new tentative image . In step S340 , the cost calculation unit 115 calculates new cost data 125 using the new tentative image 134 , the captured image 131 , and the optical characteristic data 123 . In step S350, the defocus amount update unit 116 determines whether to update the defocus amount data 121. If the defocus amount data 121 is not to be updated, the process proceeds to step S370.
[0075] In step S370, the defocus correction unit 112 outputs the latest tentative image 134 as the corrected image 132.
[0076] ***Effects of the Third Embodiment*** In the third embodiment, the process of the cost calculation unit 115 and the process of the defocus amount update unit 116 are repeated to calculate an optimal defocus amount for defocus correction. According to the third embodiment, it becomes possible to estimate the optimum defocus amount with fine granularity on the axis of the defocus amount, so that more accurate defocus correction becomes possible.
[0077] ***Additional Note to the Third Embodiment*** The image correction device 100 may include a defocus amount calculation section 119 to calculate the defocus amount data 121 .
[0078] The third embodiment may be implemented in combination with the second embodiment. That is, the image correction device 100 may include a sensor data calculation unit 113 and an aberration definition function calculation unit 114 to calculate the aberration definition function 122.
[0079] ***Supplementary explanation of implementation form*** Each embodiment is an example of a preferred embodiment, and is not intended to limit the technical scope of the present disclosure. Each embodiment may be implemented in part or in combination with other embodiments. The procedures described using flow charts, etc. may be modified as appropriate.
[0080] Each element of the image correction device 100 may be implemented in software, hardware, firmware, or a combination of these. The "unit" of each element of image correction device 100 may be interpreted as a "process," a "step," a "circuit," or a "circuitry."
[0081] Various aspects of the present disclosure are described below as appendices. (Appendix 1) An image correction device that corrects a captured image obtained by photographing a subject using an imaging device, an optical characteristic calculation unit that calculates optical characteristic data indicating a calculation result of a point spread function of an optical system of the imaging device based on defocus amount data indicating a defocus amount for the object and an aberration definition function; a defocus correction unit that performs defocus correction on the captured image by using the optical characteristic data; An image correction device comprising:
[0082] (Appendix 2) a sensor data calculation unit that calculates sensor data indicating a point spread function of the optical system and a wavefront aberration of the optical system based on a reference image obtained by imaging using the imaging device; an aberration definition function calculation unit that calculates the aberration definition function based on the sensor data; Equipped 2. An image correction device as described in claim 1.
[0083] (Appendix 3) The image correction device includes a cost calculation unit and a defocus amount update unit, the defocus correction unit performs the defocus correction on the captured image using the optical characteristic data to generate a provisional image; the cost calculation unit calculates cost data indicating a cost representing a degree of mismatch of the optical characteristic data with respect to the captured image, using the provisional image, the captured image, and the optical characteristic data; the defocus amount update unit updates the defocus amount data based on the cost data; the optical characteristic calculation unit calculates new optical characteristic data based on the updated defocus amount data and the aberration definition function; The defocus correction unit performs the defocus correction on the captured image using the new optical characteristic data. 3. An image correction device according to claim 1 or 2.
[0084] (Appendix 4) The aberration definition function is a function that defines the defocus characteristic of the wavefront aberration. 4. An image correction device according to claim 1.
[0085] (Appendix 5) A defocus amount calculation unit that analyzes the captured image and calculates the defocus amount Equipped 5. An image correction device according to claim 1.
[0086] (Appendix 6) a defocus amount calculation unit that calculates the defocus amount based on imaging condition data indicating imaging conditions of the object; Equipped 5. An image correction device according to claim 1.
[0087] (Appendix 7) The imaging device is mounted on an air vehicle, The captured image is an image obtained by photographing from the flying object. 7. An image correction device according to any one of claims 1 to 6.
[0088] (Appendix 8) An image correction method for correcting an image obtained by photographing a subject using an imaging device, calculating optical characteristic data indicating a calculation result of a point spread function of an optical system of the imaging device based on defocus amount data indicating a defocus amount for the object and an aberration definition function; The optical characteristic data is used to perform defocus correction on the captured image. Image correction methods.
[0089] (Appendix 9) An image correction program for correcting a captured image obtained by photographing a subject using an imaging device, an optical characteristic calculation process for calculating optical characteristic data indicating a calculation result of a point spread function of an optical system of the imaging device based on defocus amount data indicating a defocus amount for the object and an aberration definition function; a defocus correction process for performing defocus correction on the captured image using the optical characteristic data; An image correction program for causing a computer to execute the above. [Explanation of symbols]
[0090] 100 image correction device, 101 processor, 102 memory, 103 auxiliary storage device, 104 input / output interface, 111 optical characteristic calculation unit, 112 defocus correction unit, 113 sensor data calculation unit, 114 aberration definition function calculation unit, 115 cost calculation unit, 116 defocus amount update unit, 119 defocus amount calculation unit, 120 storage unit, 121 defocus amount data, 122 aberration definition function, 123 optical characteristic data, 124 sensor data, 125 cost data, 126 defocus amount data, 131 captured image, 132 corrected image, 133 reference image, 134 provisional image.
Claims
1. An image correction device that corrects a captured image obtained by photographing a subject using an imaging device, an optical characteristic calculation unit that calculates optical characteristic data indicating a calculation result of a point spread function of an optical system of the imaging device based on defocus amount data indicating a defocus amount for the object and an aberration definition function; a defocus correction unit that performs defocus correction on the captured image by using the optical characteristic data; An image correction device comprising:
2. a sensor data calculation unit that calculates sensor data indicating a point spread function of the optical system and a wavefront aberration of the optical system based on a reference image obtained by imaging using the imaging device; an aberration definition function calculation unit that calculates the aberration definition function based on the sensor data; Equipped 2. The image correction device according to claim 1.
3. The image correction device includes a cost calculation unit and a defocus amount update unit, the defocus correction unit performs the defocus correction on the captured image using the optical characteristic data to generate a provisional image; the cost calculation unit calculates cost data indicating a cost representing a degree of mismatch of the optical characteristic data with respect to the captured image, using the provisional image, the captured image, and the optical characteristic data; the defocus amount update unit updates the defocus amount data based on the cost data; the optical characteristic calculation unit calculates new optical characteristic data based on the updated defocus amount data and the aberration definition function; The defocus correction unit performs the defocus correction on the captured image using the new optical characteristic data.
2. The image correction device according to claim 1.
4. The aberration definition function is a function that defines the defocus characteristic of the wavefront aberration.
2. The image correction device according to claim 1.
5. A defocus amount calculation unit that analyzes the captured image and calculates the defocus amount Equipped 2. The image correction device according to claim 1.
6. a defocus amount calculation unit that calculates the defocus amount based on imaging condition data indicating imaging conditions of the object; Equipped 2. The image correction device according to claim 1.
7. The imaging device is mounted on an air vehicle, The captured image is an image obtained by photographing from the flying object. The image correction device according to any one of claims 1 to 6.
8. An image correction method for correcting an image obtained by photographing a subject using an imaging device, calculating optical characteristic data indicating a calculation result of a point spread function of an optical system of the imaging device based on defocus amount data indicating a defocus amount for the object and an aberration definition function; The optical characteristic data is used to perform defocus correction on the captured image. Image correction methods.
9. An image correction program for correcting a captured image obtained by photographing a subject using an imaging device, an optical characteristic calculation process for calculating optical characteristic data indicating a calculation result of a point spread function of an optical system of the imaging device based on defocus amount data indicating a defocus amount for the object and an aberration definition function; a defocus correction process for performing defocus correction on the captured image using the optical characteristic data; An image correction program for causing a computer to execute the above.
Citation Information
Patent Citations
Large depth-of-field imaging method and system based on multipoint spread function generative adversarial network
CN114612362A
A system and method for minimizing the effects of aberrations in an imaging system.
JP2006523330A
Imaging apparatus, image processing arithmetic unit, and image processing program
JP2016192620A
Light Microscopy and Microscopy
JP2022507849A
Image processing device, imaging device, image processing method, image processing program, and storage medium
JP6537228B2
