Image Correction Device
The image correction device addresses color shifts in multi-band imaging by aligning pixel values across band images, effectively reducing phase-related distortions.
Patent Information
- Application Number
- JP2020172103
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2020-10-12
- Publication Date
- 2024-11-13
- Estimated Expiration
- 2040-10-12
AI Technical Summary
Existing image acquisition devices using one-dimensional array sensors suffer from color shifts due to phase differences between bands, which are difficult to correct using conventional band image shifting and linear interpolation methods.
An image correction device that determines positional differences between band images and creates a corrected band image by retaining pixel values based on the relationship between overlapping pixels, using one band image as a reference and others as targets, to minimize color shifts.
The device effectively reduces color shifts by aligning pixel values across band images, ensuring accurate color representation without distortions.
Smart Images

Figure 0007581758000001 
Figure 0007581758000002 
Figure 0007581758000003
Abstract
Description
[Technical field]
[0001] The present invention relates to an image correction device, an image correction method, and a program. [Background technology]
[0002] Push-broom type image acquisition devices are widely used as devices for acquiring images of the earth's surface from aircraft or satellites. In this type of device, a one-dimensional array sensor is used as an image sensor, and a linear image extending in the X-axis direction is acquired. Then, a two-dimensional image is formed by translating the entire image acquisition device in a direction perpendicular to the linear acquired image (Y-axis direction) by the movement of the aircraft or satellite. When acquiring images of multiple wavelength bands using this type of image acquisition device, each of the multiple one-dimensional array sensors is configured to be equipped with multiple filters with different bands, which are the wavelength bands of light to be transmitted, to capture an image of the subject. Images of each wavelength band are called band images.
[0003] Furthermore, for example, Patent Document 1 discloses a technique for reducing color shifts that occur in this type of image acquisition device by combining a shift of a band image according to the amount of positional shift with a normal interpolation process such as linear interpolation. [Prior art documents] [Patent documents]
[0004] [Patent Document 1] Patent No. 6305328 Summary of the Invention [Problem to be solved by the invention]
[0005] However, when distortion occurs due to the characteristics of the optical system, a phase difference occurs between bands, resulting in color shift due to the phase difference. Color shift due to phase difference here refers to the occurrence of different colors from the object when the same part of the object is captured in multiple bands such as RGB (red, green, blue) because the same part of the object is located in different positions within the pixels of each band. It has been difficult to reduce color shift due to this type of phase difference by combining a shift in the band images according to the amount of positional shift when correcting the color shift with normal interpolation processing such as linear interpolation.
[0006] An object of the present invention is to provide an image correction device that solves the above-mentioned problem, namely, that it is difficult to reduce color shift due to phase difference by combining shifting of band images with conventional interpolation processing such as linear interpolation. [Means for solving the problem]
[0007] An image correction device according to one aspect of the present invention includes: a band image acquisition means for acquiring a plurality of band images obtained by imaging a subject; a position difference acquisition means for acquiring a position difference between the target band image and the reference band image, the target band image being an object band image and the reference band image being an object band image; a correction band image creation means for creating a correction band image that holds pixel values of light related to the target band image at pixel positions of the reference band image based on a pixel value relationship between the pixel values of the corresponding pixels of the reference band image, the pixel values of the pixel values of the corresponding pixels of the reference band image being corresponding pixels of the pixel values of the corresponding pixels of the target band image being corresponding pixels of the pixel values of the corresponding pixels of the reference band image, the correction band image being corresponding pixels of the pixel values of the corresponding pixels of the target band image being corresponding pixels of the pixel values of the corresponding pixels of the reference band image being corresponding pixels of the pixel values of the ... a correction band image output means for outputting the correction band image; The device is configured to include:
[0008] Further, an image correction method according to another aspect of the present invention includes: A plurality of band images are obtained by imaging the subject, one of the plurality of band images is a reference band image, and at least one of the remaining band images is a target band image; and a position difference between the target band image and the reference band image is obtained; a pixel of the target band image is set as a target pixel, and a plurality of pixels of the reference band image that overlap with the target pixel when the target pixel is shifted by the position difference are set as corresponding pixels, and a correction band image is created that holds pixel values of light related to the target band image at pixel positions of the reference band image based on a pixel value relationship between the plurality of corresponding pixels; outputting the correction band image; It is structured as follows.
[0009] In addition, a program according to another aspect of the present invention includes: On the computer, A process of acquiring a plurality of band images obtained by capturing an image of a subject; a process of determining one of the plurality of band images as a reference band image and at least one of the remaining band images as a target band image, and acquiring a position difference between the target band image and the reference band image; a process of setting a pixel of the target band image as a target pixel, setting each of a plurality of pixels of the reference band image that overlap with the target pixel when the target pixel is shifted by the position difference as a corresponding pixel, and creating a correction band image that holds pixel values of light related to the target band image at pixel positions of the reference band image based on the pixel value relationships between the plurality of corresponding pixels; outputting the correction band image; The present invention is configured to cause the Effect of the Invention
[0010] The present invention, having the above-mentioned configuration, can reduce color shift caused by phase difference. [Brief description of the drawings]
[0011] [Figure 1] 1 is a block diagram of an image correction device according to a first embodiment of the present invention. [Diagram 2] 1 is a schematic diagram for explaining the phenomenon in which the same part of a subject is mapped to different locations within the pixels of each band in a push-broom type image acquisition device. [Diagram 3] 11A and 11B are diagrams illustrating pixel position differences of target band images. [Figure 4] 13 is a diagram showing an example of the configuration of position difference information of an R band image. FIG. [Diagram 5] 5 is a flowchart showing an example of the operation of the image correction device according to the first embodiment of the present invention. [Figure 6] 2 is a block diagram showing an example of a position difference acquisition unit in the image correction device according to the first embodiment of the present invention. FIG. [Figure 7] 5 is a flowchart showing an example of processing by a position difference information creation unit in the image correction device according to the first embodiment of the present invention. [Figure 8] 4 is a diagram showing an example in which a position difference information generating unit in the image correction device according to the first embodiment of the present invention divides each image of a reference band and a target band into small regions. FIG. [Figure 9] 13 is a flowchart showing another example of the process of the position difference information creation unit in the image correction device according to the first embodiment of the present invention. [Figure 10] 11 is a diagram showing an example in which a position difference information creation unit in the image correction device according to the first embodiment of the present invention divides the images of the reference band and the target band so as to leave gaps between small regions. FIG. [Figure 11] 13 is a block diagram showing another example of the position difference acquisition unit in the image correction device according to the first embodiment of the present invention. FIG. [Figure 12] 5 is a flowchart showing an example of processing by a corrected multiband image creation unit in the image correction device according to the first embodiment of the present invention. [Figure 13] 1 is a schematic diagram showing an overlap state between a pixel (target pixel) of a target band image and a pixel (corresponding pixel) of a reference band image. FIG. [Figure 14A] FIG. 2 is a diagram showing mathematical expressions used in the image correction device according to the first embodiment of the present invention. [Figure 14B]FIG. 2 is a diagram showing mathematical expressions used in the image correction device according to the first embodiment of the present invention. [Figure 15] FIG. 2 is a diagram showing a rectangular white board that is a subject. [Figure 16] FIG. 13 is a diagram showing an example of pixel values of each pixel in a G-band image obtained by capturing an image of a white board. [Figure 17] FIG. 13 is a diagram showing an example of pixel values of each pixel in an R band image obtained by capturing an image of a white board. [Figure 18] FIG. 13 is a diagram showing an example of an overlap state between an R band image and a G band image when an R band pixel is shifted by a position difference (0.5, 0.5). [Figure 19] FIG. 13 is a diagram showing an example of pixel values of each pixel in a corrected R band image. [Figure 20] 13 is a flowchart showing another example of processing by the corrected multiband image creation unit configured to use a ratio of pixel values as the relationship between pixel values. [Figure 21] 13 is a flowchart showing another example of processing by the corrected multiband image creation unit configured to use a difference in pixel values as the relationship between pixel values. [Figure 22] FIG. 11 is a block diagram of an image correction device according to a second embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0012] Next, an embodiment of the present invention will be described in detail with reference to the drawings. [First embodiment] Referring to FIG. 1, an image correction device 10 according to a first embodiment of the present invention is composed of a plurality of one-dimensional array sensors 11, a communication interface (I / F) unit 12, an operation input unit 13, a screen display unit 14, a memory unit 15, and an arithmetic processing unit 16.
[0013] The multiple one-dimensional array sensors 11 are, for example, one-dimensional CCD (Charge-Coupled Device) sensors or one-dimensional CMOS (Complementary MOS) sensors, and constitute a push-broom type image acquisition device that captures an image of a subject 18. The multiple one-dimensional array sensors 11 are fitted with multiple filters 19 that have different bands, which are the wavelength bands of light that are transmitted. The number of band images and wavelength bands to be acquired are determined by the combination and number of sets of the one-dimensional array sensors 11 and filters 19 used. For example, the multi-band sensor mounted on the high-resolution optical satellite ASNARO-1 acquires the following six-band images: Band 1: Wavelength band 400-450nm (Ocean Blue) Band 2: Wavelength range 450-520nm (Blue) Band 3: Wavelength band 520-600nm (Green) Band 4: Wavelength band 630-690nm (Red) Band 5: Wavelength band 705-745nm (Red Edge) Band 6: Wavelength band 760-860nm (NIR)
[0014] The communication I / F unit 12 is composed of, for example, a dedicated data communication circuit, and is configured to perform data communication with various devices connected via wire or wirelessly. The operation input unit 13 is composed of operation input devices such as a keyboard and a mouse, and is configured to detect an operation by an operator and output the operation to the calculation processing unit 16. The screen display unit 14 is composed of a screen display device such as an LCD (Liquid Crystal Display) or a PDP (Plasma Display Panel), and is configured to display a correction band image, etc. on the screen in response to an instruction from the calculation processing unit 16.
[0015] The storage unit 15 is composed of storage devices such as a hard disk and a memory, and is configured to store processing information and a program 151 required for various processes in the arithmetic processing unit 16. The program 151 is a program that realizes various processing units by being read into the arithmetic processing unit 16 and executed. The program 151 is read in advance from an external device (not shown) or a storage medium (not shown) via a data input / output function such as the communication I / F unit 12, and is stored in the storage unit 15.
[0016] The main processing information stored in the storage unit 15 includes a multiband image 152 , position difference information 153 , and a corrected multiband image 154 .
[0017] The multiband image 152 is a set of multiple band images acquired by a push-broom type image acquisition device. The multiband image 152 may be a set of all band images acquired by the push-broom type image acquisition device, or may be a set of some of the band images. In this embodiment, the multiband image 152 is composed of three bands, an R band image 152-1, a G band image 152-2, and a B band image 152-3. For example, in the case of the above-mentioned ASNARO-1, band 4 may be assigned as the R band image 152-1, band 3 as the G band image 152-2, and band 2 as the B band image 152-3.
[0018] The position difference information 153 is information regarding the position difference between a reference band image and a target band image when one of the multiple band images constituting the multiband image 152 is a reference band image and the remaining are target band images. In this embodiment, the G band image 152-2 is the reference band image, and the R band image 152-1 and the B band image 152-3 are each the target band images. Therefore, the position difference information 153 is composed of position difference information 153-1 that records the position difference of the R band image 152-1 relative to the G band image 152-2, and position difference information 153-2 that records the position difference of the B band image 152-3 relative to the G band image 152-2.
[0019] When distortion occurs due to the characteristics of the optical system in a push-broom type image acquisition device, a phenomenon occurs in which the same part of the subject 18 is mapped to different places in the pixels of each band when the subject 18 is imaged. FIG. 2 is a schematic diagram for explaining such a phenomenon. In FIG. 2, the solid rectangle indicates the range imaged by the pixel (x, y) of the reference band image, the dashed rectangle indicates the range imaged by the pixel (x, y) of the target band image, and the black circle indicates a part of the subject 18. In the example shown in FIG. 2, the black circle, which is a part of the subject 18, is located almost in the center in the pixel (x, y) of the reference band, whereas it is located in the lower right in the pixel (x, y) of the target band. In this case, as shown in FIG. 3, if the pixel (x, y) of the target band is moved by s pixels (0≦s<1) to the right of the paper and by t pixels (0≦t<1) downward of the paper, the range imaged by the pixel (x, y) of the reference band image and the pixel (x, y) of the target band image will match. When the imaging ranges of multiple band images match in this way, it is said that the pixel boundaries of the multiple band images match. In addition, (s, t) in this case is called the position difference. The position difference may differ for each target band and pixel. In this embodiment, the position difference information 153 is recorded for each target band and pixel.
[0020] FIG. 4 is a diagram showing a configuration example of position difference information 153-1 of R band image 152-1. In this example, position difference information 153-1 is composed of items of target band ID and information for each pixel. Identification information that uniquely identifies the target band ID is recorded in the item of target band ID. The item of information for each pixel is composed of a combination of an item of pixel position of the target band image and an item of position difference at that pixel position. In the item of pixel position, xy coordinate values (x, y) that identify the position of the pixel on the target band image are recorded. In the item of position difference, (s, t) described with reference to FIG. 3 is recorded. Although not shown, position difference information 153-2 of B band image 152-3 also has a configuration similar to that of position difference information 153-1.
[0021] The corrected multiband image 154 is a multiband image obtained by performing correction so as to prevent color shift from occurring on the multiband image 152. The corrected multiband image 154 is composed of a corrected R band image 154-1, a corrected G band image 154-2, and a corrected B band image 154-3.
[0022] The arithmetic processing unit 16 has a microprocessor such as an MPU and its peripheral circuits, and is configured to read and execute a program 151 from the storage unit 15, thereby realizing various processing units by causing the above hardware and the program 151 to cooperate with each other. Main processing units realized by the arithmetic processing unit 16 include a multiband image acquisition unit 161, a position difference acquisition unit 162, a corrected multiband image creation unit 163, and a corrected multiband image output unit 164.
[0023] The multiband image acquisition unit 161 is configured to acquire the multiband image 152 from a push-broom type image acquisition device configured with the one-dimensional array sensor 11, and record it in the storage unit 15. However, the multiband image acquisition unit 161 is not limited to a configuration in which it acquires the multiband image 152 from the image acquisition device. For example, in a case in which the multiband image 152 acquired from the image acquisition device is stored in an image server device (not shown), the multiband image acquisition unit 161 may be configured to acquire the multiband image 152 from the image server device.
[0024] The position difference acquisition unit 162 is configured to acquire position difference information 153 of the multiband image 152 acquired by the multiband image acquisition unit 161 , and record it in the storage unit 15 .
[0025] The corrected multiband image creation unit 163 is configured to read the multiband image 152 and the position difference information 153 from the memory unit 15, create a corrected multiband image 154 from the multiband image 152 and the position difference information 153, and record it in the memory unit 15.
[0026] The corrected multiband image output unit 164 is configured to read out the corrected multiband image 154 from the storage unit 15, and display the corrected multiband image 154 on the screen display unit 14 and / or output the corrected multiband image 154 to an external device via the communication I / F unit 12. The corrected multiband image output unit 164 may display or output the individual corrected R band image 154-1, corrected G band image 154-2, and corrected B band image 154-3 constituting the corrected multiband image 154 individually, or may display a color image obtained by combining the corrected R band image 154-1, the corrected G band image 154-2, and the corrected B band image 154-3 on the screen display unit 14 and / or output the color image to an external device via the communication I / F unit 12.
[0027] FIG. 5 is a flowchart showing an example of the operation of the image correction device 10 according to the present embodiment. Referring to FIG. 5, first, the multiband image acquisition unit 161 acquires the multiband image 152 captured by the image acquisition device composed of the one-dimensional array sensor 11, and records it in the storage unit 15 (step S1). Next, the position difference acquisition unit 162 acquires the position difference information 153 of the multiband image 152 acquired by the multiband image acquisition unit 161, and records it in the storage unit 15 (step S2). Next, the corrected multiband image creation unit 163 reads out the multiband image 152 and the position difference information 153 from the storage unit 15, creates a corrected multiband image 154 based on them, and records it in the storage unit 15 (step S3). Next, the corrected multiband image output unit 164 reads out the corrected multiband image 154 from the storage unit 15, displays it on the screen display unit 14, and / or outputs it to an external device via the communication I / F unit 12 (step S4).
[0028] Next, a detailed description will be given of the main components of the image correction device 10. First, a detailed description will be given of the position difference acquisition unit 162.
[0029] 6 is a block diagram showing an example of the position difference acquiring unit 162. The position difference acquiring unit 162 in this example is configured to include a position difference information generating unit 1621.
[0030] The position difference information creation unit 1621 is configured to read out the multiband image 152 from the memory unit 15, create R band position difference information 153-1 from the G band image 152-2 and the R band image 152-1, and create B band position difference information 153-2 from the G band image 152-2 and the B band image 152-3.
[0031] Fig. 7 is a flowchart showing an example of the processing of the position difference information creating unit 1621. First, the position difference information creating unit 1621 divides each image of the reference band and the target band into a plurality of small regions having a predetermined shape and size, for example as shown in Fig. 8 (step S11). In Fig. 8, the small regions are rectangular, but they may be of a shape other than rectangular. It is desirable to set the size of the small regions sufficiently larger than one pixel.
[0032] Next, the position difference information creation unit 1621 focuses on one of the target bands (e.g., the R band) (step S12). Next, the position difference information creation unit 1621 initializes the position difference information of the target band being focused on (step S13). For example, if the position difference information of the target band has the format shown in Fig. 4, the position difference (s, t) of each pixel position (x, y) is initialized to, for example, a NULL value.
[0033] Next, the position difference information creation unit 1621 focuses on one small area of the target band under consideration (step S14). Next, the position difference information creation unit 1621 focuses on one small area of the reference band that corresponds to the small area of the target band under consideration (step S15). In this embodiment, a situation is assumed in which the position difference of the target band is one pixel or less. Therefore, one small area of the reference band that corresponds to the small area of the target band under consideration is a small area at the same position as the small area of the target band. In other words, if the small area of the target band under consideration is the small area in the upper left corner of FIG. 8, the small area of interest of the reference band is also the small area in the upper left corner of FIG. 8.
[0034] Next, the position difference information creation unit 1621 calculates the shift amount (s, t) by which the small area of the target band under consideration most closely matches the small area of the reference band under consideration (step S16). For example, if the image of the small area of the target band under consideration is shifted by, for example, 0.2 pixels in the X-axis direction and 0.7 pixels in the Y-axis direction, the shift amount in the X-axis direction s = 0.2 pixels and the shift amount in the Y-axis direction t = 0.7 pixels will be the desired shift amount. Such a shift amount may be calculated using a subpixel matching method that can calculate the shift amount with an accuracy of less than one pixel, such as a phase-only correlation method or an SSD parabolic fitting method. Next, the position difference information creation unit 1621 updates the position difference information of the target band under consideration by setting the calculated shift amount (s, t) as the position difference (s, t) of all pixels included in the small area of the target band under consideration (step S17).
[0035] Next, the position difference information creating unit 1621 shifts its attention to another small area of the currently focused target band (step S18), returns to the process of step S15, and executes the same process as the process described above for the newly focused small area of the target band. Then, when the position difference information creating unit 1621 finishes focusing on all small areas of the currently focused target band (YES in step S19), it shifts its attention to one of the other target bands (e.g., band B) (step S20), returns to the process of step S13, and executes the same process as the process described above for the newly focused target band. Then, when it finishes focusing on all target bands (YES in step S21), it saves the created position difference information of each target band in the storage unit 15 (step S22). Then, the position difference information creating unit 1621 ends the process shown in FIG. 7.
[0036] FIG. 9 is a flowchart showing another example of the process of the position difference information creation unit 1621. The process shown in FIG. 9 is different from the process shown in FIG. 7 in that step S17 is replaced with step S17A and a new step S23 is provided between step S19 and step S20, and the rest is the same as the process shown in FIG. 7. In step S17A of FIG. 9, the position difference information creation unit 1621 updates the position difference information of the target band of interest by using the shift amount (s, t) calculated in step S16 as the position difference of the pixel at the center position of the small area of the target band of interest. Therefore, in step S17A of FIG. 9, the position differences of pixels other than the pixel at the center position of the small area of interest are not updated and remain as the initial NULL value. In step S23 of FIG. 9, the position difference information creation unit 1621 calculates the position differences of pixels other than the center positions of each small area of the target band of interest by interpolation from the position differences of the pixels at the center positions of each small area calculated in step S17A, and updates the position difference information of the target band of interest. The interpolation method may be, for example, interpolation by weighted averaging according to the distance to the center of the adjacent small region.
[0037] When the shift amount calculated for each small region is used as the position difference of all pixels in the small region as shown in Fig. 7, the position difference may become discontinuous at the boundaries of the small regions. On the other hand, in the method shown in Fig. 9, the position difference changes continuously according to the pixel position, and the position difference does not become discontinuous. As a result, the method shown in Fig. 9 has the effect of eliminating the occurrence of step differences in color in the corrected image at the boundaries of the small regions.
[0038] 7 and 9, the entire image of the reference band and the target band are divided into small regions without exception, but as shown in FIG. 10, the position difference information creation unit 1621 may divide the images so that there are gaps between the small regions (hatched parts in the figure). The position difference information creation unit 1621 may then obtain the position difference of the pixels included in the gaps by interpolation from the position difference of the pixel at the center position calculated in step S17 or step S17A. The interpolation at this time may be, for example, an interpolation by weighted averaging according to the distance from the center of the adjacent small region. According to the method of dividing the reference band and the target band into a plurality of small regions so that there are gaps between the small regions and creating the position difference, the calculation time can be reduced compared to the case of dividing the images so that there are no gaps.
[0039] In the method shown in FIG. 7 and FIG. 9, the position difference information creating unit 1621 creates the position difference of each pixel of the target band image from the multiband image 152. However, the position difference information creating unit 1621 may create the position difference of each pixel of the target band image from the position and attitude information of the platform (such as an artificial satellite or an aircraft) that acquired the multiband image. In general, images captured from an artificial satellite or an aircraft are processed into an image product by projecting the images onto a map using the position and attitude of the platform at the time of image acquisition. In the case of a multispectral image, since the images are projected onto a map for each band, the position difference information is also obtained during the map projection process. Therefore, the position difference information creating unit 1621 may create the position difference information of each pixel of the target band image using a general map projection method.
[0040] 11 is a block diagram showing another example of the position difference acquiring unit 162. The position difference acquiring unit 162 in this example includes a position difference information input unit 1622.
[0041] The position difference information input unit 1622 is configured to input position difference information 153 from an external device (not shown) through the communication I / F unit 12, and store it in the storage unit 15. Alternatively, the position difference information input unit 1622 is configured to input position difference information 153 from an operator of the image correction device 10 through the operation input unit 13, and store it in the storage unit 15. In other words, the position difference information input unit 1622 is configured to input position difference information 153 of the multiband image 152 calculated by a device other than the image correction device 10, and store it in the storage unit 15.
[0042] As described above, the position difference acquisition unit 162 is configured to generate the position difference information 153 of the multiband image 152 by itself or to input it from an external source and store it in the storage unit 15 .
[0043] Next, the corrected multiband image generating unit 163 will be described in detail.
[0044] Fig. 12 is a flowchart showing an example of processing by the corrected multiband image creation unit 163. Referring to Fig. 12, the corrected multiband image creation unit 163 first focuses on one of the target bands (e.g., the R band) (step S31). Next, the corrected multiband image creation unit 163 initializes pixel values of the correction target band image of the target band being focused on to 0 (step S32). The correction target band image of the target band is an image that holds pixel values of the color of the target band at each pixel position of the reference band. Therefore, the pixel position (x, y) of the correction target band image corresponds to the pixel position (x, y) of the reference band image.
[0045] Next, the corrected multiband image creation unit 163 focuses on a pixel at one pixel position of the target band under consideration (step S33). The pixel at this pixel position of the target band under consideration is called the target pixel. Next, the corrected multiband image creation unit 163 extracts up to four pixels at the pixel position of the reference band that overlaps when the target pixel under consideration is shifted by the position difference (s, t) of the target pixel under consideration (step S34). The pixel of the reference band that overlaps with this target pixel is called a corresponding pixel. Here, s and t in the position difference (s, t) of each pixel of the target band are 0 or more and less than 1 pixel, and there is no position difference that is an integer multiple of a pixel. Then, when both s and t are not 0, the target pixel at pixel position (x, y) of the target band under consideration overlaps with four pixels of the reference band, that is, the four corresponding pixels at pixel positions (x, y), (x-1, y), (x, y-1), and (x-1, y-1). FIG. 13 is a schematic diagram showing such an overlap state, where the solid line rectangle indicates the corresponding pixel of the base band, and the dashed line rectangle indicates the target pixel of the target band. Also, when either s or t is 0, the target pixel at pixel position (x, y) of the target band under consideration overlaps with two pixels of the base band, and when both s and t are 0, the target pixel at pixel position (x, y) of the target band under consideration overlaps with one pixel of the base band. In this way, when either s or t, or both, are 0, the number of corresponding pixels is not 4. However, since the area of overlap between the corresponding pixel that does not overlap with the target pixel and the target pixel is 0, there is no effect even if it is treated the same as the corresponding pixel that overlaps with the target pixel. Therefore, in the following formulas, the case where either s or t, or both, are 0 will be treated without distinction.
[0046] Next, the corrected multiband image creation unit 163 calculates the area where the target pixel of the target band under consideration overlaps with each corresponding pixel of the reference band (step S35). If the areas where the target pixel at pixel position (x, y) overlaps with the corresponding pixels at pixel positions (x, y), (x, y-1), (x-1, y), and (x-1, y-1) are u(0, 0), u(0, 1), u(1, 0), and u(1, 1), respectively, these areas are given by Equation 1 shown in FIG. 14A using s and t.
[0047] Next, the corrected multiband image creation unit 163 determines values to be allocated to multiple pixels of the correction target band image corresponding to multiple corresponding pixels of the reference band from the pixel value of the target pixel of interest, based on the pixel value relationship between the corresponding pixels of the reference band and the area where each corresponding pixel overlaps with the target pixel of interest (step S36). That is, the corrected multiband image creation unit 163 focuses on the fact that luminance values or brightness have a strong correlation between bands, and allocates the pixel value of the target pixel of the target band of interest to multiple pixels of the correction target band image corresponding to multiple corresponding pixels of the reference band so that the pixel value relationship is the same as that between the corresponding pixels of the reference band.
[0048] The pixel value ratio is one of the pixel value relationships that can be used in the present invention. Also, instead of the pixel value ratio, the pixel value difference can be used. When using the pixel value ratio, the corrected multiband image creation unit 163 calculates, in step S36, a value to be assigned to multiple pixels of the band image to be corrected according to Equation 2 in FIG. 14A. Also, when using the pixel value difference, the corrected multiband image creation unit 163 calculates, in step S36, a value to be assigned to multiple pixels of the band image to be corrected according to Equation 3 in FIG. 14A. In Equations 2 and 3, T(x,y) is the pixel value of the target pixel at pixel position (x,y) of the target band under consideration, C(xp,yq) (p,q=0,1) is the pixel value of the corresponding pixel in the reference band, u(p,q) is the area of the region where the target pixel and the corresponding pixel in the reference band overlap, as described with reference to FIG. 13, and TC(x,y,p,q) is the value assigned to the pixel value T'(xp,yq) (p,q=0,1) of the corresponding pixel (xp,yq) (p,q=0,1) of the correction band image.
[0049] It is optional whether to use the pixel value ratio or the pixel value difference as the pixel value relationship. For example, in an environment where the condition that the brightness ratio is the same for corresponding pixels in the target band and the reference band is satisfied, the pixel value ratio may be used. That is, in order to be able to compare the brightness ratio, the pixel value ratio may be used in an environment where the pixel value is determined in proportion to the brightness entering each band from the reference pixel value 0, which is a state where no light is shining thereon. However, especially in an image of the ground captured from an artificial satellite, not only the light reflected by the earth's surface, which is the signal to be obtained, but also the light scattered by the atmosphere enters the sensor. Therefore, the pixel value becomes larger by the amount of light scattered by the atmosphere. Since the light scattered by the atmosphere has a larger wavelength when it is shorter, the amount by which the pixel value becomes larger differs depending on the band. Therefore, in an image of the ground captured from an artificial satellite, the pixel value ratio may not be the brightness ratio. Therefore, in such an environment, it is preferable to use the pixel value difference as the pixel value relationship. The reason is that the pixel value difference does not change even if the pixel values of each band are added by a certain amount. By using the pixel value difference, it is possible to remove the effect of adding output due to light scattered in the atmosphere to images taken of the ground from artificial satellites. Therefore, by using the pixel value difference, it is possible to create a corrected image without color shift even if the pixel value ratio is not the brightness ratio.
[0050] 12 again, the corrected multiband image creation unit 163 adds the values to be assigned to the multiple pixels of the band image to be corrected calculated in step S36 to the pixel values (initial value is 0) of those multiple pixels of the band image to be corrected (step S37). That is, if the pixel value of a pixel (x, y) in the band image to be corrected that is in phase with the reference band is T'(x, y), then T'(x, y) can be calculated using equation 8 in FIG. 14A.
[0051] Next, the corrected multiband image creation unit 163 shifts its attention to one target pixel at another pixel position of the target band under consideration (step S38), and returns to step S34 via step S39, and repeats the same processing as described above. Then, when the corrected multiband image creation unit 163 has finished focusing on all pixels of the target band under consideration (YES in step S39), it shifts its attention to one of the other target bands (e.g., B band) (step S40), and returns to step S32 via step S41, and repeats the same processing as described above. Then, when the corrected multiband image creation unit 163 has finished focusing on all the target bands (YES in step S41), it saves the corrected target band image in the storage unit 15 (step S42). That is, the corrected multiband image creation unit 163 writes the corrected R band image 154-1 and the corrected B band image 154-3 created as described above into the storage unit 15. 12. Furthermore, the corrected multiband image creation unit 163 writes the G-band image 152-2 of the reference band directly as the corrected G-band image 154-2 in the storage unit 15. Then, the corrected multiband image creation unit 163 ends the processing shown in FIG.
[0052] As described above, the image correction device 10 according to this embodiment can reduce color shift due to phase difference. The reason is that the image correction device 10 uses one of a plurality of band images obtained by capturing an image of a subject as a reference band image and at least one of the remaining images as a target band image, obtains a position difference between the target band image and the reference band image, and creates a correction band image that holds pixel values of light related to the target band image at pixel positions of the reference band image based on the relationship of pixel values between a plurality of pixels of the reference band image that overlap with pixels of the target band image when the pixels of the target band image are shifted by the position difference. The above effect will be described below with a simple example.
[0053] Now, consider a rectangular white board WB as shown in FIG. 15 as the subject 18. The background of the white board WB is black. The band image is made up of 4×4 pixels, and the pixel value of each pixel ranges from 0 to 255. FIG. 16 shows an example of pixel values of each pixel G11 to G44 of a G band image obtained by capturing the white board WB. In this example, the white board WB is captured only by the pixel G33 of the G band, and the pixel value of the pixel G33 is 255, and the pixel values of the other pixels are 0. FIG. 17 shows an example of pixel values of each pixel R11 to R44 of an R band image obtained by capturing the white board WB. In this example, the white board WB is captured by the pixels R34, R44, R33, and R43 of the R band, and the pixel values of these pixels are 255 / 4, and the pixel values of the other pixels are 0. In this case, the position differences (s, t) of the pixels R34, R44, R33, and R43 of the R band are (0.5, 0.5), respectively.
[0054] FIG. 18 shows an example of the overlap state between the R band image and the G band image when the R band pixels R34, R44, R33, and R43 are shifted by a position difference (0.5, 0.5), respectively. In this example, the R band pixel R34 overlaps with four pixels G24, G34, G23, and G33 in the G band image. The R band pixel R44 overlaps with four pixels G34, G44, G33, and G43 in the G band image. The R band pixel R33 overlaps with four pixels G23, G33, G22, and G32 in the G band image. The R band pixel R43 overlaps with four pixels G33, G43, G32, and G42 in the G band image.
[0055] In the above example, the pixel value of pixel R33' in the corrected R band image corresponding to pixel G33 is given as the sum of the pixel value assigned to pixel R33' from pixel R34, the pixel value assigned to pixel R33' from pixel R44, the pixel value assigned to pixel R33' from pixel R33, and the pixel value assigned to pixel R33' from pixel R43.
[0056] The pixel value allocated from pixel R34 to pixel R33' is 255 / 4 when formula 2 is used. That is, all of the pixel value of pixel R34 is allocated to pixel R33'. The reason is that the pixel value relationship between the four pixels G24, G34, G23, and G33 that overlap with pixel R34 is 0:0:0:254 in terms of pixel value ratio, so if pixel R34 is allocated to pixels R24', R34', R23', and R33' so as to have a similar pixel value ratio relationship, the result is 0:0:0:254 / 4. Similarly, all of the pixel value of pixel R44 is allocated to pixel R33', all of the pixel value of pixel R33 is allocated to pixel R33', and all of the pixel value of pixel R43 is allocated to pixel R33'. As a result, the pixel value of pixel R33' in the corrected R band image becomes 255, as shown in FIG. 19. The above is the case for Equation 2, which uses the ratio of pixel values as the relationship between pixel values. Similarly, in the case of Equation 3, which uses the difference in pixel values, all of the pixel values of pixels R34, R44, R33, and R43 are assigned to pixel R33', and the pixel value of pixel R33' in the corrected R band image becomes 255.
[0057] Meanwhile, because all of the pixel values of pixels R34, R44, R33, and R43 are assigned to pixel R33' as described above, the pixel values assigned to pixels of the corrected R band image other than pixel R33' by pixels R34, R44, R33, and R43 are 0. As a result, the pixel values of all pixels in the corrected R band image other than pixel R33' become 0, as shown in Fig. 19. As a result, no color shift occurs when the G band image and the corrected R band image are combined.
[0058] In contrast, when a normal interpolation formula is used to determine the pixel values of each pixel of the corrected R band image corresponding to each pixel position of the G band image in Fig. 18 from each pixel value of the R band image after the shift shown in Fig. 18, a corrected R band image is obtained in which the R band pixel values are allocated to pixels in a certain range centered on pixel R33' (e.g., pixels R24', R34', R44', R23', R33', R43', R22', R32', and R42'). As a result, when the G band image and the corrected R band image are combined, the white board BW appears as an image of a board that is red on the outside and green in the center. In other words, color shift occurs.
[0059] The above describes the color shift effect of this embodiment using a corrected R band image as an example, but color shift in a corrected B band image can also be reduced in a similar manner. Also, the effect of color shift correction has been described using a white board as an example, but it goes without saying that the same effect can be obtained with other types of subjects. Incidentally, in the case of a diagonal white line on a black background, in color shift correction using a normal interpolation method, if the position of the edge of the diagonal white line in the pixel differs for each band, a color shift occurs in which red is emphasized in some places on both sides of the white line and blue or green is emphasized in other places. According to the present invention, such color shift can also be reduced.
[0060] The above has described the configuration, operation, and effects of the image correction device 10 according to the first embodiment. Next, some modified examples of the first embodiment will be described.
[0061] <Variation 1> Fig. 20 is a flowchart showing another example of the process of the corrected multiband image creation unit 163 configured to use a pixel value ratio as the pixel value relationship. Referring to Fig. 20, when the corrected multiband image creation unit 163 starts the process of Fig. 20, it normalizes the pixel values of each pixel of the G band image 152-2 which is the reference band image and the R band image 152-1 and the B band image 152-3 which are the target band images so that the brightness ratio becomes the pixel value ratio (step S51). Then, the corrected multiband image creation unit 163 executes the processes of steps S31 to S42 shown in Fig. 12 using the normalized pixel values of the reference band and the target band.
[0062] The pixel value ratio is used as the pixel value relationship between corresponding pixels because it is assumed that the pixel value ratio is the brightness ratio. However, for example, if the absence of light is not a pixel value of 0 due to the inclusion of light scattered by the atmosphere, and an offset for atmospheric scattering is added, the pixel value ratio in the reference band image and the target band image will not exactly match the brightness ratio. Therefore, if color shift correction is performed using the original reference band image and target band image without normalization, the pixel value of the corrected target band will not be based on the pixel value ratio of the reference band.
[0063] Therefore, the corrected multiband image creation unit 163 normalizes pixel values so that correction can be performed correctly using the ratio of pixel values even if an offset is added to the multiband image acquired by the multiband image acquisition unit 161. In this example, the corrected multiband image creation unit 163 assumes that there is a pixel somewhere in the image that is not illuminated, and performs normalization by regarding the minimum value of each band image as the offset component when there is no light.
[0064] First, for each target band, the corrected multiband image creation unit 163 normalizes the pixel values of the target band using the minimum pixel value of the target band, as shown in Equation 4 in Fig. 14A. In Equation 4, T(x, y) represents the pixel value at the pixel position (x, y) of the target band acquired by the multiband image acquisition unit 161, minT(i, j) represents the minimum pixel value of the target band, and T"(x, y) represents the normalized pixel value of the target band. In this way, the corrected multiband image creation unit 163 normalizes the R band image using the minimum pixel value in the R band image, and normalizes the B band image using the minimum pixel value in the B band image.
[0065] 14A, the corrected multiband image creation unit 163 normalizes the pixel values of the reference band using the minimum pixel value of the reference band. In equation 5, C(x, y) represents the pixel value at the pixel position (x, y) of the reference band acquired by the multiband image acquisition unit 161, minC(i, j) represents the minimum pixel value of the reference band, and C"(x, y) represents the normalized pixel value of the reference band. In this way, the corrected multiband image creation unit 163 normalizes the G-band image using the minimum pixel value in the G-band image.
[0066] The corrected multiband image creation unit 163 executes the processes from step S31 in Fig. 12 using the R band image, G band image, and B band image normalized as described above, instead of the R band image 152-1, G band image 152-2, and B band image 152-3. At this time, the corrected multiband image creation unit 163 uses, for example, Equation 2' shown in Fig. 14B, instead of Equation 2 in Fig. 14A. Note that the last term on the right side of Equation 2' is a correction term for keeping the absolute value of the pixel value of the target band relatively unchanged from before correction. The correction term may be omitted.
[0067] <Variation 2> FIG. 21 is a flowchart showing another example of the process of the corrected multiband image creation unit 163 configured to use the difference in pixel values as the relationship of pixel values. Referring to FIG. 21, when the corrected multiband image creation unit 163 starts the process of FIG. 21, it focuses on one of the target bands (e.g., the R band) as in FIG. 12 (step S31). Next, the corrected multiband image creation unit 163 performs normalization so that the difference in brightness between the target band under consideration and the reference band becomes the difference in pixel value (step S61). Then, the corrected multiband image creation unit 163 uses the pixel values of the normalized reference band and the target band to execute the process from step S32 shown in FIG. 12 onward. Note that when NO is determined in step S41 shown in FIG. 12, the process returns to step S61 and the same process as described above is repeated.
[0068] The reason why the pixel value difference is used as the pixel value relationship between corresponding pixels is because it is assumed that the pixel value difference is the brightness difference. However, for example, due to differences in sensitivity between bands, the value difference may differ depending on the band even for the same brightness difference. In such a case, the pixel value difference will not exactly match the brightness difference, so if color shift correction is performed using the original reference band image and target band image without normalization, the corrected pixel value of the target band will not be based on the pixel value difference of the reference band.
[0069] Therefore, the corrected multiband image creation unit 163 normalizes pixel values so that correction can be performed correctly using the difference in pixel values even when there is a difference in sensitivity between bands of the multiband image acquired by the multiband image acquisition unit 161. In step S61 of FIG. 21, the corrected multiband image creation unit 163 calculates the normalized pixel value of the reference band using equation 6 of FIG. 14A. In equation 6, C(x, y) is the pixel value of the G band image before normalization, σ(C(x, y)) is the standard deviation of the pixel values of the reference band, σ(T(x, y)) is the standard deviation of the pixel values of the target band, and C"(x, y) is the normalized pixel value of the reference band.
[0070] According to Equation 6, the variance (standard deviation) of pixel values of the normalized reference band becomes the same as the variance (standard deviation) of pixel values of the target band, so that the contrast of the reference band and the target band can be made uniform. Furthermore, by making the contrast uniform, the sensitivity to changes in brightness can be made uniform between the reference band and the target band. The corrected multiband image creation unit 163 uses the G band image normalized as described above instead of the G band image 152-2 to execute the processes from step S32 onward in FIG. 12. At this time, the corrected multiband image creation unit 163 uses, for example, Equation 3' shown in FIG. 14B instead of Equation 3 in FIG. 14A.
[0071] In the above, the sensitivity to changes in brightness between the reference band and the target band is matched by correcting the pixel values of the reference band, but it is also possible to match the sensitivity to changes in brightness between the reference band and the target band by correcting the pixel values of the target band.
[0072] <Modification 3> In the example shown in FIG. 4, the position difference information 153 is recorded as a list of pairs of pixel positions and position differences for each pixel of the target band image. However, the position difference information 153 is not limited to the above recording method. For example, the position difference information 153 may be recorded as a list of pairs of pixel positions and position differences for each partial region by dividing the target band image into a plurality of partial regions consisting of a collection of a plurality of pixels having the same position difference. The shape of the partial region may be, for example, a rectangle. Furthermore, if the partial region is rectangular, the pixel positions may be a pair of pixel positions of the upper left pixel and the lower right pixel. Furthermore, if the position differences of all pixels of the target band image are approximately the same, the configuration may be such that only one position difference is recorded.
[0073] Furthermore, instead of recording the position difference information 153 as numerical information, it may be recorded as a mathematical formula or the coefficients of a mathematical formula. For example, when the position difference occurs due to optical distortion, the position difference is determined by the positional relationship and optical characteristics between the pixels of the target band and the pixels of the reference band on the focal plane. Therefore, the position difference for each pixel can be expressed by a formula determined by optical characteristics with the pixel position as an argument. Therefore, this formula or its coefficients may be recorded as the position difference information 153. The position difference for each pixel can be calculated from the above formula.
[0074] The position difference acquisition unit 162 may calculate an approximate plane from the calculated position difference of each pixel position for each target band, and record the formula or the coefficient of the formula representing the calculated approximate plane as the position difference information 153. For example, when each pixel of the target band is three-dimensional point cloud data composed of three-dimensional data (x, y, (s, t)) of pixel position x, pixel position y, and position difference (s, t), the approximate plane may be a plane that minimizes the sum of the square distances with the point cloud. For example, when a plane given by the formula 7 in FIG. 14A is used as the approximate plane, the position difference acquisition unit 162 can obtain an equation representing the position difference information 153 of all pixels by obtaining the most fitting matrix A and vector b using the calculated position difference of each pixel position. In the above, the coefficient of formula 7 is obtained using the position difference of each pixel position of the target band, but the most fitting matrix A and vector b may be obtained using the position difference of the pixel at the center position of each small region described with reference to FIG. 7.
[0075] <Variation 4> In the above, the multiband image 152 is an image of three bands, RGB. However, the multiband image 152 may be other than the above. For example, the multiband image 152 may be an image of four bands, including three bands of RGB and a near-infrared band. In this way, there is no restriction on the number of bands in the multiband image 152, and any number of bands in any wavelength band may be used.
[0076] <Variation 5> In the above description, the position difference (s,t) between pixel (x,y) of the target band and pixel (x,y) of the same pixel position of the reference band is described as 0 or more and less than 1, but the position difference (s,t) may be less than 0 or 1 or more. For any position difference (s,t), the maximum integer not exceeding s is s0, the maximum integer not exceeding t is t0, and s'=s-s0 and t'=t-t0. Then, for pixel (x,y) of the reference band, if x'=x-s0 and y'=y-t0, the position difference between pixel (x,y) of the target band and pixel (x',y') of the reference band is (s',t') which is 0 or more and less than 1. Therefore, by replacing pixel (x,y) of the reference band with pixel (x',y') and position difference (s,t) with position difference (s',t'), the pixel value of the correction band image can be obtained by the same process as above.
[0077] [Second embodiment] Fig. 22 is a block diagram of an image correction device 20 according to a second embodiment of the present invention. Referring to Fig. 22, the image correction device 20 includes a band image acquiring means 21, a position difference acquiring means 22, a corrected band image creating means 23, and a corrected band image output means 24.
[0078] The band image acquiring means 21 is configured to acquire a plurality of band images obtained by capturing an image of a subject. The band image acquiring means 21 can be configured similarly to the multiband image acquiring unit 161 in FIG. 1, for example, but is not limited thereto.
[0079] The position difference acquisition means 22 is configured to set one of the multiple band images acquired by the band image acquisition means 21 as a reference band image, and at least one of the remaining band images as a target band image, and to acquire a position difference between the target band image and the reference band image acquired by the band image acquisition means 21. The position difference acquisition means 22 can be configured similarly to, for example, the position difference acquisition unit 162 in Fig. 1, but is not limited thereto.
[0080] The correction band image creation means 23 is configured to set a pixel of the target band image as a target pixel, set a plurality of pixels of the reference band image that overlap with the target pixel when the target pixel is shifted by the position difference acquired by the position difference acquisition means 22 as corresponding pixels, and create a correction band image that holds the pixel values of light related to the target band image at the pixel position of the reference band image based on the pixel value relationship between the plurality of corresponding pixels. The correction band image creation means 23 can be configured similarly to, for example, the corrected multiband image creation unit 163 in FIG. 1, but is not limited thereto.
[0081] The correction band image output means 24 is configured to output the correction band image created by the correction band image creation means 23. The correction band image output means 24 can be configured similarly to, for example, the corrected multiband image output unit 164 in Fig. 1, but is not limited thereto.
[0082] The image correction device 20 having the above configuration operates as follows. That is, first, the band image acquisition means 21 acquires a plurality of band images obtained by capturing an image of a subject. Next, the position difference acquisition means 22 sets one of the plurality of band images acquired by the band image acquisition means 21 as a reference band image and at least one of the remaining images as a target band image, and acquires a position difference between the target band image and the reference band image acquired by the band image acquisition means 21. Next, the correction band image creation means 23 sets a pixel of the target band image as a target pixel, sets each of a plurality of pixels of the reference band image that overlaps with the target pixel when the target pixel is shifted by the position difference acquired by the position difference acquisition means 22 as a corresponding pixel, and creates a correction band image that holds the pixel value of light related to the target band image at the pixel position of the reference band image based on the pixel value relationship between the plurality of corresponding pixels. Next, the correction band image output means 24 outputs the correction band image created by the correction band image creation means 23.
[0083] According to the image correction device 20 configured and operating as described above, it is possible to reduce color shift due to phase difference. The reason for this is that the image correction device 20 uses one of the multiple band images obtained by capturing an image of a subject as a reference band image and at least one of the remaining images as a target band image, obtains a position difference between the target band image and the reference band image, uses a pixel of the target band image as a target pixel, and uses each of the multiple pixels of the reference band image that overlap with the target pixel when the target pixel is shifted by the position difference as a corresponding pixel, and creates a correction band image that holds the pixel value of light related to the target band image at the pixel position of the reference band image based on the pixel value relationship between the multiple corresponding pixels.
[0084] Although the present invention has been described with reference to the above-mentioned embodiments, the present invention is not limited to the above-mentioned embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention. [Industrial Applicability]
[0085] The present invention can be used as an image correction device, an image correction method, and an image correction program for correcting a multiband image (multispectral image) to an image without color shift. It can also be used to correct color shift during image geometric projection, such as projecting an image obtained by capturing an image of the ground from a satellite or an airplane onto a map.
[0086] A part or all of the above-described embodiments can be described as, but is not limited to, the following supplementary notes. [Appendix 1] a band image acquisition means for acquiring a plurality of band images obtained by imaging a subject; a position difference acquisition means for acquiring a position difference between the target band image and the reference band image, the target band image being an object band image and the reference band image being an object band image; a correction band image creation means for creating a correction band image that holds pixel values of light related to the target band image at pixel positions of the reference band image based on a pixel value relationship between the pixel values of the corresponding pixels of the reference band image, the pixel values of the pixel values of the corresponding pixels of the reference band image being corresponding pixels of the pixel values of the corresponding pixels of the target band image being corresponding pixels of the pixel values of the corresponding pixels of the reference band image, the correction band image being corresponding pixels of the pixel values of the corresponding pixels of the target band image being corresponding pixels of the pixel values of the corresponding pixels of the reference band image being corresponding pixels of the pixel values of the ... a correction band image output means for outputting the correction band image; An image correction device comprising: [Appendix 2] The correction band image creating means includes: for each of the target pixels, a pixel value of light related to the target band image to be allocated to each of the positions of the plurality of corresponding pixels is determined from the pixel value of the target pixel based on a pixel value relationship between the plurality of corresponding pixels and an overlapping area between the target pixel and the corresponding pixel, and for each pixel position of the reference band image, a sum of pixel values of light related to the target band image allocated from the plurality of pixels of the target band image that overlap with pixels of the reference band image is calculated. 2. An image correction device as described in claim 1. [Appendix 3] the correction band image creation means uses a ratio of pixel values as the relationship of the pixel values; 3. An image correction device according to claim 1 or 2. [Appendix 4] the correction band image creation means normalizes each pixel value of the reference band image using a minimum pixel value of the reference band image. 4. An image correction device as described in claim 3. [Appendix 5] the correction band image creation means uses a difference between pixel values as the relationship between the pixel values; 3. An image correction device according to claim 1 or 2. [Appendix 6] the correction band image creation means normalizes each pixel value of the reference band image using a standard deviation of the pixel values of the reference band image and a standard deviation of the pixel values of the target band image. 6. An image correction device as described in claim 5. [Appendix 7] the position difference acquisition means calculates the position difference based on image correlation between the reference band image and the target band image; 7. An image correction device according to any one of claims 1 to 6. [Appendix 8] the position difference acquisition means divides the reference band image and the target band image into a plurality of small regions, and calculates, for each small region, a shift amount by which the small region of the target band image most closely matches the small region of the reference band image, as the position difference of all pixels of the small region of the target band image; 8. An image correction device according to any one of claims 1 to 7. [Appendix 9] the position difference acquisition means divides the reference band image and the target band image into a plurality of small regions, calculates, for each small region, a shift amount by which the small region of the target band image most closely matches the small region of the reference band image as the position difference of a pixel at a center position of the small region of the target band image, and calculates the position difference of pixels other than the pixel at the center position by an interpolation process from the position difference of the pixel at the center position. 8. An image correction device according to any one of claims 1 to 7. [Appendix 10] A plurality of band images are obtained by imaging the subject, one of the plurality of band images is a reference band image, and at least one of the remaining band images is a target band image; and a position difference between the target band image and the reference band image is obtained; a pixel of the target band image is set as a target pixel, and a plurality of pixels of the reference band image that overlap with the target pixel when the target pixel is shifted by the position difference are set as corresponding pixels, and a correction band image is created that holds pixel values of light related to the target band image at pixel positions of the reference band image based on a pixel value relationship between the plurality of corresponding pixels; outputting the correction band image; Image correction methods. [Appendix 11] On the computer, A process of acquiring a plurality of band images obtained by capturing an image of a subject; a process of determining one of the plurality of band images as a reference band image and at least one of the remaining band images as a target band image, and acquiring a position difference between the target band image and the reference band image; a process of setting a pixel of the target band image as a target pixel, setting each of a plurality of pixels of the reference band image that overlap with the target pixel when the target pixel is shifted by the position difference as a corresponding pixel, and creating a correction band image that holds pixel values of light related to the target band image at pixel positions of the reference band image based on the pixel value relationships between the plurality of corresponding pixels; outputting the correction band image; A program to carry out the above. [Explanation of symbols]
[0087] 10. Image correction device 11 One-dimensional array sensor 12 Communication I / F section 13 Operation input section 14 Screen display section 15 Storage section 16 Processing unit 17 Optical system 18 Subject 20 Image correction device 21 Band Image Acquisition Method 22 Position difference acquisition means 23 Correction band image creation method 24 Correction band image output means 151 Programs 152 multiband images 152-1 R-band image 152-2 G-band image 152-3 B-band image 153 Location difference information 153-1 Position difference information 153-2 Position difference information 154 Corrected Multiband Images 154-1 Corrected R-band image 154-2 Corrected G-band image 154-3 Corrected B-band image 161 Multiband image acquisition unit 162 Position difference acquisition unit 163 Corrected multi-band image creation section 164 Corrected multi-band image output section 1621 Position Difference Information Creation Unit 1622 Position difference information input section WB white board
Claims
1. a band image acquisition means for acquiring a plurality of band images obtained by imaging a subject; a position difference acquisition means for acquiring a position difference between the target band image and the reference band image, the target band image being an object band image and the reference band image being an object band image; a correction band image creation means for creating a correction band image that holds pixel values of light related to the target band image at pixel positions of the reference band image based on a pixel value relationship between the pixel values of the corresponding pixels and an overlapping area between the target pixel and the corresponding pixel, the correction band image creation means defining a pixel of the target band image as a target pixel, defining each of a plurality of pixels of the reference band image that overlap with the target pixel when the target pixel is shifted by the position difference as a corresponding pixel, and creating a correction band image that holds pixel values of light related to the target band image at pixel positions of the reference band image based on a pixel value relationship between the corresponding pixels and an overlapping area between the target pixel and the corresponding pixel; a correction band image output means for outputting the correction band image; An image correction device comprising:
2. The correction band image creating means includes: for each of the target pixels, a pixel value of light related to the target band image to be allocated to each of the positions of the plurality of corresponding pixels is determined from the pixel value of the target pixel based on a pixel value relationship between the plurality of corresponding pixels and an overlapping area between the target pixel and the corresponding pixel, and for each pixel position of the reference band image, a sum of pixel values of light related to the target band image allocated from the plurality of pixels of the target band image that overlap with pixels of the reference band image is calculated.
2. The image correction device according to claim 1.
3. the correction band image creation means uses a ratio of pixel values as the relationship of the pixel values; 3. The image correction device according to claim 1.
4. the correction band image creation means normalizes each pixel value of the reference band image using a minimum pixel value of the reference band image.
4. The image correction device according to claim 3.
5. the correction band image creation means uses a difference between pixel values as the relationship between the pixel values; 3. The image correction device according to claim 1.
6. the correction band image creation means normalizes each pixel value of the reference band image using a standard deviation of the pixel values of the reference band image and a standard deviation of the pixel values of the target band image.
6. The image correction device according to claim 5.
7. the position difference acquisition means divides the reference band image and the target band image into a plurality of small regions, and calculates, for each small region, a shift amount by which the small region of the target band image most closely matches the small region of the reference band image, as the position difference of all pixels of the small region of the target band image; 7. An image correction device according to claim 1.
8. the position difference acquisition means divides the reference band image and the target band image into a plurality of small regions, calculates, for each small region, a shift amount by which the small region of the target band image most closely matches the small region of the reference band image as the position difference of a pixel at a center position of the small region of the target band image, and calculates the position difference of pixels other than the pixel at the center position by an interpolation process from the position difference of the pixel at the center position.
7. An image correction device according to claim 1.
9. A plurality of band images are obtained by imaging the subject, one of the plurality of band images is a reference band image, and at least one of the remaining band images is a target band image; and a position difference between the target band image and the reference band image is obtained; a pixel of the target band image is set as a target pixel, and each of a plurality of pixels of the reference band image that overlap with the target pixel when the target pixel is shifted by the position difference is set as a corresponding pixel, and a correction band image is created that holds pixel values of light related to the target band image at pixel positions of the reference band image based on the relationship in pixel values between the plurality of corresponding pixels and the overlapping area between the target pixel and the corresponding pixel; outputting the correction band image; Image correction methods.
10. On the computer, A process of acquiring a plurality of band images obtained by capturing an image of a subject; a process of determining one of the plurality of band images as a reference band image and at least one of the remaining band images as a target band image, and acquiring a position difference between the target band image and the reference band image; a process of creating a correction band image that holds pixel values of light related to the target band image at pixel positions of the reference band image based on a pixel value relationship between the pixel values of the corresponding pixels and an overlapping area between the target pixel and the corresponding pixels, the correction band image being created based on a pixel value relationship between the pixel values of the corresponding pixels and an overlapping area between the target pixel and the corresponding pixels, the correction band image being created based on a pixel value relationship between the pixel values of the corresponding pixels and an overlapping area between the target pixel and the corresponding pixels; outputting the correction band image; A program to carry out the above.
Citation Information
Patent Citations
Optical switch matrix
JP1988005328A
Registration correction circuit
JP1991151784A
Image synthesizer
JP2016110320A
Moving body detecting device and moving body detecting program
JP2018061262A
Processing of remotely acquired imaging data including moving objects
US20100189363A1
