White balance correction device and white balance correction method

Abstract

To provide a white balance correction device which performs more natural white balance correction.SOLUTION: A white balance correction device comprises: a division section which divides an image into first and second regions; a reference point calculation section which calculates first and second reference points; a correction value calculation section which calculates first and second correction values of white balance with respect to the first and second reference points; a plane distance calculation section which calculates a first plane distance between the first reference point and an arbitrary point and a second plane distance between the second reference point and the arbitrary point; a depth distance calculation section which calculates a first depth distance between the first reference point and the arbitrary point and a second depth distance between the second reference point and the arbitrary point; an interpolation section which interpolates the correction value of the white balance with respect to the arbitrary point on the basis of the first and second correction values, the first and second plane distances, and the first and second depth distances; and a correction section which corrects the white balance of the arbitrary point.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a white balance correction device and a white balance correction method. [Background technology]

[0002] A technique is known in which white balance correction coefficients for non-target pixels excluding a plurality of target pixels are individually calculated from white balance correction coefficients of a plurality of target pixels existing around each non-target pixel by linear interpolation based on the distance from each target pixel (see, for example, Patent Document 1).

[0003] For example, a case will be described in which a white balance correction value for a point (pixel) p in an image 10 shown in Fig. 1 is obtained by interpolation using conventional technology. FIG. 2 shows the positional relationship between the camera 11 that captured the image 10 and the subject when viewed from above. The image 10 shows a person A and a wall B as subjects. When the image 10 is captured, the light sources used are a flash light (first light source) from the camera 11 and a fluorescent light (second light source) installed in a position close to the wall B. The person A is strongly irradiated with the flash light from the camera 11, and the wall B is irradiated with the fluorescent light that is stronger than the flash light from the camera 11.

[0004] In the conventional white balance correction method, first, a white balance correction value C1 of the pixel of interest P1 is calculated from the colors of the pixels of interest P1 in the area of ​​person A in image 10, and a white balance correction value C2 of the pixel of interest P2 is calculated from the colors of the pixels of interest P2 in the area of ​​wall B in image 10. Then, the correction value of point p is calculated by linear interpolation from the white balance correction value C1 and the white balance correction value C2 based on the ratio of the distance L1 between point p and pixel of interest P1 and the distance L2 between point p and pixel of interest P2. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2005-347811 A Summary of the Invention [Problem to be solved by the invention]

[0006] However, with conventional white balance correction methods, when two pixels in an image are far apart in the depth direction in the actual shooting situation, but are close in the planar direction of the captured image, appropriate white balance correction is not performed, and unnatural colors may result.

[0007] For example, as shown in FIG. 2, point p is on wall B, and the color of point p is more influenced by the light of a second light source (fluorescent light) that is installed closer to wall B than the first light source (flash light of camera 10). However, as shown in image 10 in FIG. 1, point p is closer to pixel of interest P1 than pixel of interest P2 on the image plane, so in the conventional white balance correction method, the white balance correction value of point p is more strongly influenced by the white balance correction value of pixel of interest P1, which is more influenced by the first light source. As a result, appropriate white balance correction is not performed, and the color of point p becomes unnatural.

[0008] An object of one aspect of the present invention is to provide a white balance correction device that performs more natural white balance correction. [Means for solving the problem]

[0009] A white balance correction device according to one aspect of the present invention includes a division unit that divides an image into at least a first region and a second region, a reference point calculation unit that calculates a first reference point in the first region and a second reference point in the second region, a correction value calculation unit that calculates a first correction value of white balance for the first reference point and a second correction value of white balance for the second reference point, a first planar distance between the first reference point and an arbitrary point in the image in a planar direction of the image, and a second planar distance between the second reference point and the arbitrary point in the planar direction of the image. the first depth distance between the first reference point and the arbitrary point in the depth direction and a second depth distance between the second reference point and the arbitrary point in the depth direction; an interpolation unit that interpolates a white balance correction value of the arbitrary point based on the first correction value, the second correction value, the first planar distance, the second planar distance, the first depth distance, and the second depth distance; and a correction unit that corrects the white balance of the arbitrary point using the white balance correction value of the arbitrary point.

[0010] A white balance correction method according to one aspect of the present invention includes: dividing an image into at least a first region and a second region; calculating a first reference point in the first region and a second reference point in the second region; calculating a first correction value of white balance for the first reference point and a second correction value of white balance for the second reference point; calculating a first planar distance between the first reference point and an arbitrary point in the image in a planar direction of the image and a second correction value of white balance for the second reference point; the first correction value, the second correction value, the first planar distance, the second planar distance, the first depth distance, and the second depth distance, the first correction value, the second correction value, the first depth distance, and the second depth distance, and the white balance of the arbitrary point is corrected using the white balance correction value of the arbitrary point. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram illustrating a conventional white balance correction. [Diagram 2] FIG. 2 is a plan view showing the positional relationship of subjects. [Diagram 3] 1 is a functional block diagram of a white balance correction device according to an embodiment; [Figure 4] 4 is a flowchart of a white balance correction method according to an embodiment. [Diagram 5] 1 is an example of an image before white balance correction. [Figure 6] 5A to 5C are diagrams illustrating white balance correction according to an embodiment. [Figure 7] FIG. 2 is a plan view showing the positional relationship of subjects. [Figure 8] 13 is an example of a correction value table according to another embodiment. [Figure 9] 13 is an example of an index value table according to another embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, embodiments will be described with reference to the drawings. In the drawings, the same or equivalent elements are designated by the same reference numerals, and duplicated explanations will be omitted.

[0013] FIG. 3 is a functional block diagram of the white balance correction device according to the embodiment.

[0014] The white balance correction device 101 has a control unit 111, an imaging unit 121, a distance measurement unit 131, a storage unit 141, and a communication unit 151. The white balance correction device 101 is, for example, a mobile terminal, a digital camera, or a personal computer.

[0015] The control unit 111 controls the white balance correction device 101. The control unit 111 performs white balance correction on an image before white balance correction captured by the imaging unit 121. The control unit 111 has a division unit 112, a reference point calculation unit 113, a correction value calculation unit 114, a planar distance calculation unit 115, a depth distance calculation unit 116, an interpolation unit 117, and a correction unit 118.

[0016] The division unit 112 divides the image captured by the imaging unit 121 into a plurality of regions. For example, the division unit 112 divides the image captured by the imaging unit 121 into at least a first region and a second region. For example, the division unit 112 divides the image into a plurality of regions by analyzing the image and grouping pixels having similar features.

[0017] The reference point calculation unit 113 calculates a reference point (reference pixel) for each of the multiple regions. For example, the reference point calculation unit 113 calculates a first reference point in a first region and a second reference point in a second region. For example, the reference point calculation unit 113 determines, for each region, a point (pixel) that is closest to an achromatic color as the reference point.

[0018] The correction value calculation unit 114 calculates a white balance correction value for each reference point. For example, the correction value calculation unit 114 calculates a first white balance correction value C1 for the first reference point and a second white balance correction value C2 for the second reference point.

[0019] The planar distance calculation unit 115 calculates the distance (planar distance) between each reference point in the planar direction of the image and all points in the image other than the reference point. For example, the planar distance calculation unit 115 calculates a first planar distance L1 between a first reference point in the planar direction of the image and an arbitrary point p in the image, and a second planar distance L2 between a second reference point in the planar direction of the image and the arbitrary point p.

[0020] The depth distance calculation unit 116 calculates the distance (depth distance) between each reference point in the depth direction and the location of the object corresponding to each of all points in the image other than the reference point. For example, the depth distance calculation unit 116 calculates a first depth distance D1 between a first reference point and an arbitrary point in the depth direction, and a second depth distance D2 between a second reference point and an arbitrary point in the depth direction.

[0021] The interpolation unit 117 interpolates the white balance correction value of each point other than the reference point in the image based on the white balance correction value for each reference point, the planar distance between each point other than the reference point in the image and each reference point, and the depth distance between each point other than the reference point in the image and each reference point. For example, the interpolation unit 117 interpolates the white balance correction value C of an arbitrary point p based on the first correction value C1, the second correction value C2, the first planar distance L1, the second planar distance L2, the first depth distance D1, and the second depth distance D2.

[0022] Specifically, for example, when an index value calculated from a first planar distance L1 between a first reference point P1 and an arbitrary point p and a first depth distance D1 between the first reference point P1 and the arbitrary point p is K1, and an index value calculated from a second planar distance L1 between a second reference point P2 and the arbitrary point p and a second depth distance D2 between the second reference point P2 and the arbitrary point p is K2, the white balance correction value C of the arbitrary point p is calculated by the following formula (1).

[0023] C=C1*K2 / (K1+K2)+C2*K1 / (K1+K2) ···(1) Moreover, the index value K1 is, for example, the square root of the sum of the squares of the first planar distance L1 and the first depth distance D1 (Euclidean distance) as shown in the following formula (2).

[0024] K1=(L1 2 +D1 2 ) 1 / 2 (2) Moreover, the index value K2 is, for example, the square root of the sum of the squares of the second planar distance L2 and the first depth distance D2 (Euclidean distance) as shown in the following formula (3).

[0025] K2=(L2 2 +D2 2 ) 1 / 2 (3) The interpolation unit 117 may also determine the white balance correction value C by referring to a correction value table in which correction values ​​are described. The interpolation unit 117 may also determine the index values ​​K1 and K2 by referring to an index value table in which index values ​​are described.

[0026] The correction unit 118 corrects the white balance for all points in the image using the calculated white balance correction value. For example, the correction unit 118 corrects the white balance of an arbitrary point p using the calculated white balance correction value of the arbitrary point p. The correction unit 118 also corrects the white balance of the reference points P1 and P2 using the white balance correction value calculated by the correction value calculation unit 114.

[0027] In the conventional white balance correction method, as shown in the image 10 of FIG. 1, since the point p is closer to the pixel of interest P1 than the pixel of interest P2 in the image plane, the white balance correction value of the point p is more strongly influenced by the white balance correction value of the pixel of interest P1, which is more influenced by the first light source. As a result, appropriate white balance correction is not performed, and the color of the point p becomes unnatural. On the other hand, in the white balance correction method according to the embodiment, as shown in FIGS. 6 and 7, the point p is closer to the reference point P1 than the reference point P2 in the image plane, but the second depth distance D2 between the point p and the reference point P2 in the depth direction is almost nonexistent, and the first depth distance D1 between the point p and the reference point P1 is large. As a result, the white balance correction value of the point p according to the embodiment is less influenced by the white balance correction value of the reference point P1 than in the conventional white balance correction method, and more natural white balance correction is performed.

[0028] The imaging unit 121 captures an image of a predetermined range and generates an image before white balance correction. The imaging unit 121 includes, for example, a lens and an imaging element.

[0029] The distance measuring unit 131 is a sensor that measures the distance to an object, and measures the distance to an object that exists in the shooting direction of the imaging unit 121. The distance measuring unit 131 measures the distance between the white balance correction device 101 and an object (such as a person, an object, and a background wall) in an image captured by the imaging unit 121. In the specification, the distance to a point (pixel) p in an image in the depth direction refers to the distance in the depth direction between the white balance correction device 101 and a location of the object corresponding to the point p in the image captured by the imaging unit 121. The distance measuring unit 131 is, for example, a ToF (Time of Flight) sensor.

[0030] Instead of separately providing the distance measuring unit 131 and the control unit 111, the control unit 111 may further include a distance measuring unit that calculates a distance between the white balance correction device 101 and an object in the image captured by the imaging unit 121. The distance measuring unit of the control unit 111 may calculate the depth distance between the white balance correction device 101 and an object in the image captured by the imaging unit 121 by, for example, a distance calculation method using a stereo camera. In addition, the distance measuring unit of the control unit 111 may calculate the depth distance between the white balance correction device 101 and an object in the image captured by the imaging unit 121 by monocular depth estimation using deep learning.

[0031] The storage unit 141 stores data, programs, and the like used in the white balance correction device 101. The storage unit 141 stores, for example, an image whose white balance has been corrected by the control unit 111. The storage unit 141 is, for example, a Read Only Memory (ROM), a Random Access Memory (RAM), a flash memory, or a portable recording medium.

[0032] The communication unit 151 communicates with other devices via a network such as a local area network (LAN) or a wide area network (WAN).

[0033] Next, a white balance correction method according to the embodiment will be described with reference to FIGS.

[0034] Fig. 4 is a flowchart of a white balance correction method according to an embodiment. Fig. 5 is an example of an image before white balance correction. Fig. 6 is a diagram for explaining white balance correction according to an embodiment. Fig. 7 is a plan view showing the positional relationship of subjects.

[0035] In step S201, the imaging unit 121 captures an image of a predetermined range and generates an image before white balance correction. For example, an image 501 as shown in Fig. 5 is generated. The image 501 shows a person A and a wall B in the background. The person A is standing in front of the wall B, and is located closer to the white balance correction device 101 that captured the image 501 than the wall B.

[0036] In step S202, the division unit 112 divides the image captured by the imaging unit 121 into a plurality of regions. Specifically, for example, the division unit 112 divides the image captured by the imaging unit 121 into at least a first region and a second region. Specifically, for example, in FIG. 5, the division unit 112 divides the image 501 into a region of a person A (first region) and a region of a wall B (second region). In the image 501 in FIG. 5, the image is divided into a region of the person A shown in gray and a region of the wall B shown in white. Note that, although an example of division into two regions will be described here, the number of divided regions is not limited to two and may be three or more.

[0037] In step S203, the reference point calculation unit 113 calculates a reference point (reference pixel) for each of the multiple regions. For example, the reference point calculation unit 113 calculates a first reference point P1 in the region (first region) of person A and a second reference point P2 in the region (second region) of wall B. Note that the reference point calculation unit 113 determines, for example, a point (pixel) that is closest to an achromatic color as the reference point for each region.

[0038] In step S204, the correction value calculation unit 114 calculates a white balance correction value for each reference point. Specifically, for example, the correction value calculation unit 114 calculates a first white balance correction value C1 for the first reference point P1 and a second white balance correction value C2 for the second reference point P2.

[0039] In step S205, the distance measuring unit 131 measures the distance in the depth direction between the white balance correction device 101 and a location on the object corresponding to the reference point. Specifically, for example, the distance measuring unit 131 measures a depth distance DP1 in the depth direction (Z-axis direction in FIG. 7) between the white balance correction device 101 and a location on person A corresponding to reference point P1. Furthermore, the distance measuring unit 131 measures a depth distance DP2 in the depth direction between the white balance correction device 101 and a location on wall B corresponding to reference point P2.

[0040] In step S206, the planar distance calculation unit 115 calculates the distance (planar distance) in the planar direction between each reference point in the planar direction of the image and all points other than the reference points in the image. Specifically, for example, the planar distance calculation unit 115 calculates a first planar distance L1 between a first reference point P1 in the planar direction of the image (XY plane in FIG. 6) and an arbitrary point p in the image, and a second planar distance L2 between a second reference point P2 and the arbitrary point p in the planar direction of the image. FIG. 6 shows the first planar distance L1 and the second planar distance L2 of a certain arbitrary point p. For example, the planar distance calculation unit 115 sequentially selects one point other than the reference point in the image as the arbitrary point p, and repeatedly calculates the first planar distance L1 and the second planar distance L2 of the selected point, thereby calculating the first planar distance L1 and the second planar distance L2 of all points (pixels) in the image 501 other than the reference point.

[0041] In step S207, the depth distance calculation unit 116 calculates the distance (depth distance) between each reference point in the depth direction and the location of the object corresponding to each of all points other than the reference point in the image. Specifically, for example, as shown in FIG. 6, the distance measurement unit 131 measures the depth distance Dp in the depth direction (Z-axis direction in FIG. 7) between the white balance correction device 101 and the location of the wall B corresponding to the arbitrary point p. Then, the depth distance calculation unit 116 subtracts the depth distance DP1 measured in step S205 from the depth distance Dp to calculate a first depth distance D1 in the depth direction between the reference point P1 and the arbitrary point p. Similarly, the depth distance calculation unit 116 calculates a second depth distance D2 in the depth direction between the reference point P2 and the arbitrary point p. FIG. 7 shows the first depth distance D1 and the second depth distance D2 of a certain arbitrary point p.

[0042] The distance measurement unit 131 and the depth distance calculation unit 116, for example, sequentially select one point other than the reference point in the image as an arbitrary point p, and repeatedly calculate the first depth distance D1 and the second depth distance D2 of the selected point, thereby calculating the first depth distance D1 and the second depth distance D2 of each of all points (pixels) in the image 501 other than the reference point.

[0043] In step S208, the interpolation unit 117 interpolates the white balance correction value of each point other than the reference point in the image based on the white balance correction value for each reference point, the planar distance between each point other than the reference point in the image and each reference point, and the depth distance between each point other than the reference point in the image and each reference point. For example, the interpolation unit 117 interpolates the white balance correction value C of an arbitrary point P based on the first correction value C1, the second correction value C2, the first planar distance L1, the second planar distance L2, the first depth distance D1, and the second depth distance D2.

[0044] Specifically, for example, the interpolation unit 117 calculates the white balance correction value C of an arbitrary point p using the above formulas (1) to (3). The interpolation unit 117 interpolates the white balance correction values ​​of all points in the image 501 except for the reference points P1 and P2.

[0045] In step S209, the correction unit 118 corrects the white balance for all points in the image using the calculated white balance correction value. Specifically, for example, the correction unit 118 corrects the white balance for the reference points P1 and P2 using the white balance correction value calculated in step S204. The correction unit 118 also corrects the white balance for all points other than the reference points P1 and P2 using the white balance correction value calculated by interpolation in step S208. The interpolation unit 118 stores the image with the corrected white balance in the storage unit 141 as the final captured image.

[0046] According to the white balance correction device of the embodiment, even if two points are close to each other in the image plane, if there is a difference in the depth direction in the actual shooting situation, the points are less likely to be affected by the more distant light source, so that more natural white balance correction can be performed. According to the white balance correction device of the embodiment, more natural white balance correction can be performed, especially near the boundary between areas.

[0047] (Other embodiments) In the above-described embodiment, the white balance correction value C of an arbitrary point p other than the reference point is calculated by the above formula (1). However, as in other embodiments described below, the white balance correction value C of an arbitrary point p other than the reference point may be interpolated using a correction value table.

[0048] FIG. 8 is an example of a correction value table according to another embodiment.

[0049] In the correction value table 801, a plurality of ranks are associated with a plurality of correction values ​​C. In the correction value table 801, a plurality of correction values ​​C are associated with a plurality of ranks according to the distance of an arbitrary point p from a first reference point P1 and the distance from a second reference point P2. For example, the correction value table 801 in FIG. 6 is divided into five ranks, and a correction value C corresponding to each rank is described. The correction value table 801 is stored in the storage unit 141.

[0050] The rank indicates, for example, the degree to which the point p is closer to the first reference point P1 or the second reference point P2. For example, the point p is classified based on a preset criterion, and the closer the point p is to the first reference point P1, the lower the rank, and the closer the point p is to the second reference point P2, the higher the rank.

[0051] The correction value C is a white balance correction value. For example, in the correction value table 801, rank 1 corresponds to the white balance correction value C1 of the reference point P1. rank 5 corresponds to the white balance correction value C2 of the reference point P2. rank 3 corresponds to the average value of the white balance correction value C1 of the reference point P1 and the white balance correction value C2 of the reference point P2. rank 2 corresponds to (3*C1+C2) / 4. rank 4 corresponds to (C1+3*C2) / 4.

[0052] 4, the interpolation unit 117 selects a rank corresponding to the arbitrary point p from among the multiple ranks based on the first planar distance L1, the second planar distance L2, the first depth distance D1, and the second depth distance D2 of the arbitrary point p. The interpolation unit 117 acquires a correction value corresponding to the selected rank from the correction value table 801 as a white balance correction value for the arbitrary point.

[0053] For example, when the first planar distance L1 of an arbitrary point p is smaller than the second planar distance L2 and the first depth distance D1 of the point p is smaller than the second depth distance D2, the interpolation unit 117 selects rank 1 as the rank corresponding to the point p, and acquires the correction value C1 corresponding to rank 1 as the white balance correction value of the point p. When the reference point P1 is closer to the arbitrary point p than the reference point P2 in both the planar direction and the depth distance, the white balance correction value C1 of the reference point P1 is used as the white balance correction value of the point p.

[0054] Also, for example, when the first planar distance L1 of an arbitrary point p is greater than the second planar distance L2 and the first depth distance D1 of the point p is greater than the second depth distance D2, the interpolation unit 117 selects rank 5 as the rank corresponding to the point p, and acquires the correction value C2 corresponding to rank 5 as the white balance correction value of the point p. When the reference point P2 is closer to the arbitrary point p than the reference point P1 in both the planar direction and the depth distance, the white balance correction value C2 of the reference point P2 is used as the white balance correction value of the point p.

[0055] Also, for example, when the first planar distance L1 of an arbitrary point p is smaller than the second planar distance L2 and the first depth distance D1 of the point p is larger than the second depth distance D2, the interpolation unit 117 selects rank 3 as the rank corresponding to the point p, and acquires the correction value (C1+C2) / 2 corresponding to rank 3 as the white balance correction value of the point p. When the reference point 1 is closer to the arbitrary point p than the reference point P2 in the planar direction and the reference point P2 is closer to the reference point P1 in the depth distance, the white balance correction value of the point p uses the average value of the white balance correction value C1 of the reference point P1 and the white balance correction value C2 of the reference point P2.

[0056] In addition, for example, in step S204 where the white balance correction value for each reference point is calculated, the interpolation unit 117 calculates the correction value corresponding to each rank, writes the correction value corresponding to each rank in the correction value table 801, and stores it in the memory unit 141.

[0057] According to a white balance correction device of another embodiment, by using a correction value table in which white balance correction values ​​are recorded, the process of calculating the white balance correction values ​​is simplified and the processing time can be reduced.

[0058] In addition, in the above embodiment, the index values ​​K1 and K2 used in the above equation (1) for calculating the correction value C are calculated using equations (2) and (3). However, the index values ​​K1 and K2 may be calculated using an index value table as in other embodiments described below.

[0059] FIG. 9 is an example of an index value table according to another embodiment.

[0060] A plurality of index values ​​K corresponding to combinations of a plurality of ranks according to distances in the planar direction and a plurality of ranks according to distances in the depth direction are associated and described in the index value table 901. The index value table 901 is stored in the storage unit 141.

[0061] The leftmost column of index value table 901 indicates rank L of planar distance, and the top row of index value table 901 indicates rank D of depth distance.

[0062] The rank L of the planar distance indicates the magnitude of the planar distance between an arbitrary point p and a reference point. The larger the planar distance between an arbitrary point p and a reference point, the higher the rank of the planar distance. In the index value table 901, the rank of the planar distance is divided into five ranks.

[0063] The rank D of the depth distance indicates the magnitude of the depth distance between an arbitrary point p and a reference point. The greater the depth distance between an arbitrary point p and the reference point, the higher the rank of the depth distance. In the index value table 901, the rank of the depth distance is divided into five ranks.

[0064] The interpolation unit 117 ranks the planar distance and the depth distance based on a preset criterion. The interpolation unit 117 acquires an index value corresponding to a combination of a rank according to the first planar distance L1 and a rank according to the first depth distance D1 from the index value table 901 as the index value K1. For example, when the rank L of the planar distance according to the first planar distance L1 for an arbitrary point p is 2 and the rank D of the depth distance according to the depth distance D1 is 4, the interpolation unit 117 acquires 5 (indicated by a dotted circle in FIG. 9) as the first index value K1.

[0065] The interpolation unit 117 acquires, from the index value table 901, an index value corresponding to a combination of the rank according to the second planar distance L2 and the rank according to the second depth distance D2 as a second index value K2.

[0066] The interpolation unit 117 calculates a white balance correction value for an arbitrary point p using the above formula (1) based on, for example, the first correction value C1, the second correction value C2, the first index value K1, and the second index value K2.

[0067] According to the white balance correction device of the other embodiment, by using an index value table in which index values ​​are described, the process of calculating the white balance correction value is simplified and the processing time can be shortened. In addition, when the depth distance is classified into a plurality of ranks and an index value table in which index values ​​corresponding to each rank are described is used, it is sufficient to obtain the depth distance with an accuracy high enough to know which rank it corresponds to, so that an inexpensive distance measuring sensor that can obtain only a rough value can be used to measure the depth distance between the white balance correction device and an object in an image.

[0068] (Example of software implementation) The control block (particularly, the control unit 111) of the white balance correction device 101 can be realized by a logic circuit (hardware) formed in an integrated circuit (IC (Integrated Circuit) chip) or the like, or may be realized by software using a processor such as a CPU (Central Processing Unit). In the latter case, for example, the white balance correction device 101, which is a computer, includes a CPU that executes instructions of a program that is software that realizes each function, a ROM or storage device (these are referred to as "recording medium") in which the program and various data are recorded so as to be readable by the computer (or CPU), and a RAM in which the program is expanded. The computer (or CPU) reads the program from the recording medium and executes it, thereby operating as the control unit 111, thereby achieving the object of the present invention. As the recording medium, a "non-transient tangible medium", for example, a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like can be used. The program may be supplied to the computer via any transmission medium that can transmit the program.

[0069] The present invention is not limited to the above-described embodiment and may be modified, and the above configurations may be replaced with substantially the same configurations, configurations that achieve the same effects, or configurations that can achieve the same purpose. [Explanation of symbols]

[0070] 111 Control section 112 Division 113 Reference point calculation section 114 Correction value calculation unit 115 Plane distance calculation section 116 Depth distance calculation unit 117 Interpolation section 118 Correction section 121 Imaging unit 131 Ranging section 141 Storage section 151 Communications Department

Claims

1. a division unit for dividing an image into at least a first region and a second region; a reference point calculation unit that calculates a first reference point in the first region and a second reference point in the second region; a correction value calculation unit that calculates a first correction value of white balance for the first reference point and a second correction value of white balance for the second reference point; a plane distance calculation unit that calculates a first plane distance between the first reference point and an arbitrary point in the image in a plane direction of the image, and a second plane distance between the second reference point and the arbitrary point in the plane direction of the image; a depth distance calculation unit that calculates a first depth distance between the first reference point and the arbitrary point in a depth direction, and a second depth distance between the second reference point and the arbitrary point in a depth direction; an interpolation unit that interpolates a white balance correction value of the arbitrary point based on the first correction value, the second correction value, the first planar distance, the second planar distance, the first depth distance, and the second depth distance; a correction unit that corrects the white balance of the arbitrary point by using a white balance correction value of the arbitrary point; A white balance correction device comprising:

2. The interpolation unit is calculating a first Euclidean distance between the first reference point and the arbitrary point from the first planar distance and the first depth distance; calculating a second Euclidean distance between the second reference point and the arbitrary point from the second planar distance and the second depth distance; calculating a white balance correction value for the arbitrary point based on the first correction value, the second correction value, and a ratio between the first Euclidean distance and the second Euclidean distance; 2. The white balance correction device according to claim 1.

3. a storage unit that stores a correction value table in which a plurality of correction values ​​corresponding to a plurality of ranks according to the distance from the first reference point and the distance from the second reference point are described; The interpolation unit is selecting a rank corresponding to the arbitrary point from among the plurality of ranks based on the first planar distance, the second planar distance, the first depth distance, and the second depth distance; 2. The white balance correction device according to claim 1, wherein a correction value corresponding to the selected rank is obtained from the correction value table as the white balance correction value for the arbitrary point.

4. A storage unit is further provided for storing an index value table in which a plurality of index values ​​corresponding to combinations of a plurality of ranks according to distances in a planar direction and a plurality of ranks according to distances in a depth direction are described, The interpolation unit is obtaining a first index value corresponding to a combination of the rank according to the first planar distance and the rank according to the first depth distance from the index value table; obtaining a second index value corresponding to a combination of the rank according to the second planar distance and the rank according to the second depth distance from the index value table; 2. The white balance correction device according to claim 1, further comprising: a white balance correction value for the arbitrary point calculated based on the first correction value, the second correction value, the first index value, and the second index value.

5. Dividing the image into at least a first region and a second region; Calculating a first reference point within the first region and a second reference point within the second region; Calculating a first correction value of white balance for the first reference point and a second correction value of white balance for the second reference point; Calculating a first planar distance between the first reference point and an arbitrary point in the image in a planar direction of the image, and a second planar distance between the second reference point and the arbitrary point in a planar direction of the image; Calculating a first depth distance between the first reference point and the arbitrary point in a depth direction, and a second depth distance between the second reference point and the arbitrary point in a depth direction; Interpolating a white balance correction value of the arbitrary point based on the first correction value, the second correction value, the first planar distance, the second planar distance, the first depth distance, and the second depth distance; correcting the white balance of the arbitrary point using the white balance correction value of the arbitrary point; A white balance correction method comprising the steps of:

Images