Image processing device and image processing method

Abstract

To provide an image processing device and an image processing method which enable further natural white balance correction.SOLUTION: An image processing device 10 comprises: a distance information acquisition unit which acquires distance information indicating a distance to a subject captured in a patch image from an imaging device which captured an image, about each of the plurality of patch images included in the image; and an image processing unit which divides the image into a plurality of regions, on the basis of a position of a light source in the image, and the distance information, and corrects white balance for each of the plurality of divided regions.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to an image processing device and an image processing method. [Background technology]

[0002] Conventionally, methods for correcting white balance have been known. For example, Patent Document 1 describes a method for dividing an image into a plurality of regions based on distance information and color difference information, and correcting the white balance for each divided region. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2009-212787 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the method described in Patent Document 1, an image is divided into a plurality of regions based on distance information and color difference information, so that the image is divided into a plurality of regions such that the distance in the depth direction is close in each divided region. Therefore, in the method described in Patent Document 1, an image may not necessarily be divided into regions suitable for white balance correction.

[0005] The present disclosure has been made in view of the above-mentioned problems, and an object of the present disclosure is to provide an image processing device and an image processing method capable of performing more natural white balance correction. [Means for solving the problem]

[0006] An image processing device of one embodiment of the present disclosure includes a distance information acquisition unit that acquires, for each of a plurality of patch images included in an image, distance information indicating the distance from an imaging device that captured the image to a subject appearing in the patch image, and an image processing unit that divides the image into a plurality of regions based on a position of a light source in the image and the distance information, and corrects white balance for each of the divided regions.

[0007] An image processing method of one embodiment of the present disclosure is an image processing method executed by an image processing device, which obtains, for each of a plurality of patch images included in an image, distance information indicating the distance from an imaging device that captured the image to a subject appearing in the patch image, divides the image into a plurality of regions based on the position of a light source in the image and the distance information, and corrects the white balance for each of the divided regions. Effect of the Invention

[0008] According to the present disclosure, it is possible to provide an image processing device and an image processing method capable of performing more natural white balance correction. [Brief description of the drawings]

[0009] [Figure 1] 1 is a functional block diagram showing a schematic configuration of an image processing device according to a first embodiment. [Diagram 2] 10 is a flowchart illustrating an example of a correction process executed by the image processing device. [Diagram 3] FIG. 13 is a schematic diagram showing an example of a pre-processing image. [Figure 4] FIG. 13 is a diagram showing a state in which an unprocessed image is divided into a plurality of patch images. [Diagram 5] FIG. 13 is a diagram showing a state in which distance information of each patch image is superimposed on an unprocessed image. [Figure 6] 11 is a histogram showing the relationship between distance information and the number of patches. [Figure 7] FIG. 13 is a diagram showing an image of a divided area 0. [Figure 8]FIG. 2 is a diagram showing an image of a divided area 1. [Figure 9] FIG. 2 is a diagram showing an image of a divided area 2. [Figure 10] FIG. 11 is a functional block diagram showing a schematic configuration of an image processing device according to a second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, some embodiments of the present disclosure will be described with reference to the drawings. In the drawings, the same or equivalent components are denoted by the same reference numerals, and descriptions of the same or equivalent components will be omitted if they are repeated.

[0011] [First embodiment] Fig. 1 is a functional block diagram showing a schematic configuration of an image processing device 10 according to the first embodiment. The image processing device 10 is configured by, for example, a terminal device such as a computer or a smartphone, a server device, or any other device capable of executing the processing described in this embodiment. As shown in Fig. 1, the image processing device 10 according to the first embodiment includes, as functional blocks, an image acquisition unit 101, a distance information acquisition unit 102, an image processing unit 103, and a storage unit 104.

[0012] The image acquisition unit 101 acquires an image. The image acquisition unit 101 may be configured, for example, by a camera that captures an image. In this case, the image acquisition unit 101 acquires the image by capturing the image with the camera. The image acquisition unit 101 may acquire an image, for example, from another functional unit or another device. For example, the image acquisition unit 101 may acquire an image stored in the storage unit 104 from the storage unit 104. Alternatively, for example, the image acquisition unit 101 may be configured by a communication interface that executes information communication with an external device, and may acquire an image from the external device through information communication. In this case, the image acquisition unit 101 can execute information communication by wire or wirelessly. The image acquisition unit 101 may acquire an image via a network such as the Internet. The image acquisition unit 101 may acquire an image by another communication method such as Bluetooth (registered trademark) or wireless LAN (Local Area Network). The image acquisition unit 101 provides the acquired image to the distance information acquisition unit 102 and the image processing unit 103. In addition, the image acquired by the image acquisition unit 101 and in which the white balance has not been corrected is also referred to as a "pre-processing image" in this specification.

[0013] The distance information acquisition unit 102 and the image processing unit 103 can be configured by a control device such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). The control device performs various controls, for example, by running a control program stored in the storage unit 104. In the image processing device 10, independent control devices may be used for the distance information acquisition unit 102 and the image processing unit 103, or one control device may function as the distance information acquisition unit 102 and the image processing unit 103.

[0014] Distance information acquisition unit 102 acquires distance information indicating the distance to a subject appearing in a preprocessing image acquired by image acquisition unit 101. Specifically, distance information acquisition unit 102 divides the preprocessing image provided by image acquisition unit 101 into a plurality of patch images. Then, distance information acquisition unit 102 acquires distance information indicating the distance from an imaging device that captured the preprocessing image to a subject appearing in the patch image for each of the plurality of patch images included in the preprocessing image (i.e., the plurality of divided patch images).

[0015] The division unit for dividing the pre-processed image into a plurality of patch images is, for example, predetermined. The distance information acquisition unit 102 divides the pre-processed image into a plurality of patch images according to the division unit. For example, the distance information acquisition unit 102 divides the pre-processed image into X equal parts horizontally and Y equal parts vertically to divide the image into patch images. X and Y are predetermined integers of 1 or more. In this case, one pre-processed image is divided into (X×Y) patch images. The distance information acquisition unit 102 may divide the pre-processed image in units of pixels. In this case, one divided patch image is an image of one pixel. The distance information acquisition unit 102 may divide the pre-processed image in units of image segmentation. Note that the division unit of the pre-processed image is not limited to the example shown here. The division unit may be any unit for dividing the pre-processed image into a plurality of patch images.

[0016] The distance information acquisition unit 102 can acquire distance information using a known method. For example, the distance information acquisition unit 102 estimates the distance in the depth direction of the subject by machine learning. The distance information acquisition unit 102 can acquire distance information by using the estimated distance in the depth direction as distance information. The distance information acquisition unit 102 acquires distance information for each of the divided multiple patch images. The distance information acquisition unit 102 provides the acquired distance information to the image processing unit 103.

[0017] The image processing unit 103 corrects the white balance of the pre-processed image acquired by the image acquisition unit 101 based on the pre-processed image acquired by the image acquisition unit 101 and the distance information acquired by the distance information acquisition unit 102. Specifically, the image processing unit 103 divides the pre-processed image provided by the image acquisition unit 101 into a plurality of regions based on the position of the light source in the pre-processed image and the distance information acquired from the distance information acquisition unit 102. The image processing unit 103 corrects the white balance for each of the divided regions. The position of the light source is represented by, for example, light source coordinates indicating the position of the light source. In this case, the image processing unit 103 divides the pre-processed image into a plurality of regions based on the light source coordinates and the distance information, and corrects the white balance for each of the plurality of regions.

[0018] The image processing unit 103 generates and outputs an image after the white balance correction is completed by integrating the images for each area in which the white balance has been corrected. In this specification, the image after the white balance correction is completed is also referred to as a "processed image." For example, the image processing unit 103 stores the processed image in the storage unit 104, displays it on a display unit (not shown) such as a display, or transmits it to an external device.

[0019] The distance information acquisition unit 102 and the image processing unit 103 can be configured by a control device such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). The control device performs various controls, for example, by running a control program stored in the storage unit 104. In the image processing device 10, independent control devices may be used for the distance information acquisition unit 102 and the image processing unit 103, or one control device may function as the distance information acquisition unit 102 and the image processing unit 103. Details of the processes (correction processes) executed by the distance information acquisition unit 102 and the image processing unit 103 will be described later.

[0020] The storage unit 104 is a storage medium capable of storing programs and data. The storage unit 104 can be configured with, for example, a semiconductor memory or a magnetic memory. Specifically, the storage unit 104 can be configured with, for example, an EEPROM (Electrically Erasable Programmable Read-Only Memory). The storage unit 104 stores, for example, a program for operating the distance information acquisition unit 102 and the image processing unit 103. Specifically, the storage unit 104 stores, for example, a program for executing a correction process for correcting the white balance of an image. The storage unit 104 may also store a pre-processing image before the white balance is corrected, or a post-processing image after the white balance is corrected.

[0021] Next, a description will be given of details of the correction process executed by the image processing device 10. The correction process is executed by the distance information acquisition unit 102 and the image processing unit 103. FIG 2 is a flowchart showing an example of the correction process executed by the image processing device 10.

[0022] In the correction process, the image processing unit 103 acquires light source coordinates in the pre-processed image provided from the image acquisition unit 101 (step S11). Specifically, the image processing unit 103 identifies coordinates indicating the position of the light source in the pre-processed image. The position of the light source can be identified by a known method. For example, the image processing unit 103 can identify the position in the pre-processed image where the luminance is the highest as the position of the light source. The coordinates indicating the position of the light source may be indicated, for example, by the position of a patch image in the pre-processed image, or may be indicated by the position of a pixel in the pre-processed image, or may be indicated using a coordinate system different from these.

[0023] Fig. 3 is a schematic diagram showing an example of an unprocessed image. In the unprocessed image shown in Fig. 3, a person is shown in the lower center, and there are walls on the left and right of the image. There is also a recessed space in the center of the image, and the person in the lower center of the image is located in front of the recessed space.

[0024] Lighting fixture 30 attached to the ceiling is seen in the upper center of the pre-processing image shown in Fig. 3. In the pre-processing image, lighting fixture 30 is turned on, so the brightness of lighting fixture 30 and its surroundings is higher than other parts. In this case, in step S11, image processing unit 103 identifies the coordinates indicating the position of lighting fixture 30 in the pre-processing image as light source coordinates.

[0025] Meanwhile, distance information acquisition unit 102 divides the pre-processed image provided from image acquisition unit 101 into a plurality of patch images (step S12). Distance information acquisition unit 102 divides the pre-processed image into a plurality of patch images according to a predetermined division unit, for example, as described above. In this embodiment, distance information acquisition unit 102 divides the pre-processed image into 10 equal parts horizontally and 8 equal parts vertically, thereby dividing the image into a total of 80 patch images.

[0026] Fig. 4 is a diagram showing a state in which a pre-processing image is divided into a plurality of patch images. In Fig. 4, the boundaries between the patches are indicated by dashed lines. For example, as shown in Fig. 4, the distance information acquisition unit 102 divides the pre-processing image into 80 patch images by dividing it into 10 equal parts horizontally and 8 equal parts vertically.

[0027] The distance information acquisition unit 102 acquires distance information for each divided patch image (step S13). In this example, the distance information acquisition unit 102 acquires distance information for each of the 80 patch images shown in Fig. 4. The distance information acquisition unit 102 acquires the distance information by estimating the distance of the subject in the depth direction by machine learning, for example.

[0028] Fig. 5 is a diagram showing a state in which distance information of each patch image is superimposed on a pre-processing image. Note that Fig. 5 is shown for the purpose of explanation, and the distance information acquisition unit 102 does not necessarily need to generate a diagram as shown in Fig. 5. In Fig. 5, the numbers shown in each patch are values ​​indicating distance information. The larger this value is, the longer the distance from the imaging device that captured the pre-processing image to the subject that appears in the patch image is, and the shorter this value is, the shorter the distance from the imaging device that captured the pre-processing image to the subject that appears in the patch image is.

[0029] In the example shown in FIG. 5, the distance information of the patch image in which a person appears is the smallest, "1.0," among the multiple patch images shown in FIG. 5. In other words, the person is located at the forefront of the subjects in the pre-processing image. The distance information of the patch images in which the left and right walls of the image are located indicates a value between "1.7" and "2.3." The distance information of the patch image in which the recessed space in the center of the image appears indicates a value larger than the other patch images, with a maximum of "4.0." The distance information acquisition unit 102 provides the distance information of each patch image to the image processing unit 103.

[0030] Note that step S11, step S12, and step S13 do not necessarily have to be executed in the order shown in Fig. 2. Step S11 may be executed after step S12 and step S13, and step S11, step S12, and step S13 may be executed in parallel.

[0031] Image processing unit 103 divides the preprocessed image into multiple (n) regions based on the light source coordinates acquired in step S11 and the distance information acquired in step S143 (step S14). At this time, image processing unit 103 divides the preprocessed image so that patch images with similar distance information are included in the same region. For example, image processing unit 103 divides the preprocessed image so that patch images with distance information included within a predetermined range are included in the same region. The predetermined range may be determined appropriately depending on the calculation or display method of the value of the distance information, etc.

[0032] Furthermore, the image processing unit 103 divides the pre-processed image so that a patch image having a predetermined relationship with the patch image including the light source is included in the same region as the patch image including the light source. The predetermined relationship is a relationship in which it is preferable to perform the same correction as the patch image including the light source from the viewpoint of white balance correction. For example, the predetermined relationship is a relationship in which the patch image is directly or indirectly adjacent to the patch image including the light source and the distance information of the patch image including the light source is included within a predetermined range.

[0033] The number n of regions to be divided may be predetermined, or may be appropriately determined for each pre-processed image according to the light source coordinates and distance information. When the number n of regions to be divided is predetermined, the image processing unit 103 determines the above-mentioned predetermined range so that the number of regions after division is n, and divides the pre-processed image into n regions so that the patch images included in the determined predetermined range are included in the same region according to the distance information. When the number n of regions after division is not determined, the image processing unit 103 divides the pre-processed image into n regions based on the light source coordinates and distance information.

[0034] Here, further details of the process executed in step S14 will be described. FIG. 6 is a histogram showing the relationship between distance information and the number of patches. Specifically, FIG. 6 shows the relationship between the value indicated by the distance information and the number of patches taking each value, based on the distance information of each patch image shown in FIG. 5. In the histogram in FIG. 6, the horizontal axis indicates the value of the distance information, and the vertical axis indicates the number of patches. For example, in FIG. 5, there are 14 patch images with distance information of "1.0". Therefore, in FIG. 6, the number of patches with distance information of "1.0" is "14". The same applies to other values ​​of distance information.

[0035] The image processing unit 103 divides the preprocessed image so that patch images whose distance information falls within a predetermined range are included in the same region. Here, it is assumed that the predetermined range is "0.5". When dividing the preprocessed image, the image processing unit 103 determines distance information that is to be a reference for the predetermined range. For example, the image processing unit 103 refers to the histogram shown in FIG. 6 and determines the reference distance information so that all of the multiple patches can be included in any of the multiple regions. Here, it is assumed that the image processing unit 103 determines "1.0", "2.0", and "3.5" as the reference distance information.

[0036] The image processing unit 103, for example, includes in the same region patch images included within a predetermined range (0.5 to 1.5) from the reference distance information "1.0". In this example, as shown in FIG. 6, the patch images within the predetermined range (0.5 to 1.5) from the distance information "1.0" are 14 patch images with distance information "1.0". Therefore, the image processing unit 103 determines the 14 patch images as one region (region 0). Similarly, the image processing unit 103 includes in the same region patch images included within a predetermined range (1.5 to 2.5) from the reference distance information "2.0". In this example, as shown in FIG. 6, the patch images within the predetermined range (1.5 to 2.5) from the distance information "2.0" are a total of 48 patch images with distance information "1.7", "2.0", and "2.3". Therefore, the image processing unit 103 determines the 48 patch images as one region (region 1). Furthermore, the image processing unit 103 includes, for example, patch images whose reference distance information is within a predetermined range (3.0 to 4.0) from "3.5" in the same region. In this example, as shown in FIG. 6, the patch images whose distance information is within a predetermined range (3.0 to 4.0) from "3.5" are a total of 18 patch images whose distance information is "3.0", "3.5", "3.8", and "4.0". Therefore, the image processing unit 103 determines the 18 patch images as one region (region 2). In this way, the image processing unit 103 divides the pre-processing image into three regions, region 0 to region 2.

[0037] The image processing unit 103 divides the pre-processed image so that a patch image having a predetermined relationship with the patch image including the light source is included in the same region as the patch image including the light source. For example, the predetermined relationship is a relationship in which the patch image is directly or indirectly adjacent to the patch image including the light source and the distance information of the patch image including the light source is included within a range of "0.5". In this case, since the distance information of the patch image including the light source is "3.5", the image processing unit 103 includes in the same region the patch image that is directly or indirectly adjacent to the patch image including the light source and has distance information in the range of 3.0 to 4.0. In this example, even by this division method, region 2 is determined as one divided region.

[0038] 7, 8, and 9 are diagrams showing images of divided regions 0, 1, and 2, respectively. Region 0, as shown in FIG. 7, is composed of a patch image in which a person is captured, located at the lower center of the pre-processing image. Region 1, as shown in FIG. 8, is composed of patch images in which wall surfaces are captured, located on the left and right sides of the pre-processing image. Region 2, as shown in FIG. 9, is composed of a patch image in which the ceiling is captured, located at the upper center of the pre-processing image, and a patch image in which the rear wall surface surrounding the person is captured.

[0039] 2 again, the image processing unit 103 divides the pre-processed image into n regions, and then corrects the white balance for each of the n regions. Specifically, it is determined whether or not a light source is present in region i (i is an integer equal to or greater than 0 and less than n) (step S15). The image processing unit 103 can determine whether or not a light source is present in region i, for example, based on whether or not a patch image including the specified light source coordinates is included in region i.

[0040] 7 to 9, in this example, region 0 and region 1 do not include patch images including light source coordinates, and region 2 includes patches including light source coordinates. Therefore, the image processing unit 103 determines that no light source exists in region 0 and region 1, and determines that a light source exists in region 2.

[0041] The image processing unit 103 determines a correction coefficient for each of a plurality of regions. In this example, since there are three regions, the image processing unit 103 determines a correction coefficient for each of the three regions. Here, the correction coefficient is a reference coefficient used for correcting the white balance, and can be a known coefficient. For example, the correction coefficient may include any of color temperature, gain, and chromaticity. However, the correction coefficient is not limited to the example shown here, and other indices may be used. The image processing unit 103 can calculate the correction coefficient by a known method.

[0042] The image processing unit 103 sets a correction coefficient based on either the color of the entire region or the color of the light source depending on whether or not a light source is present in the region i. Specifically, the image processing unit 103 determines the correction coefficient for a region that does not include a patch image including a light source based on the color of the entire region. On the other hand, the image processing unit 103 determines the correction coefficient for a region that includes a patch image including a light source based on the color of the light source.

[0043] 2, when it is determined that no light source is present in region i (No in step S15), image processing unit 103 sets a correction coefficient based on the color of the entire region (step S16). That is, when no light source is present in region i, region i is an region that does not include a patch image that includes a light source, so image processing unit 103 determines the correction coefficient based on the color of the entire region. For example, for region 0 and region 1 where no light source is present, image processing unit 103 calculates the color temperature of the region as a correction coefficient based on the color of the entire region.

[0044] On the other hand, if it is determined that a light source is present in region i (Yes in step S15), image processing unit 103 sets a correction coefficient based on the color of the light source (step S17). That is, if a light source is present in region i, region i is a region that includes a patch image that includes a light source, so image processing unit 103 determines the correction coefficient based on the color of the light source. For example, for region 2 where a light source is present, image processing unit 103 calculates the color temperature of the region as the correction coefficient based on the color of the light source.

[0045] The image processing unit 103 corrects the white balance based on the correction coefficient. Specifically, the image processing unit 103 corrects the white balance of the area i based on the correction coefficient set in step S16 or step S17 (step S18). In this example, the image processing unit 103 corrects the white balance for the areas 0 and 1 using a correction coefficient set based on the color of the entire area. Furthermore, the image processing unit 103 corrects the white balance for the area 2 using a correction coefficient set based on the color of the light source. In this way, an image after the white balance correction is obtained for each area.

[0046] The image processing unit 103 repeatedly executes the loop process shown in steps S15 to S18 for all regions. For example, the image processing unit 103 executes the loop process shown in steps S15 to S18 for region i (i.e. region 0) with i=0, then increments i by one and executes the loop process shown in steps S15 to S18 for the incremented region i. When i=n, ​​the image processing unit 103 ends the loop process shown in steps S15 to S18. In this example, the pre-processed image is divided into three regions, so n=3. Therefore, when i=3, the image processing unit 103 ends the loop process.

[0047] The image processing unit 103 integrates the n regions into one image (step S19). Here, the images of the three regions obtained by the loop processing shown in steps S15 to S18 are integrated into one image. This generates a processed image with corrected white balance.

[0048] In this manner, in this embodiment, the image processing unit 103 divides the pre-processed image into a plurality of regions based on the position of the light source and the distance information, and corrects the white balance for each of the plurality of regions. Therefore, it is possible to use a different white balance correction method for each region. In other words, since it is possible to correct the white balance for each region using an appropriate correction method according to the image captured in the region, more natural white balance correction is possible.

[0049] In particular, in the above embodiment, the image processing unit 103 sets a correction coefficient based on either the color of the entire region or the color of the light source depending on whether or not a light source is present in each region, and corrects the white balance based on the set correction coefficient. Therefore, the white balance is corrected using a correction coefficient appropriate for each case, whether or not a light source is present in the region. This allows for more natural white balance correction.

[0050] In the above embodiment, a case where the lighting fixture 30, which is a light source, is shown in the image has been described. However, the light source does not necessarily have to be shown in the image. For example, even if the light source is not directly shown in the image, when the light from the light source is reflected by a subject shown in the image and the area where the light from the light source is reflected is brighter than other areas, the image processing device 10 can correct the white balance by executing the same process as that described in the above embodiment. In this case, more natural white balance correction is possible for the same reasons as in the above embodiment.

[0051] [Second embodiment] 10 is a functional block diagram showing a schematic configuration of an image processing device 20 according to the second embodiment. The image processing device 20 according to the second embodiment will be described while appropriately omitting explanations of similarities with the image processing device 10 according to the first embodiment.

[0052] As shown in FIG. 10, the image processing device 20 according to the second embodiment includes, as functional blocks, an image acquisition unit 201, a distance information acquisition unit 202, an image processing unit 203, and a storage unit 204.

[0053] The image acquisition unit 201 in the second embodiment acquires an image in the same manner as the image acquisition unit 101 in the first embodiment. However, in the second embodiment, the image acquisition unit 201 differs from the image acquisition unit 101 in the first embodiment in that the image acquisition unit 201 does not provide the acquired image to the distance information acquisition unit 202 but provides it to the image processing unit 203. In the second embodiment, the image acquisition unit 201 is configured by, for example, a camera that captures an image. That is, the image acquisition unit 201 in the second embodiment acquires an image by capturing the image with a camera.

[0054] The distance information acquisition unit 202 acquires distance information indicating the distance to the subject shown in the pre-processing image acquired by the image acquisition unit 201. In the second embodiment, the distance information acquisition unit 202 is configured with a distance measurement sensor that operates independently of the image acquisition unit 201. The distance information acquisition unit 202 acquires distance information by measuring the distance to the subject using the distance measurement sensor. For example, the distance information acquisition unit 202 is configured with a ToF sensor that measures distance using a time-of-flight method. The ToF sensor can measure distance using, for example, a laser or ultrasonic waves. However, the distance information acquisition unit 202 is not limited to the example described here, and can be configured with a device that can acquire distance information using other methods. As an example, the distance information acquisition unit 202 may be configured with a stereo camera that can calculate distance based on parallax.

[0055] The functions and configurations of the image processing unit 203 and the storage unit 204 may be similar to those of the image processing unit 103 and the storage unit 104 described in the first embodiment, and therefore detailed description thereof will be omitted here.

[0056] The image processing device 20 according to the second embodiment can also correct the white balance using a method similar to that described in the first embodiment. Therefore, the image processing device 20 according to the second embodiment can also perform more natural white balance correction.

[0057] Furthermore, in the second embodiment, for example, instead of estimating distance information from an image, the distance to the subject can be directly measured by the distance information acquisition unit 202. Therefore, the image processing device 20 can acquire distance information to the subject more accurately. This allows the image processing device 20 to perform area division based on the actual arrangement of the subject, and ultimately to perform more appropriate white balance correction in accordance with the actual arrangement of the subject.

[0058] Although the present disclosure has been described based on the drawings and embodiments, it should be noted that those skilled in the art can easily make various modifications and corrections based on the present disclosure. Therefore, it should be noted that these modifications and corrections are included in the scope of the present disclosure. For example, functions included in each functional unit or each step can be rearranged so as not to be logically inconsistent, and multiple functional units or steps can be combined or divided into one. [Explanation of symbols]

[0059] 10,20 Image processing device 101,201 Image acquisition unit 102,202 Distance information acquisition unit 103,203 Image Processing Unit 104,204 Storage section 30 Lighting equipment

Claims

1. a distance information acquisition unit that acquires, for each of a plurality of patch images included in an image, distance information indicating a distance from an imaging device that captured the image to a subject depicted in the patch image; an image processing unit that divides the image into a plurality of regions based on a position of a light source in the image and the distance information, and corrects white balance for each of the divided regions; An image processing device comprising:

2. The image processing device according to claim 1 , wherein when dividing the image into the plurality of regions, the image processing unit includes patch images whose distance information is within a predetermined range in the same region.

3. The image processing device according to claim 1 , wherein when dividing the image into the plurality of regions, the image processing unit includes a patch image having a predetermined relationship with the patch image including the light source in the same region as the patch image including the light source.

4. The image processing device according to claim 1 , wherein the image processing unit determines a correction coefficient for each of the plurality of regions, and corrects the white balance based on the correction coefficient.

5. The image processing unit includes: determining the correction coefficient for a region including a patch image including the light source based on a color of the light source; and determining the correction coefficient for another region not including the patch image including the light source based on the color of the entirety of the other region. The image processing device according to claim 4.

6. The image processing device according to claim 4 , wherein the correction coefficient includes any one of a color temperature, a gain, and a chromaticity.

7. The image processing device according to claim 1 , wherein the distance information acquisition unit is a distance sensor.

8. An image processing method executed by an image processing device, comprising: acquiring distance information indicating a distance from an imaging device that captured the image to a subject depicted in the patch image for each of a plurality of patch images included in the image; Dividing the image into a plurality of regions based on a position of a light source in the image and the distance information, and correcting white balance for each of the plurality of divided regions. Image processing methods.

Images