Image processing device and image processing method

Abstract

To efficiently generate an image even under a situation that halation occurs in imaging an imaging object.SOLUTION: An imaging device 23 is connected to an image processing device 20. The imaging device 23 includes an imaging part 24 and a complement image imaging part 25. The complement image imaging part 25 includes an imaging part 25a and a wavelength restriction filter 25b for transmitting light of a specific wavelength band and blocking light of the other wavelength bands. The imaging part 24 and the imaging part 25a are cameras capable of imaging light of a plurality of wavelength bands of visible light. The image processing device 20 corrects a standard image on the basis of the standard image obtained by imaging a plurality of imaging objects having different relative reflectances by using the imaging part 24 for imaging a color image and a complement image obtained by imaging the plurality of imaging objects having different relative reflectances by using the complement image imaging part 25 for imaging transmitted light transmitted through the wavelength restriction filter 25b for transmitting only the light of the specific wavelength band.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an image processing apparatus and an image processing method capable of quickly and efficiently correcting an image with blooming when imaging an imaging object.

Background Art

[0002] Conventionally, when light is irradiated from a light source onto the surface of an imaging object and the reflected wave is imaged by an imaging device, a blooming phenomenon may occur in a part of the imaged image. If this blooming phenomenon occurs, the image quality of the blooming part deteriorates.

[0003] For this reason, techniques for preventing deterioration of the image quality of the blooming part are known. For example, Patent Document 1 discloses an image synthesis method in which a plurality of images of an imaging object are imaged while changing the exposure time between the shortest exposure time and the longest exposure time at which blooming cannot occur, and then synthesized.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, the method of Patent Document 1 has a problem that it takes time to synthesize images because a plurality of images of an imaging object must be imaged while changing the exposure time, and thus it is not efficient.

[0006] The present invention has been made to solve the above problems of the prior art, and an object of the present invention is to provide an image processing apparatus and an image processing method capable of quickly and efficiently correcting an image with blooming when imaging an imaging object.

Means for Solving the Problems

[0007] In order to solve the above-described problems and achieve the object, the present invention includes: a first imaging unit that receives a reflected wave obtained by reflecting light irradiated from a predetermined light source onto an imaging object by the imaging object and forms a reference image; a second imaging unit that receives light in a predetermined wavelength band included in the reflected wave and forms a complementary image; and a correction processing unit that corrects the reference image based on the complementary image.

[0008] Further, in the present invention, the second imaging unit has a wavelength limiting filter that transmits light in a predetermined wavelength band and blocks light in other wavelength bands.

[0009] Further, in the present invention, the wavelength band of the light to be transmitted by the wavelength limiting filter is determined according to the color of the paint applied to the surface of the imaging object or the material of the surface.

[0010] Further, in the present invention, when the pixel value of a pixel forming the reference image exceeds a predetermined threshold value, the correction processing unit uses the pixel value of the pixel of the complementary image located at the pixel position of the pixel to correct the pixel value of the pixel forming the reference image.

[0011] Further, in the present invention, when the pixel values of a plurality of pixels forming a predetermined region of the reference image each exceed a predetermined threshold value, the correction processing unit uses the pixel values of the pixels corresponding to the predetermined region of the complementary image to correct the pixel values of the pixels forming the predetermined region.

[0012] The present invention also relates to an image processing method executed by an image processing apparatus that corrects an image captured by a predetermined imaging unit. The method includes a reference image forming step of forming a reference image by receiving, with a first imaging unit, a reflected wave of light irradiated from a predetermined light source onto an imaging object and reflected by the imaging object; a complementary image forming step of forming a complementary image by receiving, with a second imaging unit, light in a predetermined wavelength band included in the reflected wave; and a correction processing step of correcting the reference image based on the complementary image.

Effects of the Invention

[0013] According to the present invention, when imaging an imaging object, it is possible to quickly and efficiently correct an image with whiteout.

Brief Description of the Drawings

[0014]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Embodiments for Carrying Out the Invention

[0015] Hereinafter, embodiments of an image processing apparatus and an image processing method according to the present invention will be described in detail with reference to the drawings.

[0016] <Overview of Image Processing Apparatus 20> The outline of the image processing apparatus 20 according to this embodiment will be described. FIG. 1 is a diagram showing the outline of the image processing apparatus 20 according to the embodiment. The image processing apparatus 20 according to this embodiment corrects a standard image based on a standard image obtained by imaging a plurality of imaging objects having different relative reflectivities using an imaging unit 24 that captures a color image, and a complementary image obtained by imaging transmitted light that has passed through a wavelength-limiting filter 25b that transmits only light in a specific wavelength band. Thus, the apparatus can efficiently correct an image even in a situation where white bleeding occurs in the standard image.

[0017] As shown in FIG. 1(a), an imaging device 23 is connected to the image processing apparatus 20. The imaging device 23 includes an imaging unit 24 and a complementary image imaging unit 25. The complementary image imaging unit 25 includes an imaging unit 25a and a wavelength-limiting filter 25b that transmits light in a specific wavelength band and blocks light in other wavelength bands. The imaging unit 24 and the imaging unit 25a are cameras capable of imaging light in a plurality of wavelength bands of visible light. The wavelength-limiting filter 25b is an optical filter that passes light in a specific wavelength band among the light in the wavelength band of visible light and blocks light in other wavelength bands.

[0018] The image processing apparatus 20 captures a standard image with the imaging unit 24 (S1). Then, the complementary image imaging unit 25 captures a complementary image (S2). Thereafter, the image processing apparatus 20 corrects the standard image based on the standard image and the complementary image (S3).

[0019] For example, as shown in FIG. 1(b), the imaging objects 50 and 52 have a large relative reflectivity on the surface, and the imaging object 51 has a small surface reflectivity. When imaging the imaging objects 50, 51, and 52 simultaneously and using an auxiliary light source such as a flash (not shown) to irradiate auxiliary light for imaging, the standard image data 26a exhibits a phenomenon of white bleeding where the images of the imaging objects 50 and 52 with a large surface reflectivity are white and missing, and the shapes of the imaging objects 50 and 52 are not captured.

[0020] On the one hand, since the complementary image data 26b captured by the complementary image capturing unit 25 captures an image in a specific wavelength band among the auxiliary light (white light), the imaging objects 50 and 52 can be imaged without whiteout. Then, by correcting the pixels with whiteout in the standard image data 26a with the pixels in the complementary image data 26b and correcting the standard image, it is possible to efficiently correct the images of imaging objects with different surface reflectivities.

[0021] <Configuration of the image processing apparatus 20> Next, the configuration of the image processing apparatus 20 shown in FIG. 1 will be described. FIG. 2 is a functional block diagram showing the configuration of the image processing apparatus 20 shown in FIG. 1. As shown in FIG. 2, the image processing apparatus 20 includes a storage unit 26 and a control unit 27, and a display unit 21, an input unit 22, and an imaging device 23 are connected thereto. The display unit 21 is a display device such as a liquid crystal display that displays various information. The input unit 22 is an input device such as a mouse or a keyboard.

[0022] The imaging device 23 includes an imaging unit 24 and a complementary image capturing unit 25, and the complementary image capturing unit 25 includes an imaging unit 25a and a wavelength limiting filter 25b. The imaging unit 24 and the imaging unit 25a are cameras capable of imaging light in a plurality of wavelength bands of visible light using an image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal - Oxide - Semiconductor).

[0023] The wavelength limiting filter 25b is an optical filter that transmits light in a specific wavelength band among the light in the wavelength band of white light and blocks light in other wavelength bands. The wavelength limiting filter 25b is realized by applying an optical thin film to optical glass or colored glass.

[0024] The storage unit 26 is a storage device such as a hard disk drive or a non-volatile memory, and stores standard image data 26a, complementary image data 26b, and corrected image data 26c. The standard image data 26a is data of a color image captured using the imaging unit 24. The complementary image data is data of an image captured by transmitting light in a specific wavelength band out of the light in the wavelength band of white light using the complementary image capturing unit 25a of the imaging device 23 and blocking light in other wavelength bands. The corrected image data 26c is data of an image corrected based on the standard image data 26a and the complementary image data 26b.

[0025] The control unit 27 is a control unit that controls the entire image processing apparatus 20, and includes an imaging processing unit 27a and a correction processing unit 27b. Actually, by loading these programs into the CPU and executing them, processes corresponding to the imaging processing unit 27a and the correction processing unit 27b are respectively executed.

[0026] The imaging processing unit 27a is a processing unit that captures an imaging object using the imaging unit 24 of the imaging device 23, stores it in the storage unit 26 as the standard image data 26a, and captures a complementary image of the imaging object using the complementary image capturing unit 25 of the imaging device 23 and stores it in the storage unit 26 as the complementary image data 26b.

[0027] The correction processing unit 27b is a processing unit that corrects the standard image based on the standard image data 26a and the complementary image data 26b. For example, the luminance is calculated using the pixel values (R, G, B) of all the pixels of the color image of the standard image data 26a. Here, for the calculation of the luminance, the formula luminance = 0.299R + 0.587G + 0.114B is used with the NTSC (National Television Standard Committee) coefficients.

[0028] Then, the luminance of all pixels in the complementary image data 26b is obtained using the pixel values of all pixels in a specific wavelength band of the complementary image data 26b. Here, when the wavelength-limiting filter 25b transmits light in the blue wavelength band and blocks light in other wavelength bands, the luminance of the complementary image data 26b is the value of the blue pixel value B of the pixels in the complementary image data 26b as the luminance.

[0029] After that, the correction processing unit 27b performs template matching of the image of the complementary image data 26b using the image of the standard image data 26a as a template to match the pixels of the imaging object imaged in the standard image data 26a with the pixels of the imaging object imaged in the complementary image data 26b.

[0030] Then, for all pixels in the standard image data 26a, when the pixel value (luminance) exceeds a predetermined threshold value, the pixel value of the corresponding pixel in the complementary image data 26b is replaced to correct the standard image. Also, when the pixel value (luminance) of the pixels forming a predetermined region in the standard image data 26a exceeds a predetermined threshold value, the pixel value (luminance) of the pixels forming the standard image may be corrected using the pixel value (luminance) of the corrected image.

[0031] <Suppression of blooming> Next, suppression of blooming during imaging of the complementary image data 26b will be described. Here, the case where the wavelength-limiting filter 25b is a filter that transmits light in the blue wavelength band and blocks light in other wavelength bands will be described. FIG. 3 is an explanatory diagram for explaining suppression of blooming during imaging.

[0032] As shown in FIG. 3(a), when imaging an imaging object 53a of color C1 using the imaging unit 24, the imaging object 53a is irradiated with white light (R0, G0, B0). Then, the imaging object 53a absorbs the green absorption light G2 and the blue absorption light B2 corresponding to the color C1 with respect to the white light, and the red reflected light R1, the green reflected light G1, and the blue reflected light B1 corresponding to the color C1 are reflected on the surface of the imaging object 53a, and the imaging unit 24 images the reflected light.

[0033] Then, as shown in Fig. 3(b), when imaging the imaging object 53b with color C2 whose surface color is different from that of the imaging object 53a in Fig. 3(a) using the imaging unit 24, similar to Fig. 3(a), the imaging object 53b is irradiated with white light (R0, G0, B0). Then, the imaging object 53b absorbs the green absorption light G2 and the blue absorption light B2 corresponding to the color C2 with respect to the white light, and the red reflected light R1, green reflected light G1, and blue reflected light B1 corresponding to the color C2 are reflected on the surface of the imaging object 53b, and the imaging unit 24 images the reflected light.

[0034] Then, as shown in Fig. 3(c), when imaging the imaging object 53c with white color C3 using the imaging unit 24, similar to Fig. 3(a), the imaging object 53c is irradiated with white light (R0, G0, B0). Then, since the color C3 is white, the red reflected light R1, green reflected light G1, and blue reflected light B1 corresponding to the color C3 are reflected on the surface of the imaging object 53b, and the imaging unit 24 images the reflected light.

[0035] Here, when the pixel values of the red reflected light R1, green reflected light G1, and blue reflected light B1 of the imaging object 53a with color C1 are 255, 50, and 25 respectively, the luminance is calculated as 108. When the pixel values of the red reflected light R1, green reflected light G1, and blue reflected light B1 of the imaging object 53b with color C2 are 255, 130, and 50 respectively, the luminance is calculated as 158. When the pixel values of the red reflected light R1, green reflected light G1, and blue reflected light B1 of the imaging object 53c with color C3 (white) are 255, 255, and 255 respectively, the luminance is calculated as 255.

[0036] On the other hand, in the case of the complementary image data 26b obtained by imaging the imaging object 53a of color C1 with the imaging unit 25a via the wavelength-limiting filter 25b as shown in FIG. 3(d), the imaging object 53a is irradiated with white light (R0, G0, B0). Then, the imaging object 53a absorbs the green absorption light G2 and the blue absorption light B2 corresponding to the color C1 with respect to the white light, and the red reflected light R1, the green reflected light G1, and the blue reflected light B1 corresponding to the color C1 are reflected on the surface of the imaging object 53a. Only the blue reflected light B1 passes through the wavelength-limiting filter 25b, and the green reflected light G1 and the red reflected light R1 are blocked by the wavelength-limiting filter 25b. Therefore, the imaging unit 25a images the blue reflected light B1.

[0037] Then, as shown in FIG. 3(e), when imaging the imaging object 53b of color C2, whose surface color is different from that of the imaging object 53a in FIG. 3(d), using the imaging unit 25a of the complementary image imaging unit 25, similar to FIG. 3(d), the imaging object 53b is irradiated with white light (R0, G0, B0). Then, the imaging object 53b absorbs the green absorption light G2 and the blue absorption light B2 corresponding to the color C2 with respect to the white light, and the red reflected light R1, the green reflected light G1, and the blue reflected light B1 corresponding to the color C2 are reflected on the surface of the imaging object 53b. Only the blue reflected light B1 passes through the wavelength-limiting filter 25b, and the green reflected light G1 and the red reflected light R1 are blocked by the wavelength-limiting filter 25b. Therefore, the imaging unit 25a images the blue reflected light B1.

[0038] Then, as shown in FIG. 3(f), when imaging the imaging object 53c of white color C3 using the imaging unit 25a of the complementary image imaging unit 25, similar to FIG. 3(d), the imaging object 53c is irradiated with white light (R0, G0, B0). Then, since the color C3 is white, the red reflected light R1, the green reflected light G1, and the blue reflected light B1 corresponding to the color C3 are reflected on the surface of the imaging object 53b. Only the blue reflected light B1 passes through the wavelength-limiting filter 25b, and the green reflected light G1 and the red reflected light R1 are blocked by the wavelength-limiting filter 25b. Therefore, the imaging unit 25a images the blue reflected light B1.

[0039] Here, when the pixel values of the red reflected light R1, green reflected light G1, and blue reflected light B1 of the imaging object 53a with color C1 are 255, 50, and 25 respectively, only the blue reflected light B1 is imaged, so the luminance is calculated as 25. When the pixel values of the red reflected light R1, green reflected light G1, and blue reflected light B1 of the imaging object 53b with color C2 are 255, 130, and 50 respectively, only the blue reflected light B1 is imaged, so the luminance is calculated as 50. When the pixel values of the red reflected light R1, green reflected light G1, and blue reflected light B1 of the imaging object 53c with color C3 (white) are 255, 255, and 255 respectively, only the blue reflected light B1 is imaged, so the luminance is calculated as 255.

[0040] In addition, when imaging an imaging object in the region where the characteristics of the charge capacity with respect to the input intensity of light of the CCD or CMOS sensor used in the imaging unit 24 are linear, the luminance values as described above will be shown. However, when the intensity of the white light irradiated saturates the charge capacity characteristics with respect to the input intensity of light of the sensor, since the luminance difference between the imaging object 53a with color C1, the imaging object 53b with color C2, and the imaging object 53c with color C3 is small, the luminance of these captured images will all become 255, resulting in a washed-out image.

[0041] On the other hand, in the case of the complementary image imaging unit 25, even when the intensity of the white light irradiated saturates the charge capacity characteristics with respect to the input intensity of light of the sensor, since the luminance difference between the imaging object 53a with color C1, the imaging object 53b with color C2, and the imaging object 53c with color C3 is large, the luminance of the captured images of the imaging object 53a with color C1 and the imaging object 53b with color C2 can be captured without saturation.

[0042] Therefore, by replacing the pixel values of the washed-out pixels of the standard image captured by the imaging unit 24 with the pixel values of the complementary image captured by the complementary image imaging unit 25, a corrected image with the washed-out part corrected can be generated.

[0043] <Selection of the Wavelength Band of the Wavelength Limiting Filter> Next, the selection of the wavelength band of the wavelength-limiting filter will be described. FIG. 4 is an explanatory diagram for explaining the selection of the wavelength band of the wavelength-limiting filter. As shown in FIG. 4, the relative reflectance of a plurality of imaging objects to be imaged is measured in advance. The relative reflectance is obtained by irradiating the imaging object with white light and imaging it with a hyperspectral camera. Here, the relative reflectance Rf1 of the imaging object coated with a dull-colored paint, the relative reflectance Rf2 of the imaging object coated with a gray paint, the relative reflectance Rf3 of the imaging object coated with a yellow paint, the relative reflectance Rf4 of the imaging object made of an aluminum material, and the relative reflectance Rf5 of the imaging object made of an iron material are shown.

[0044] In all the measured materials, since the relative reflectance shows a low value in the blue wavelength band (440 - 460 nm), by selecting the wavelength band of the wavelength-limiting filter 25b to be the blue wavelength band, it is possible to capture a complementary image in which reflections that cause overexposure are suppressed.

[0045] <Processing Procedure of Image Processing Apparatus 20> Next, the processing procedure of the image processing apparatus 20 shown in FIG. 1 will be described. FIG. 5 is a flowchart showing the processing procedure of the image processing apparatus shown in FIG. 1. Here, it is assumed that the wavelength band of the filter used for the wavelength-limiting filter 25b of the complementary image imaging unit 25 is determined in advance. As shown in FIG. 5, the image processing apparatus 20 captures and stores a standard image by the imaging unit 24 (step S101). Then, the image processing apparatus 20 captures and stores a complementary image by the complementary image imaging unit 25 (step S102).

[0046] Thereafter, the image processing apparatus 20 reads out the standard image data and the complementary image data (step S103), corrects the standard image based on the standard image data and the complementary image data (step S104). Then, the image processing apparatus 20 stores the corrected image (step S105).

[0047] As described above, in the present embodiment, the image processing apparatus 20 includes an imaging unit 24 that captures a color image of a plurality of imaging objects having different relative reflectivities, and a complementary image imaging unit 25 that captures transmitted light that has passed through a wavelength limiting filter 25b that transmits only light in a specific wavelength band and blocks light in other wavelength bands. By correcting the standard image based on the standard image captured using the imaging unit 24 and the complementary image captured using the complementary image imaging unit 25, it is possible to efficiently correct the image even in a situation where whiteouts occur in the standard image.

[0048] In the above embodiment, the case where the operator has previously installed the wavelength band of the wavelength limiting filter 25b that transmits light in a predetermined wavelength band and blocks light in other wavelength bands has been described. However, a configuration may be adopted in which the complementary image imaging unit 25 is provided with a plurality of filters that transmit light in a specific wavelength band and do not transmit light outside the wavelength band, and the control unit selects the filters.

[0049] <Modification Example> Incidentally, in the above embodiment, the case of correcting the standard image based on the standard image and the complementary image has been described. In the modification example, the case of generating a 3D model based on the standard image and the complementary image will be described.

[0050] FIG. 6 is a diagram showing an overview of the modification example. The same parts as those of the image processing apparatus 20 shown in FIG. 1 are denoted by the same reference numerals, and detailed description thereof is omitted. Here, the case where the image generation system captures an image of the manufacturing line 60 and generates a 3D model will be described.

[0051] As shown in FIG. 6, the image generation system includes an image processing apparatus 30 and a 3D model generation apparatus 40, which are connected by a network N. An imaging apparatus 23 is connected to the image processing apparatus 30. The imaging apparatus 23 includes an imaging unit 24 and a complementary image imaging unit 25.

[0052] The image processing device 30 captures a standard image of the manufacturing line 60 using the imaging unit 24 (S11). Then, the image processing device 30 captures a complementary image using the complementary image imaging unit 25 (S12). After that, the image processing device 30 changes the imaging positions (23a, 23b, 23c, 23d, 23e, 23f, 23g) of the imaging device 23 in order to capture the manufacturing line 60 from various angles, and captures a standard image and a complementary image at each imaging position (S13)

[0053] Then, the image processing device 30 transmits the plurality of standard image data and complementary image data captured by changing the imaging positions to the 3D model generation device 40 (S14). The 3D model generation device 40 receives the plurality of standard image data and complementary image data from the image processing device 30 (S15). Then, the 3D model generation device 40 corrects the standard image based on the standard image data and the complementary image data (S16). After that, the 3D model generation device 40 generates a 3D model based on the plurality of corrected composite images (S17).

[0054] Note that the generation of the 3D model is performed, for example, by using the SfM (Structure From Motion) technique to generate a 3D shape from other viewpoint images using a plurality of images, and then generating high-density point cloud data by the MVS (Multi-View Stereo) technique.

[0055] In this modification example, the case where the 3D model generation device 40 corrects the standard image based on the standard image and the complementary image and generates a 3D model using the corrected image has been described. However, a 3D model may be generated using the standard image, another 3D model may be generated using the complementary image, and a complemented 3D model may be generated by synthesizing the two generated 3D models.

[0056] Also, in this modification example, the case where the 3D model generation device 40 performs the process of correcting the standard image based on the standard image and the complementary image has been described. However, the process of correcting the standard image may be performed by the image processing device 30, the corrected image may be transmitted to the 3D model generation device 40, and the 3D model may be generated.

[0057] Each configuration illustrated in each of the above embodiments is a functional schematic, and it is not necessarily physically configured as illustrated. That is, the form of distribution and integration of each device is not limited to that illustrated, and all or part of it can be functionally or physically distributed and integrated in any unit according to various loads, usage conditions, etc.

Industrial Applicability

[0058] The image processing apparatus and the image processing method according to the present invention are suitable for quickly and efficiently correcting an image with whiteout when imaging an imaging object.

Explanation of Signs

[0059] 20 Image processing apparatus 21 Display unit 22 Input unit 23, 23a, 23b, 23c, 23d, 23e, 23f, 23g Imaging device 24 Imaging unit 25 Complementary image imaging unit 25a Imaging unit 25b Wavelength limiting filter 26 Storage unit 26a Standard image data 26b Complementary image data 26c Corrected image data 27 Control unit 27a Imaging processing unit 27b Correction processing unit 30 Image processing apparatus 40 3D model generation apparatus 50, 51, 52, 53a, 53b, 53c Imaging object 60 Manufacturing line

Claims

1. A first imaging unit that forms a reference image by receiving a reflected wave obtained by reflecting light irradiated from a predetermined light source onto an imaging object by the imaging object; A second imaging unit that forms a complementary image by receiving light in a predetermined wavelength band included in the reflected wave; A correction processing unit that corrects the reference image based on the complementary image An image processing apparatus characterized by comprising.

2. The second imaging unit The image processing apparatus according to claim 1, further comprising a wavelength limiting filter that transmits light in a predetermined wavelength band and blocks light in other wavelength bands.

3. The wavelength limiting filter The image processing apparatus according to claim 2, wherein a wavelength band of light to be transmitted is determined according to the color of a paint applied to the surface of the imaging object or the material of the surface.

4. The correction processing unit When the pixel value of a pixel forming the reference image exceeds a predetermined threshold, the pixel value of the pixel forming the reference image is corrected using the pixel value of the pixel of the complementary image located at the pixel position of the pixel. The image processing apparatus according to claim 1.

5. The correction processing unit When the pixel values of a plurality of pixels forming a predetermined region of the reference image each exceed a predetermined threshold, the pixel values of the pixels forming the predetermined region are corrected using the pixel values of the pixels corresponding to the predetermined region of the complementary image. The image processing apparatus according to claim 1.

6. An image processing method executed by an image processing apparatus that corrects an image captured by a predetermined imaging unit, the method comprising: A reference image forming step of forming a reference image by receiving, by a first imaging unit, a reflected wave obtained by reflecting light irradiated from a predetermined light source onto an imaging object by the imaging object; A complementary image forming step of forming a complementary image by receiving, by a second imaging unit, light in a predetermined wavelength band included in the reflected wave; A correction processing step of correcting the reference image based on the complementary image An image processing method characterized by including.

Images