Color spot evaluation device, color spot evaluation system, color spot evaluation method and storage medium

Abstract

The present invention relates to a color spot evaluation device, a color spot evaluation system, a color spot evaluation method, a storage medium, and a computer device, and is intended to accurately evaluate color spots. The color spot evaluation device (100) of the present invention includes an imaging unit (2) for capturing an image of an object (S); a projected component (3) projected onto the surface of the object (S); and an output unit for outputting a color spot evaluation value of the object obtained based on a captured image captured by the imaging unit, wherein the captured image includes an image of the projected component projected onto the surface of the object.

Description

Technical Field

[0001] The present invention relates to a color spot evaluation device, a color spot evaluation system, a color spot evaluation method, a storage medium and a computer device. Background Art

[0002] Product quality evaluation sometimes involves evaluating surface defects such as scratches, bumps, and roughness, as well as color defects on painted surfaces known as color spots. Currently, this defect evaluation is performed using images captured by an imaging unit such as a camera.

[0003] In the prior art, patent document 1 (JP Patent Publication No. 2011-75534) discloses a technical solution, which includes a camera unit and a multi-joint robot, wherein the camera unit has an area camera, a plurality of planar lighting devices arranged at a certain angle to the axial direction of the area camera, and a diffusion plate arranged around the lens of the area camera and used to diffuse the illumination light of the planar lighting device, and the multi-joint robot has a function of controlling the position and angle of the camera unit relative to the object being photographed, wherein an image obtained by irradiating the object with illumination light is used to detect defects of the object.

[0004] However, the technique disclosed in Patent Document 1 cannot accurately evaluate color spots in some cases. Summary of the Invention

[0005] An object of the present invention is to provide a color spot evaluation device, a color spot evaluation system, a color spot evaluation method, a storage medium, and a computer device capable of accurately evaluating color spots.

[0006] In order to achieve the above-mentioned purpose, the present invention provides a color spot evaluation device, which includes a camera unit for capturing an image of an object; a projected component for projecting onto the surface of the object; and an output unit for outputting a color spot evaluation value of the object obtained based on the camera image captured by the camera unit, wherein the camera image includes an image of the projected component projected onto the surface of the object.

[0007] The present invention has the effect of enabling accurate evaluation of color spots. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 It is a schematic diagram showing an example of the overall structure of the color spot evaluation device according to the first embodiment.

[0009] Figure 2 This is an example functional block diagram of the processing unit.

[0010] Figure 3 This is a flowchart of an example of processing executed by the processing unit involved in the embodiment.

[0011] Figure 4This is a schematic diagram of an example of extraction processing of evaluation images.

[0012] Figure 5 This is a schematic diagram of an example of filter processing, where (a) is the image before processing and (b) is the image after processing.

[0013] Figure 6 This is a schematic diagram of an example of the ideal image generation process, where (a) is the original image and (b) is the ideal image.

[0014] Figure 7 This is a schematic diagram of an example of image difference processing, where (a) is the original image, (b) is the ideal image, and (c) is the difference image.

[0015] Figure 8 This is a schematic diagram of the first accuracy verification result.

[0016] Figure 9 This is a schematic diagram of the second accuracy verification result.

[0017] Figure 10 This is a schematic diagram showing an example of the configuration of a color spot evaluation device according to a first modification of the first embodiment.

[0018] Figure 11 This is a schematic diagram showing an example of the configuration of a color spot evaluation device according to a second modification of the first embodiment.

[0019] Figure 12 Schematic diagrams of variations of the process performed by the extraction unit, where (a) is before the process and (b) is after the process.

[0020] Figure 13 It is a schematic diagram showing an example of the configuration of a color spot evaluation device according to the second embodiment.

[0021] Figure 14 It is a schematic diagram showing a first example of the configuration of the color spot evaluation device according to the third embodiment.

[0022] Figure 15 It is a schematic diagram showing a second example of the configuration of the color spot evaluation device according to the third embodiment. DETAILED DESCRIPTION

[0023] Hereinafter, the embodiment of the present invention will be described with reference to the accompanying drawings. In each of the drawings, the same components are given the same reference numerals, and repeated description will be omitted.

[0024] The color spot evaluation device illustrated in the following embodiments is intended to embody the technical principles of the present invention. However, the following embodiments do not limit the present invention. Unless otherwise noted, the sizes, materials, shapes, and relative arrangements of components described below are illustrative and are not intended to limit the scope of the present invention. Furthermore, the sizes and positional relationships of components shown in the drawings are sometimes exaggerated for clarity.

[0025] A color unevenness evaluation device according to an embodiment includes an imaging unit that captures an image of an object and an evaluation unit that evaluates color unevenness of the object based on the captured image.

[0026] Color spots are defects that occur on a colored surface, specifically areas where the brightness and hue differ from the original color. There are various types of color spots, such as bands or streaks of different colors, and seemingly blurred color differences. Color spots are also called color unevenness.

[0027] The object for color spot evaluation is not particularly limited, and examples thereof include three-dimensional objects such as automobiles. The color spot evaluation apparatus according to the embodiment can evaluate color spots of paint colors applied to automobile bodies.

[0028] For example, when evaluating color spots, images of surrounding objects may be reflected on the surface of the object. This reflection is more noticeable on glossy objects. This reflection can cause brightness unevenness within the image area of ​​the object included in the captured image, making accurate color spot evaluation impossible in some cases.

[0029] The surface shape of a three-dimensional object causes uneven illumination within a captured image, making it impossible to accurately evaluate color unevenness in some cases.

[0030] The color speckle evaluation device according to the embodiments includes a projection component that projects onto the surface of an object, and a captured image includes the image of the projection component projected onto the surface of the object. By evaluating color speckles based on this captured image, the effects of images of surrounding objects, etc., projected onto the surface of the object, and uneven illumination can be suppressed, enabling accurate color speckle evaluation.

[0031] [First embodiment]

[0032] First, the color spot evaluation device 100 according to the first embodiment will be described.

[0033] <Configuration of Color Spot Evaluation Device 100>

[0034] Overall composition

[0035] Figure 1 1 is a schematic diagram showing an example of the overall structure of the color spot evaluation device 100.

[0036] like Figure 1 As shown, the color spot evaluation device 100 includes an illumination unit 1 , an imaging unit 2 , a projection target member 3 , and a processing unit 4 .

[0037] In the color spot evaluation device 100, the illumination unit 1 illuminates the object S, and the imaging unit 2 captures an image of the object S. The positional relationship between the object S, the illumination unit 1, the imaging unit 2, and the projection component 3 is preset so that the image of the projection component 3 is projected onto the surface of the object S during imaging. Based on the image captured by the imaging unit 2, the processing unit 4 of the color spot evaluation device 100 evaluates the color spots of the object S.

[0038] The lighting unit 1 includes a light source and emits light to illuminate an object S. This embodiment uses an LED (Light Emitting Diode), TLWA325X-22WD-4 manufactured by I-Tech Systems, which emits white diffused light, as an example of a light source.

[0039] The imaging unit 2 includes an area camera and a lens, and uses the area camera to capture an image of the object S roughly formed by the lens. In this embodiment, the STC-MCS241U3V, a Bayer array color area camera manufactured by Omron Sentech, is used as an example of the area camera, and the FL-CC1214-2M, a lens with a focal length of 12 mm manufactured by Ricoh Industrial Solutions, is used as an example of the lens.

[0040] The projected component 3 is a component reflected on the surface of the object S. The image of the projected component 3 is projected on the surface of the object S. When viewed from the imaging unit 2, the image of the projected component 3 is a virtual image of the projected component 3 reflected by the surface of the object S.

[0041] This embodiment uses #188-E20, a white film manufactured by Toray Industries, as an example of the projection component 3. The projection component 3 preferably has a substantially uniform color throughout. However, it is not limited to white and may be a color other than white as long as the color is substantially uniform.

[0042] exist Figure 1 In the example shown, the lighting unit 1 is arranged such that the angle formed by the central axis 11 of the lighting unit 1 and the central axis S1 of the object S, i.e., the lighting angle θ1, is approximately 35 degrees relative to the object S. The central axis 11 of the lighting unit 1 corresponds to the central axis of the LED, which is approximately perpendicular to the light-emitting surface of the LED.

[0043] The imaging unit 2 is arranged relative to the object S so that the angle formed by the central axis 21 of the imaging unit 2 and the central axis S1 of the object S, i.e., the imaging angle θ2, is approximately 45 degrees. The central axis 21 of the imaging unit 2 corresponds to the central axis of the imaging plane that is substantially orthogonal to the imaging plane of the area camera.

[0044] The projection component 3 is positioned relative to the object S so that the angle formed by the central axis 31, which is approximately perpendicular to the planar portion of the projection component 3, and the central axis S1 of the object S, i.e., the projection angle θ3, is approximately 45 degrees. The projection component 3 and the illumination unit 1 are positioned on either side of the central axis S1 of the object S. In other words, the projection component 3 is positioned on the side of the object S opposite to the illumination unit 1, centered on the central axis S1.

[0045] The working distance corresponds to the distance between the object S and the imaging unit 2 and is approximately 380 mm. The resolution of the image captured by the imaging unit 2 is approximately 200 μm / pixel. The resolution of the captured image refers to the length of the image corresponding to the length of one side of each pixel constituting the captured image.

[0046] The processing unit 4 evaluates color unevenness of the object S based on the captured image captured by the imaging unit 2. The processing unit 4 is, for example, an arithmetic processing device implemented by a computer.

[0047] The color spot evaluation device 100 acquires a captured image as 12-bit RAW data. It also performs demosaicing using bilinear interpolation on each of the colors red (R), green (G), and blue (B) to obtain grayscale values ​​for each color, thereby acquiring image data for the captured image.

[0048] The arrangement of the illumination unit 1, imaging unit 2, and projection target 3 is not particularly limited and can be selected as appropriate. However, to ensure that the projection target 3 is reflected on the surface of the object S, it is preferably positioned in the direction of regular reflection from the imaging unit 2. In other words, it is preferable that the projection target 3 and imaging unit 2 are positioned on either side of the central axis S1 of the object S, with the imaging angle θ2 and the projection angle θ3 being approximately equal. Furthermore, it is preferable that the illumination unit 1 be positioned so that it does not appear in the image of the object S captured by the imaging unit 2.

[0049] Function of projected component 3

[0050] Here, the operation in which the image captured by the imaging unit 2 includes the image of the projection member 3 projected onto the surface of the object S will be described.

[0051] The first function is to prevent images of surrounding objects, such as walls and interior lighting fixtures, from being reflected onto the surface of the object S. For example, if the object S is an automobile body, the painted surface is a clear coat with a high gloss. Therefore, the painted surface may act as a reflector, causing images of surrounding objects to be reflected onto the painted surface.

[0052] When taking an image of the car body in this state, it is difficult to extract only the color spots on the car body for evaluation due to the influence of the image reflected on the painted surface.

[0053] By providing the projection component 3, the images of objects such as walls and indoor lighting fixtures surrounding the target object S can be shielded, preventing these images from being reflected on the surface of the target object S. Although the automobile body is used as the target object S as an example in this description, the function of the projection component 3 is the same even for targets S other than automobile bodies.

[0054] The second effect is to suppress uneven illumination of the light irradiated by the illumination unit 1 on the object S. If the object S is a three-dimensional object with a curved surface, the illumination light may be reflected on the surface of the object S, casting a shadow, which may cause uneven illumination on the surface of the object S. When the object S is imaged in this state, the influence of uneven illumination makes it difficult to accurately extract and evaluate only the color spots of the object S.

[0055] By projecting an image of the projection component 3 having a substantially uniform color onto the surface of the object S and using a captured image of the object S on which the image of the projection component 3 is projected, the influence of such uneven illumination can be suppressed.

[0056] To suppress uneven illumination, it is also conceivable to omit the projection element 3 and instead project illumination light of a substantially uniform color onto the surface of the object S. However, in such a configuration, the regular reflection component of the illumination light generated on the surface of the object S increases. Therefore, if the surface color of the object S is a dark color such as black, the difference between the original surface color of the object S and the color of the unevenly colored portion in the captured image decreases. As a result, it becomes difficult to accurately extract and evaluate only the unevenly colored portion.

[0057] In contrast, this embodiment provides the illumination unit 1 for illuminating the object S and the projection member 3 separate from the illumination unit 1 , thereby enabling accurate extraction and evaluation of color irregularities even when the object S is dark.

[0058] Functional structure of the processing unit

[0059] Figure 2 FIG. 4 is a functional block diagram of the processing unit 4 included in the color spot evaluation device 100. Figure 4As shown, the processing unit 4 includes an extraction unit 41 , an ideal image creation unit 42 , a difference image acquisition unit 43 , an evaluation unit 44 , and an output unit 45 .

[0060] These functions can be implemented by circuits or software. These functions can also be implemented by multiple circuits or multiple software. In order to perform image processing, the processing unit 4 preferably has a frame memory such as RAM (Random Access Memory) to temporarily store image data of the captured image.

[0061] The processing unit 4 uses the ideal image creation unit 42 to create an ideal image free of color spots based on the captured image captured by the imaging unit 2. The difference image acquisition unit 43 extracts image components representing color spots from the difference image between the captured image and the ideal image. The processing unit 4 calculates a color spot evaluation value based on the size of the extracted image component using the evaluation unit 44. The processing unit 4 can obtain the color spot evaluation value solely from the captured image of the object S without capturing a reference sample as a benchmark for the absence of color spots.

[0062] The extraction unit 41 extracts an evaluation image in which the image of the projection component 3 is reflected from the image captured by the imaging unit 2 .

[0063] The ideal image creating unit 42 creates an ideal image of the object S without color unevenness through calculation based on the captured image captured by the imaging unit 2 .

[0064] The difference image acquisition unit 43 obtains a difference image between the captured image captured by the imaging unit 2 and the ideal image created by the ideal image creation unit 42 through calculation.

[0065] The evaluation unit 44 calculates the color unevenness evaluation value of the object S based on the image captured by the imaging unit 2. For example, the evaluation unit 44 calculates the color unevenness evaluation value of the object based on the sum of the absolute values ​​of the pixel values ​​included in the difference image created by the difference image acquisition unit 43. The evaluation unit 44 may also compensate the color unevenness evaluation value based on the chromaticity value of the object S.

[0066] The evaluation unit 44 outputs the obtained color spot evaluation value to an external device via the output unit 45. Examples of the external device include a personal computer (PC), a display device such as a monitor, a storage device such as an HDD (Hard Disk Drive), or a communication device such as a communication interface, which are installed outside the color spot evaluation device 100.

[0067] <Operation of Color Spot Evaluation Device>

[0068] Next, the operation of the color spot evaluation apparatus 100 will be described.

[0069] Method for acquiring camera images

[0070] First, to perform compensation, the color spot evaluation device 100 uses the imaging unit 2 to capture an image of a white reference plate positioned at the location of the object S while the illumination unit 1 is on. The exposure time of the imaging unit 2 when capturing the image of the white reference plate is Tw (sec). Based on the captured image of the white reference plate, the color spot evaluation device 100 calculates the average grayscale values ​​Rw, Gw, and Bw of the white reference plate.

[0071] Next, an object S is placed in place of the white reference plate, and an image of the object S is captured. The exposure time for capturing the image of the object S is Ts (sec), and the grayscale values ​​of the R, G, and B colors of the captured image of the object S are Rs(i, j), Gs(i, j), and B(i, j). Here, i and j represent the position coordinates within the captured image.

[0072] Next, the color spot evaluation apparatus 100 uses the brightness values ​​of the white reference plate to compensate the captured image of the object S. For example, the color spot evaluation apparatus 100 uses the following equation (1) to calculate and obtain the grayscale values ​​R(i, j), G(i, j), and B(i, j) for each color of each pixel in the compensated captured image of the object S, thereby performing compensation.

[0073]

[0074] The above-mentioned compensation process can be performed by Figure 2 The color spot evaluation device 100 may also include a functional unit for compensation processing in the processing unit 4 .

[0075] Processing method of the processing department

[0076] Figure 3 This is a flowchart of an example of processing executed by the processing unit 4. Figure 3 The processing is triggered when the image captured by the imaging unit 2 is input to the processing unit 4 .

[0077] First, in step S31 , the extraction unit 41 extracts an evaluation image in which the image of the projected component 3 is reflected from the image captured by the imaging unit 2 .

[0078] Next, in step S32 , the ideal image creating unit 42 creates an ideal image of the object S without color unevenness through calculation based on the captured image captured by the imaging unit 2 .

[0079] Next, in step S33 , the difference image acquisition unit 43 obtains a difference image between the captured image captured by the imaging unit 2 and the ideal image created by the ideal image creation unit 42 through calculation.

[0080] Next, in step S34 , the evaluation unit 44 obtains the color unevenness evaluation value of the object S through calculation based on the difference image.

[0081] Next, in step S35 , the output unit 45 outputs the color unevenness evaluation value acquired by the evaluation unit 44 to an external device.

[0082] In this way, the processing unit 4 can obtain the color unevenness evaluation value and output the obtained color unevenness evaluation value to an external device.

[0083] Specific examples of processing methods

[0084] Next, the processing method of the processing unit 4 will be described in detail.

[0085] Figure 4 Schematic diagram of the evaluation image extraction process performed by the extraction unit 41 . Figure 4 The captured image 7 shown is an image captured by the imaging unit 2. The captured image 7 includes an image 70 of the projection component 3 projected onto the surface of the object S.

[0086] The extraction unit 41 extracts an evaluation image 71 for evaluating color unevenness by cropping the image area corresponding to the image 70 of the projected component 3 included in the captured image 7. The size of the evaluation image 71 may be predetermined, or an appropriate range may be automatically extracted from the captured image 7.

[0087] Next, the processing unit 4 performs filtering processing on the evaluation image 71 extracted by the extraction unit 41. This filtering processing can be performed by Figure 2 Any one of the functions in the functional units shown is performed, and here it is assumed that the extraction unit 41 performs the processing.

[0088] Figure 5 1 is a schematic diagram of an example of filtering processing of an evaluation image 71 , wherein (a) is the evaluation image 71 before filtering processing, and (b) is the evaluation image 71 ′ after filtering processing.

[0089] Figure 5 (a) and (b) show an example of a square image with 70 pixels on one side. 70 pixels corresponds to a length of approximately 14 mm.

[0090] For example, the object S is a car body that has been painted with metallic or pearlescent paint. In this case, since the paint contains bright material, Figure 5 As shown in FIG. 7 ( a ), a fine bright spot 72 corresponding to the image of the particle appears in the evaluation image 71 .

[0091] Since the grayscale fluctuation of the bright spot 72 caused by particles is not color unevenness, it is best to remove or make it less noticeable when evaluating color unevenness. The extraction unit 41 removes or makes the bright spot 72 less noticeable by filtering using a median filter, for example. Figure 5 (b) is the result of filtering using a square filter with 11 pixels on one side as the median filter. Figure 5 As shown in (b), the bright spot 72 is removed.

[0092] Any filtering method may be used as long as it can suppress particle images such as the bright spots 72 in the evaluation image 71 .

[0093] Next, the processing unit 4 performs a chromaticity value conversion process on the evaluation image 71' after the filtering process is performed by the extraction unit 41. This conversion process is a process for converting the RGB value of the device-dependent format into the L*a*b value of the uniform color space. Figure 2 The processing is performed by any one of the functional units shown, and here, it is assumed that the extraction unit 41 performs the processing.

[0094] The extraction unit 41 converts the image represented by RGB values ​​into an image represented by XYZ values ​​using the conversion formula shown in the following formula (2), and then converts the image represented by XYZ values ​​into an image represented by L*a*b values ​​using the conversion formula shown in formula (3).

[0095]

[0096]

[0097] In formula (3), Xn, Yn, and Zn represent the tristimulus values ​​X, Y, and Z of a completely diffuse reflecting surface. In this embodiment, the values ​​Xn=94.81, Yn=100.00, and Zn=107.33 are used under a D65 illuminant and a 10-degree field of view.

[0098] Next, the processing unit 4 creates an image without color unevenness, that is, an ideal image, based on the image represented by L* in the image after the chromaticity value conversion process, through the ideal image creating unit 42 .

[0099] Figure 6 is a schematic diagram of an ideal image creation process. Figure 6 (a) and (b) represent the original image and the ideal image, respectively. The original image is the image represented by L* after the chromaticity value transformation.

[0100] exist Figure 6In (a), the color spot M is an image area including a color that is different from the uniform or continuous color change in the original image 73. The ideal image creation unit 42 creates the image as shown in FIG. Figure 6 In the ideal image 74 shown in (b), there is no color spot M and the color changes smoothly.

[0101] In this embodiment, ideal image creation unit 42 generates ideal image 74 free of color spots M by using polynomial function approximation. Specifically, ideal image creation unit 42 performs a cubic equation for each row in the horizontal direction of original image 73, and then performs a similar approximation for each column in the vertical direction of original image 73 to create ideal image 74.

[0102] Next, the processing unit 4 performs image difference processing between the original image 73 and the ideal image 74 via the difference image acquisition unit 43 to obtain a difference image.

[0103] Figure 7 This is a schematic diagram of an example of image subtraction processing. (a) is original image 73, (b) is ideal image 74, and (c) is difference image 75. The difference image acquisition unit 43 extracts the color unevenness component by obtaining a difference image 75 containing only the color unevenness component M based on the image difference between original image 73 and ideal image 74.

[0104] Next, the processing unit 4 calculates the color unevenness evaluation value through the evaluation unit 44. The following formula (4) is a calculation formula for the color unevenness evaluation value.

[0105]

[0106] The larger the brightness difference of the color unevenness or the larger the area of ​​the color unevenness, the larger the color unevenness evaluation value. The color unevenness evaluation value E1 obtained by formula (4) is an example of the color unevenness evaluation value obtained based on the sum of the absolute values ​​of the pixel values ​​included in the difference image.

[0107] Furthermore, the inventors have found from their research results that the correlation with human subjective evaluation can be improved by using the average L* of the object S as a compensation function for the total value of color spots obtained from the difference image 75 .

[0108] Equation (5) is used by the evaluation unit 44 to calculate the compensated color unevenness evaluation value E2 after compensating for the color effects of the object S. In Equation (5), p is a parameter, which is predetermined based on subjective evaluation points. The compensated color unevenness evaluation value E2 is an example of a color unevenness evaluation value compensated according to the chromaticity values ​​of the object S.

[0109]

[0110] <Verification Results of Evaluation Accuracy of Color Spot Evaluation Device>

[0111] Next, the verification results of the evaluation system of the color spot evaluation device 100 will be described.

[0112] Verification Method

[0113] The test objects S were created by coating the surface of a resin handle with a commercially available automotive refinish paint. Three metallic colors (silver, red, and blue) were used. The degree of color unevenness was varied by varying the number of repetitive coatings and drying time for each color. The number of test objects S was four silver, four red, and five blue.

[0114] The evaluation result of the color spot evaluation device 100 on the object S is compared with the subjective evaluation result of a person on the object S to verify the correlation between the color spot evaluation value of the color spot evaluation device 100 and the subjective evaluation point.

[0115] In the subjective evaluation, the state with no color spots at all was designated as evaluation level 0, and the state in which the inventor subjectively considered the worst degree of color spots among the objects S was designated as the benchmark object with evaluation level 10. Six evaluators other than the inventor observed the objects S and evaluated the degree of color spots on a scale of 0 to 11 in units of 0.5.

[0116] The evaluators observed under D65 illuminant, using Spectra Light QC manufactured by X-rite Co. The average of the evaluation grades given to each object S was calculated based on the evaluation results of the six evaluators and used as the subjective evaluation point of each object S.

[0117] Figure 8 This is a schematic diagram of the first accuracy verification result. Figure 8 The horizontal axis represents the compensated color spot evaluation value E2 obtained by equation (5), and the vertical axis represents the subjective evaluation points. The error bars on the vertical axis represent the standard deviation of the subjective evaluation points. The parameter p in equation (5) is 0.44, which is calculated based on the subjective evaluation points.

[0118] from Figure 8 The results shown in FIG. 1 show that the contribution rate is 0.9 or higher, and there is a high correlation between the compensated color spot evaluation value E2 obtained by the color spot evaluation device 100 and the evaluation result obtained by subjective evaluation. Figure 8 The results shown also show that the main cause of color spots is brightness fluctuation.

[0119] Figure 9 This is a schematic diagram of the second accuracy verification result. Figure 9 Perspective and Figure 8 same. Figure 9This is the result when the average L* as the compensation function is not used, that is, the color unevenness evaluation value E1 calculated by equation (4) is used as the color unevenness evaluation value. It can be seen that when the color unevenness evaluation value E1 is used, the correlation varies depending on the color of the object S.

[0120] <Effects of the Color Spot Evaluation Device>

[0121] Next, the effects of the color spot evaluation device 100 will be described.

[0122] Surface appearance is one of the key aspects of product quality management. Appearance evaluation typically focuses on surface defects such as scratches, bumps, and roughness, as well as color defects on painted surfaces known as color spots. These appearance evaluations are often performed visually.

[0123] In recent years, the automation of product appearance evaluation has been steadily advancing, aiming to reduce the so-called human nature of judgment, which only skilled evaluators can perform, and to improve manufacturing processes by quantifying evaluation results, or to reduce manufacturing costs by reducing manpower. As part of this process, it is widely known that cameras or other imaging devices capture the appearance of the product and evaluate defects based on the captured images.

[0124] However, color unevenness evaluation using captured images is sometimes difficult. For example, if the image of an object surrounding the object is reflected in the captured image, or if uneven illumination occurs in the captured image depending on the shape of the object, color unevenness evaluation is difficult.

[0125] Specifically, when evaluating color unevenness on a high-gloss surface such as automobile paint, when capturing an image of the object, images of objects surrounding the object may be reflected in the captured image.

[0126] Furthermore, when evaluating color spots on a three-dimensional object, uneven illumination may occur within the captured image due to the surface shape of the object. Because the difference between the original color of the spot and the color of the spot is very small, this reflection or uneven illumination can make color spot evaluation difficult.

[0127] If the imaging range of the imaging unit for imaging the object is narrowed in order to suppress the effects of reflection or uneven illumination, multiple imaging operations are required to evaluate the entire object, which increases the evaluation time.

[0128] In addition, the above-mentioned patent document 1 discloses a robot comprising: an imaging unit including: an area camera; a plurality of planar lighting devices arranged at a certain angle to the axial direction of the area camera; a diffusion plate arranged around the lens of the area camera for diffusing the illumination light emitted by the planar lighting device; and a multi-joint robot having a function of controlling the position and angle of the imaging unit relative to the object to be photographed, and detecting defects of the object based on an image obtained by photographing the object by irradiating the illumination light on the object.

[0129] However, in Patent Document 1, the entire inspection area is strongly illuminated by the surface-reflected light from the illumination. Consequently, when evaluating color spots on darker paint surfaces, such as black, the subtle color differences between the original paint color and the areas with color spots are obscured by the surface-reflected light from the illumination. As a result, grayscale value differences are lost within the captured image, making accurate evaluation of color spots impossible. Furthermore, since the area available for simultaneous inspection on the object is relatively small, evaluating color spots across the entire object takes time.

[0130] The color spot evaluation apparatus 100 according to the present embodiment includes a projection member 3 projected onto the surface of an object S, and a captured image 7 includes an image 70 of the projection member 3 projected onto the surface of the object S.

[0131] The projection component 3 shields the images of objects such as walls and indoor lighting fixtures around the object S, preventing them from being reflected on the surface of the object S, thereby preventing these images from being captured in the captured image 7 .

[0132] By making the color of the projection component 3 substantially uniform, the color of the image 70 of the projection component 3 projected onto the surface of the object S can be made substantially uniform, thereby suppressing uneven illumination of the light irradiated on the object S by the illumination unit 1 .

[0133] As a result, this embodiment can suppress the effects of images of surrounding objects appearing on the surface of the object and uneven illumination, enabling accurate evaluation of color spots. Furthermore, since there is no need to narrow the imaging range of the object S captured by the imaging unit 2, color spots can be evaluated in a shorter evaluation time.

[0134] This embodiment includes an extraction unit 41 for extracting an evaluation image 71 reflecting an image 70 of the projected component 3 from the captured image 7. By using the thus extracted evaluation image 71, the effects of images of objects surrounding the object appearing on the surface of the object and uneven illumination are suppressed, allowing accurate evaluation of color spots.

[0135] This embodiment includes an ideal image creation unit 42 for creating an ideal image 74 of the object S without color unevenness based on the captured image 7. It also includes a difference image acquisition unit 43 for acquiring a difference image 75, which is a difference image between the captured image 7 and the ideal image 74. This allows accurate extraction of the difference image 75 from the captured image 7, and accurate evaluation of color unevenness.

[0136] In the present embodiment, a color unevenness evaluation value E1 is output based on the sum of the absolute values ​​of the pixel values ​​included in the difference image 75. Color unevenness can be accurately evaluated using this color unevenness evaluation value E1.

[0137] In this embodiment, a compensated color unevenness evaluation value E2 compensated according to the chromaticity value of the object S may be output. The compensated color unevenness evaluation value E2 can suppress the difference corresponding to the color of the object S that is associated with the subjective evaluation and the color unevenness evaluation value.

[0138] <Modification>

[0139] First Modification

[0140] Here, a modified example of the first embodiment is described. Hereinafter, the same components as those described in the first embodiment are given the same reference numerals, and repeated descriptions are omitted as appropriate. This also applies to the modified examples and embodiments described below.

[0141] Figure 10 FIG. 1 is a schematic diagram showing an example of the configuration of a color spot evaluation device 100a according to a first modification of the first embodiment. Figure 10 As shown, the color spot evaluation device 100 a includes a light emitting component 5 .

[0142] The light-emitting component 5 is a surface-emitting LED lighting device with uniform surface illumination and is an example of a projected component that is reflected on the surface of the object S. This modification provides the light-emitting component 5 in place of the projected component 3 in the first embodiment, so that the image captured by the imaging unit 2 includes the image of the light-emitting component 5 projected on the surface of the object S, thereby achieving the same functional effects as the first embodiment.

[0143] Second Modification

[0144] Figure 11 FIG. 1 is a schematic diagram showing an example of the configuration of a color spot evaluation device 100b according to a second modified example of the first embodiment. Figure 11 As shown, the color spot evaluation device 100 b includes a projection unit 6 .

[0145] The projection unit 6 includes a projector 61 and a diffuser 62. The projector 61 projects light of substantially uniform color onto the diffuser 62. The diffuser 62 is a plate-shaped member formed of a material such as resin or glass, and is used to diffuse the light projected by the projector 61. The projection unit 6 is an example of a projection target member that is projected onto the surface of the object S.

[0146] This modification provides a projection unit 6 instead of the projected member 3 in the first embodiment, so that the image captured by the imaging unit 2 includes the image of the diffuser 62 projected on the surface of the object S, thereby achieving the same effects as the first embodiment.

[0147] However, if the light intensity of the light emitting component 5 or the projecting unit 6 is high, the intensity of the reflected light may overwhelm the color shading on the surface of the object S, making it impossible to accurately extract the color shading components from the captured image. Therefore, it is desirable that the light intensity of the light emitting component 5 or the projecting unit 6 be low.

[0148] While the first embodiment illustrates that the extraction unit 41 extracts the evaluation image 71 including the image 70 of the light-emitting component 5 reflected on the surface of the object S from the captured image by cropping, this is not a limitation. The extraction unit 41 may also be configured to automatically extract an appropriate region corresponding to the evaluation image 71.

[0149] Figure 12 This is a modified example of the processing performed by the extraction unit 41. (a) is an image before processing, and (b) is an image after processing.

[0150] When the projected component 3 is white, Figure 12 As shown in FIG. 1 (a), on the surface of the object S, the image region U where the image of the projection component 3 is reflected becomes brighter than the image region U′ where the image is not reflected.

[0151] Taking advantage of this, the extraction unit 41 converts the captured image 7 into an image expressed in L* and then automatically extracts the image area reflecting the projected component 3 by extracting the image area where L* is greater than a predetermined threshold. Common binarization image processing methods such as pattern methods and discriminant analysis methods can be used as a method for setting the threshold.

[0152] Then, if Figure 12 As shown in (b), the extraction unit 41 replaces the pixel values ​​of the image area other than the image area reflecting the image of the projected component 3 in the camera image 7 with 0, and extracts the evaluation image 71". Then, the ideal image creation unit 42 creates the ideal image by performing approximation only within the evaluation image 71".

[0153] When the difference image acquisition unit 43 obtains the difference image between the original image and the ideal image based on the evaluation image 71 ″, the pixel values ​​outside the reflected area are always 0, so that the color unevenness can be evaluated only in the reflected area.

[0154] Although the first embodiment transforms the RGB image into the L*a*b* space to improve evaluation accuracy, the color spot evaluation value can also be obtained directly from the RGB image before transformation. For example, the G image can be used instead of the L* in equation (4). This can reduce the computational processing load.

[0155] In addition, while the first embodiment illustrates the use of a color area camera as the imaging unit 2, this is not limiting. For example, the imaging unit 2 may include a monochrome camera, and the grayscale image acquired by the monochrome camera may be used to evaluate color spots. Alternatively, a multispectral camera or a hyperspectral camera may be used to obtain an L* image from the obtained spectral image for evaluation.

[0156] [Second embodiment]

[0157] Next, a color spot evaluation apparatus 100c according to a second embodiment will be described. In this embodiment, a method of further expanding the area reflected by the projection member 3 on the surface of the object S will be described.

[0158] Figure 13 Schematic diagram showing an example of the structure of the color spot evaluation device 100c. Figure 13 As shown in FIG. 1 , the color spot evaluation device 100c includes a projection component 3c. The projection component 3c has a curved surface.

[0159] Here, if the projection component 3 is planar as in the first embodiment, the projection area of ​​the image of the projection component 3 may be reduced depending on the shape of the object S. In particular, when the curvature of the object S is large, the projection area tends to be reduced.

[0160] In order to expand the projection area, it is conceivable to make the shape of the projection component 3 as close as possible to the shape of the object S. However, for an object S with a large curvature, it is difficult to make the entire projection component 3 close to the shape.

[0161] Therefore, in this embodiment, the projection component 3 c is formed to cover the object S, so that the projection component 3 as a whole approximates the shape of the object S. This can expand the area where the image of the projection component 3 c is reflected on the surface of the object S. The shape of the projection component 3 c may include a hemispherical curved surface such as a dome, a shape composed of multiple flat surfaces, or a shape composed of a curved surface and multiple flat surfaces.

[0162] Effects other than the above-described effects are the same as those described in the first embodiment.

[0163] [Third embodiment]

[0164] Next, a color spot evaluation device 100d according to a third embodiment will be described. In this embodiment, a method for evaluating color spots in a large area of ​​an object S will be described.

[0165] Figure 14 FIG. 1 is a schematic diagram showing a first example of the structure of the color spot evaluation device 100d. Figure 14 As shown, the color spot evaluation device 100 d includes a holding unit 8 and a moving unit 9 .

[0166] The holding portion 8 holds the lighting unit 1, the imaging unit 2, and the projected component 3. The holding portion 8 is a plate-shaped member made of metal or resin, and the lighting unit 1, the imaging unit 2, and the projected component 3 are fixed to the front with adhesive or screws to hold these components.

[0167] However, the holding portion 8 is not limited to a plate-shaped member, and may be a box-shaped member that holds the lighting unit 1, the imaging unit 2, and the projection target 3. The holding portion 8 only needs to be able to hold at least the imaging unit 2 and the projection target 3.

[0168] The moving unit 9 moves the holding unit 8 along the holding unit moving direction 10 to change the relative positions of the illumination unit 1, the imaging unit 2, and the projection target component 3 with respect to the object S. The moving unit 9 may include, for example, a linear motion stage and a drive unit such as a motor for driving the linear motion stage, or may include a robot arm or the like.

[0169] In the first and second embodiments, the size of the projection component 3 must be increased to expand the area to be evaluated within the object S. However, when evaluating large objects such as door panels and fenders of automobile bodies, the size of the projection component 3 increases, resulting in a larger color spot evaluation apparatus.

[0170] In contrast, in this embodiment, the surface of the object S is imaged multiple times while changing the relative positions of the illumination unit 1, imaging unit 2, and projection target member 3 relative to the object S. This allows evaluation of color unevenness over a large area of ​​the object S.

[0171] In this embodiment, the imaging unit 2 may include a line sensor camera, so that the illumination unit 1, the imaging unit 2, and the projection target 3 can be imaged as if scanning the object S. In this case, the imaging range of the line sensor camera only needs to be within the area projected by the image of the projection target 3, so the projection target 3 can be miniaturized.

[0172] As described above, in this embodiment, color unevenness in a large area of ​​the object S can be evaluated using the small color unevenness evaluation apparatus 100 d .

[0173] Although Figure 14 In the example described above, the holding unit 8 is moved. However, the holding unit 8 may be fixed, and the object S may be moved to change the relative positions of the illumination unit 1 , the imaging unit 2 , and the projection target member 3 with respect to the object S. Figure 15 This is a schematic diagram of a second example of the structure of the color spot evaluation device 100e.

[0174] like Figure 15 As shown, the color spot evaluation device 100e includes a moving unit 9e. The moving unit 9e moves the object S along the object movement direction 10e. The moving unit 9e can include a linear motion stage, a motor, or other drive components, or a robot arm. Alternatively, the conveyor belt used to transport the object S in a production line can be directly used as the moving unit 9e.

[0175] While the embodiments have been described above, the present invention is not limited to the specifically disclosed embodiments above, and various modifications and alterations are possible without departing from the scope of the claims.

[0176] For example, an embodiment includes a color spot evaluation system including at least one of the color spot evaluation apparatuses 100 , 100 a , 100 b , 100 c , 100 d , and 100 e and an information processing apparatus.

[0177] The information processing device is communicatively connected to at least one of the color spot evaluation devices 100, 100a, 100b, 100c, 100d, and 100e, and can process the received information related to the color spots of the object S. The information processing device can be implemented by a computer such as a cloud server.

[0178] The order, quantity, and other numbers used in the description of the embodiments are illustrative only for the purpose of specifically illustrating the technical solution of the present invention, and the present invention is not limited by the illustrative numbers. The connection relationships between the constituent elements are illustrative only for the purpose of specifically illustrating the technical solution of the present invention, and the connection relationships used to achieve the functions of the present invention are not limited to these.

[0179] Embodiments also include color spot evaluation methods. For example, the color spot evaluation method includes outputting a color spot evaluation value for an object obtained based on a captured image including an image of a projection component projected onto the surface of the object. This color spot evaluation method can achieve the same effects as the aforementioned color spot evaluation device.

[0180] Embodiments also include a program. For example, the program can be used to cause a computer to execute processing for outputting a color unevenness evaluation value for an object obtained based on a captured image including an image of a projection component projected onto the surface of the object. This program can achieve the same effects as the aforementioned color unevenness evaluation device.

[0181] Furthermore, each function of the above-described embodiments can be implemented by one or more processing circuits. The term "processing circuit" herein includes a processor similar to a processor implemented as an electronic circuit and programmed to perform various functions through software, as well as an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), or a conventional circuit module designed to perform the above functions.

[0182] Explanation of symbols

[0183] 1 Illumination unit, 2 Camera unit, 3 Projected component, 4 Processing unit, 41 Extraction unit, 42 Ideal image creation unit, 43 Differential image acquisition unit, 44 Evaluation unit, 45 Output unit, 5 Light-emitting component (an example of a projected component), 6 Projection unit (an example of a projected component), 61 Projector, 62 Diffuser, 7 Camera image, 70 Image, 71 Evaluation image, 73 Original image, 74 Ideal image, 75 Differential image, 8 Holding unit, 9 Moving unit, E1 Color nonlinearity evaluation value, E2 Compensated color nonlinearity evaluation value, M Color nonlinearity, θ1 Illumination angle, θ2 Camera angle, θ3 Projection angle, U, U' image areas.

Claims

1. A color spot evaluation device, comprising a camera unit for capturing an image of an object; A projected component is projected onto the surface of the object; as well as an ideal image creating unit configured to create an ideal image of the object without color spots based on the image captured by the imaging unit; the image including an image of the projected component projected on the surface of the object; a differential image acquiring unit configured to acquire a differential image between the captured image and the ideal image; An output unit is configured to output the color unevenness evaluation value obtained based on the sum of absolute values ​​of pixel values ​​included in the difference image.

2. The color spot evaluation device according to claim 1, further comprising an extraction unit for extracting an evaluation image in which the image of the projected component is reflected from the captured image.

3. The color spot evaluation device according to claim 1, wherein The output unit outputs the color unevenness evaluation value that has been compensated based on the chromaticity value of the object.

4. The color spot evaluation device according to any one of claims 1 to 3, wherein The projected component includes at least one of a curved surface shape and a shape formed by combining a plurality of planes.

5. The color spot evaluation device according to any one of claims 1 to 3, wherein a holding portion, configured to hold the imaging portion and the projected component; and The moving portion is used to change the relative position of the holding portion and the object.

6. A stain evaluation system comprising The color spot evaluation device according to any one of claims 1 to 5; and The information processing device is communicably connected to the color spot evaluation device and can process information on the color spots of the object.

7. A color spot evaluation method comprising outputting a color spot evaluation value of an object, the color spot evaluation value being obtained based on the following steps: creating an ideal image of the object without color spots based on a captured image including an image of a projected component projected on a surface of the object; obtaining a differential image between the camera image and the ideal image; The color unevenness evaluation value is obtained based on the sum of the absolute values ​​of pixel values ​​included in the difference image.

8. A computer-readable storage medium storing therein data for a computer to execute a process of outputting a color unevenness evaluation value of an object, the color unevenness evaluation value being obtained based on the following steps: creating an ideal image of the object without color spots based on a captured image including an image of a projected component projected on a surface of the object; obtaining a differential image between the camera image and the ideal image; The color unevenness evaluation value is obtained based on the sum of the absolute values ​​of pixel values ​​included in the difference image.

9. A computer device comprising a processor and a memory storing a program, wherein the computer device executes the program by the processor to output a color unevenness evaluation value of an object, the color unevenness evaluation value being obtained based on the following steps: creating an ideal image of the object without color spots based on a captured image including an image of a projected component projected on a surface of the object; obtaining a differential image between the camera image and the ideal image; The color unevenness evaluation value is obtained based on the sum of the absolute values ​​of pixel values ​​included in the difference image.

Images