Color determination model construction device

Abstract

A device (100) for constructing a color determination model for identifying the color of an object to be inspected comprises: an image acquisition unit (111) that acquires an image of a predetermined portion from a captured image (31) of each of a plurality of sample targets having different known colors; a region classification unit (112) that applies a classification algorithm to a classification color value acquired from the image of the predetermined portion and classifies the image of the predetermined portion into a plurality of regions of different types; and a color determination model generation unit (114) that, for each of the plurality of sample targets, acquires a color determination color value from each of the plurality of regions and registers the color determination color value associated with the known color of the sample target in each type of processing spaces of the plurality of regions, thereby generating each type of the processing spaces as the color determination model.

Description

Color determination model construction apparatus

[0001] The present disclosure relates to a technique for constructing a color determination model for color identification of an inspection target.

[0002] Various techniques have been proposed for imaging a workpiece and identifying the color of a specific portion in the image thereof. For example, according to the color identification method disclosed in Patent Document 1, first, a color identification template prepared based on images captured under various specific illumination conditions is prepared, subsequently, color values are obtained from a plurality of pixels constituting a specific portion of the captured image, the Mahalanobis distance between the obtained pixel color values and the color identification template corresponding to the illumination condition at the time of imaging is calculated, and the color of the color identification template with the closest Mahalanobis distance is determined as the color of the pixel.

[0003] Japanese Unexamined Patent Application Publication No. 2007-164737

[0004] However, in the color identification method disclosed in Patent Document 1, color identification templates corresponding to a plurality of illumination conditions are prepared, and color balance adjustment according to environmental information is required for a captured image in order to calculate the Mahalanobis distance from the color identification templates. Therefore, in order to obtain high color identification accuracy, it is necessary not only to prepare color identification templates corresponding to various illumination conditions, but also to provide a means for acquiring environmental information for specifying the illumination conditions. For this reason, according to Patent Document 1, the processing steps become complicated, making it difficult to perform accurate color identification at high speed.

[0005] Furthermore, in general, inspection targets not only have various three-dimensional shapes, but some have a mirror-like surface coating. Therefore, depending on the positional relationship among the camera, illumination, and the inspection target, the amount of diffusely reflected light entering the camera varies, which can cause a difference in how the color of the workpiece appears.

[0006] The present disclosure has been devised in view of the above circumstances, and an object of the present disclosure is to provide a color determination model construction apparatus that can achieve reduction of processing load for color identification, flexible response to color changes, and higher speed of accurate color identification.

[0007] The color determination model construction apparatus of the present disclosure is an apparatus for constructing a color determination model for identifying the color of an object to be inspected, comprising: an image acquisition unit that acquires an image of a predetermined portion from each captured image of a plurality of sample objects having different known colors; a region classification unit that applies a classification algorithm to the classification color values ​​acquired from the image of the predetermined portion and classifies the image of the predetermined portion into a plurality of regions of different types; and a color determination model generation unit that, for each of the plurality of sample objects, acquires a color determination color value from each of the plurality of regions and registers the color determination color value, which is associated with the known color of the sample object, in various processing spaces of the plurality of regions, thereby generating the various processing spaces as the color determination model.

[0008] According to the color determination model construction device disclosed herein, a color determination model is constructed by registering color determination values ​​obtained from each of multiple regions in a different processing space, associating them with the known color of the sample object. Therefore, up to the number of color determination values ​​corresponding to the number of types can be registered for a single color, and furthermore, color identification becomes possible based on the type of region suitable for color determination in the object being inspected. This makes it possible to reduce the processing load, respond flexibly to color changes, and achieve high-speed, accurate color identification.

[0009] This is a schematic diagram of the color recognition system. This is a block diagram showing the electrical connection relationships of the color judgment model construction device. This is a block diagram showing the electrical connection relationships of the color recognition device. This is a schematic diagram showing an example of the bright / dark processing space in a color judgment model for explaining the color recognition operation of test data using the kNN method. This is a flowchart illustrating the operation of the color judgment model construction device. This is an explanatory diagram showing an example of an captured image. This is an explanatory diagram showing an example of a predetermined image area. This is an explanatory diagram showing an example of a predetermined image area for explaining an example of sampling of color values ​​for type determination of the color judgment model construction device according to this embodiment. This is an explanatory diagram showing an example of a predetermined image area for explaining an example of sampling of color values ​​for type determination of the color judgment model construction device according to this embodiment. This is a schematic diagram showing an example of a color judgment model.

[0010] Preferred embodiments of the present disclosure will be described with reference to the drawings. Figure 1 is a schematic diagram showing an example of a color identification system including a color identification device according to one embodiment of the present disclosure. The color identification system 40 is used, for example, in a factory that manufactures the bodies of BEVs (Battery Electric Vehicles), HEVs (Hybrid Electric Vehicles), and PHEVs (Plug in Hybrid Electric Vehicles) to inspect the color of an object to be imaged 10 (e.g., the body) to which paint has been applied. Note that PHEV refers to a hybrid vehicle that can be externally charged to or powered from a battery. PHEVs have charging ports (inlets) for inserting charging cables into which power is supplied from external charging equipment, and outlets for external power supply.

[0011] Prior to describing the color recognition system 40, the image target 10 whose color is recognized by the color recognition system 40 will be described. The image target 10 has a three-dimensional shape and its surface is reflective. As shown in Figure 1, the image target 10 is illuminated by an illumination light source 20 and imaged by a camera 30 from a predetermined direction. The image target 10 is illuminated from different angles by one or more illumination light sources 20 located at a predetermined distance apart.

[0012] Next, the color identification device 40 will be described. As shown in Figure 1, the color identification system 40 includes a camera 30, a light source 20, a color determination model construction device 100, a color determination model storage unit 50, and a color identification device 200.

[0013] Camera 30 captures an image 31 (color image). Camera 30 outputs the captured image 31 of the object to be captured 10 as image data to a color determination model construction device 100, a color recognition device 200, etc. The captured image 31 includes an image 11 of the object to be captured 10. It is desirable that camera 30 is easy to process with software-based color image processing, highly versatile, and relatively low-cost. Considering these conditions, an RGB camera is used as camera 30 in this embodiment.

[0014] As shown in Figure 2, the color determination model construction device 100 receives an image 31 of a sample 10s from the camera 30, constructs a color determination model, and stores it in the color determination model storage unit 50. The color determination model construction device 100 has a data processing unit 110 and a program memory 120. The data processing unit 110 realizes the following color determination model construction by executing a program stored in the program memory 120. That is, the data processing unit 110 implements the functions of the image acquisition unit 111, the clustering (region classification) unit 112, the type determination unit 113, and the color identification unit 114 both mechanically and controlmatically.

[0015] As shown in Figure 3, the color identification device 200 receives an image 31 of the object to be inspected (hereinafter referred to as workpiece 10w) from the camera 30 and identifies the color of the workpiece by referring to a color determination model stored in the color determination model storage unit 50 via online processing, as will be described later. The color identification device 200 has a data processing unit 210 and a program memory 220. The data processing unit 210 realizes the following color identification functions by executing a program stored in the program memory 220. That is, the data processing unit 210 realizes the functions of the image acquisition unit 211, the clustering (region classification) unit 212, the type determination unit 213, and the color identification unit 214 both mechanically and controlmatically.

[0016] Next, the operation of the color identification system 40 will be described. First, the color identification process will be described, and then the process of constructing the color determination model used in the color identification procedure will be described. The color identification process is performed by the color identification device 200.

[0017] When the color identification device 200 starts the color identification process, it first inputs the captured image 31 of the workpiece 10w and identifies the color of the workpiece 10w by referring to the color determination model stored in the color determination model storage unit 50.

[0018] Next, the color recognition device 200 captures an image of the workpiece 10w using the camera 30 and inputs the captured image 31. Subsequently, the image acquisition unit 211 crops a predetermined portion 32 from the input image 31 of the workpiece 10w.

[0019] Next, the clustering unit 212 obtains classification color values ​​from a predetermined portion and applies a classification algorithm according to the classification color values ​​to classify the predetermined portion into multiple regions. The classification color value is, for example, lightness (L). Specifically, the clustering unit 212 classifies the cropped predetermined portion of the image into k regions by applying the K-Means method to the lightness (L). Note that the processing by the clustering unit 212 is the same as the processing by the clustering unit 112 described later, so a detailed explanation is omitted here. In this embodiment, the clustering unit 212 classifies the predetermined portion into five regions.

[0020] The clustering unit 212 then sets dark regions and bright regions. Note that the dark regions and bright regions will be explained later in the color determination model construction process, so a detailed explanation is omitted here.

[0021] When the clustering unit 212 sets dark and bright regions, the classification unit 213 randomly samples a color value for classification determination from either the dark or bright region of the workpiece 10w and calculates the color difference ΔEab. Here, the color value for classification determination is, for example, a color value in the Lab color space. The color difference ΔEab is the Euclidean distance between the color value for classification determination of the workpiece and the color value for classification determination of the color determination model. When the color value for classification determination of the workpiece is (L', a', b') and the color value for classification determination of the color determination model is (L, a, b), the classification unit 213 calculates the color difference ΔEab using the following equation (1).

[0022]

[0023] The color type determination unit 213 sets a reference value TH based on the tolerance of the color discrimination inspection and calculates the number of colors n whose color difference ΔEab is less than or equal to the reference value TH. This is because the larger the color difference ΔEab, the easier it is to distinguish colors.

[0024] Next, the type determination unit 213 determines whether the number of colors n in which the color ΔEab is less than or equal to the reference value TH is two or more. If the type determination unit 213 determines that there is only one color in which the color difference ΔEab is less than or equal to the reference value TH, it determines the type for color determination of that workpiece to be the dark region. On the other hand, if the type determination unit 213 determines that there are two or more colors in which the color difference ΔEab is less than or equal to the reference value TH, it determines the type for color determination of that workpiece to be the bright region.

[0025] When the type determination unit 213 determines the type for color determination, the color identification unit 214 randomly acquires a color value for color determination from the region of the type for color determination and executes color identification processing of the workpiece using the processing space of the color determination model of the same type. The color value for color determination is, for example, a six-dimensional value (L, a, b, H, S, V) that combines the HSV color space and the Lab color space.

[0026] When the color identification unit 214 starts the color identification process, it obtains a color value for color determination from a sample randomly sampled from the region of the type determined by the type determination (bright region or dark region).

[0027] Here, the color determination color value is a 6-dimensional color determination color value (L, a, b, H, S, V). When the type determination determines that the type is "bright", the color identification unit 214 acquires the color determination color value of a randomly selected sample from the bright region (in this embodiment, the third region 32c). On the other hand, when the type determination determines that the type is "dark", the color identification unit 214 acquires the color determination color value of a randomly selected sample from the dark region (in this embodiment, the fifth region 32e).

[0028] The color value used for color determination has a high-dimensional value that combines multiple different color space values. In this embodiment, as an example, a six-dimensional color value is used, which is a combination of values ​​from the Lab color space (L, a, b) and values ​​from the HSV color space (H, S, V).

[0029] These two color spaces are based on different color systems. Therefore, they can complement each other in areas where representation is difficult. Furthermore, one color space is specialized for representing color differences numerically, while the other is designed to be intuitively understandable to humans.

[0030] Therefore, it is believed that classification will be possible that combines human-perceived color clarity with mechanical-perceived color clarity. Furthermore, the Lab color space is an improvement on the Adams color space, which has been widely used in industry, and inherits the excellent feature that the measured color values ​​do not depend on the illumination light, which is expected to contribute to improved robustness to lighting conditions.

[0031] Next, the color identification unit 214 refers to the processing space of the type determined by the type determination from the color determination model storage unit 50 and identifies the color of the sample's color determination color value, preferably using a supervised learning class classification algorithm. Specifically, the color identification unit 214 identifies the color of the sample's color determination color value using, for example, the k-nearest neighbor (kNN) method as the class classification algorithm.

[0032] Here, the color discrimination performed by the color discrimination unit 214 using the kNN method will be explained with reference to Figure 4. The kNN method is one of the machine learning algorithms based on data proximity. The kNN method is widely used in classification tasks.

[0033] When executing the kNN method, the color recognition unit 214 receives unknown data (in this embodiment, the color value for color determination of the workpiece 10w) as input. Next, the color recognition unit 214 selects the k nearest training data for the unknown data and performs a majority vote among the classes of the selected data. Then, the color recognition unit 214 assigns the unknown data to the class with the most votes. When searching for nearby training data, the color recognition unit 214 uses the distance between data (generally the Euclidean distance).

[0034] Color identification using color determination values ​​is performed by determining the color of a sample using the kNN method, with the processing space (bright region or dark region) of the color determination model constructed by the color determination model construction process and the color determination values ​​of the sample.

[0035] In the example below, the type determined by the type determination is assumed to be, for example, the bright region, and there are two color samples. In the example below, the color values ​​for color determination of color class 1 and color class 2 based on the color samples are plotted in the 6-dimensional bright processing space (L, a, b, H, S, V) of the color determination model. Furthermore, the 6-dimensional color value for color determination (test data) of one sample randomly sampled from the bright region (third region 32c in this embodiment) of the workpiece 10w is mapped to the 6-dimensional bright processing space of the color determination model.

[0036] The color identification unit 214 first calculates the distance (e.g., Euclidean distance) between the sample's test data and the color values ​​(model data) for color determination in the 6-dimensional bright processing space of the color determination model. Next, the color identification unit 214 selects k model data with the smallest data-to-data distance. Subsequently, the color identification unit 214 determines which color class is the most numerous among them. Finally, the color identification unit 214 determines the sample's color by majority vote.

[0037] In the example shown in Figure 4, if k=3, there is one model data for color class 1 and two model data for color class 2. Therefore, the color identification unit 214 assigns color class 2 to the test data. If k=7, there are four model data for color class 1 and three model data for color class 2. Therefore, the color identification unit 214 assigns color class 1 to the test data.

[0038] In this way, the color identification unit 214 identifies the color of a sample from the processing space (bright or dark) on which the model data is plotted and from the sample's test data. Once the color identification unit 214 has finished identifying the color of all samples, it aggregates the identification results and determines the color with the largest proportion as the workpiece color.

[0039] Subsequently, the color identification unit 214 repeats the process for all sampled samples 10s. Then, when color identification is completed for all samples 10s, the color identification unit 214 aggregates the respectively identified colors of all samples, and determines the color with the largest proportion as the color of the workpiece 10w.

[0040] Next, the color determination model construction process will be described. The color determination model is used in the color identification process described above. The color determination model construction process is executed by the color determination model construction apparatus 100. FIG. 5 is a flowchart illustrating an operation of the color determination model construction apparatus.

[0041] When the color determination model construction apparatus 100 starts the color determination model construction process, first, as illustrated in FIG. 6, a camera 30 sequentially images a plurality of color samples 10s prepared in advance one by one (S101). Here, the sample 10s has the same three-dimensional shape as the workpiece 10w. Next, the image acquisition unit 111 inputs a captured image 31 of the set sample 10s (S102). Then, as shown in FIG. 6, the image acquisition unit 111 trims a predetermined portion 32 from the input captured image 31 of the sample 10s (S103). Specifically, the image acquisition unit 111 receives an input of a trimming range for the captured image 31, accompanied by an operator's operation via an input device not shown.

[0042] Next, the clustering unit 112 acquires color values for classification from the predetermined portion 32 (S104). The color value for classification is, for example, lightness (L). Subsequently, the clustering unit 112 applies a classification algorithm according to the color values for classification to classify the predetermined portion 32 into a plurality of regions (S105). Further, a clustering method based on unsupervised learning can be used as the classification algorithm. In the present embodiment, the clustering unit 112 employs a k-means algorithm (k-means method) as an example of the classification algorithm.

[0043] As shown in FIG. 7, the clustering unit 112 classifies the image into k regions using the k-means method based on the lightness L acquired from the image of the trimmed predetermined portion 32 (region classification).

[0044] Subsequently, the clustering unit 112 excludes regions that cannot be used for color identification among the first region 32a to the fifth region 32e according to the average value of lightness (L). Specifically, the clustering unit 212 excludes the region with the highest average lightness. This is because the region with the highest average lightness is highly likely to be a region where light is reflected. Then, the clustering unit 112 sets a type suitable for color determination (here, a dark region or a bright region) from the remaining regions. The dark region is a region having a different color space value (here, lightness L) from the bright region. Specifically, the dark region is a region having a smaller average value of lightness (L) than the bright region. In the present embodiment, the clustering unit 112 sets the fifth region 32e having the smallest average lightness as a dark region, and the third region 32c having the third highest average lightness as a bright region. The dark region and the bright region have different color space values (here, lightness), and the dark region is a region having a smaller average value of lightness than the bright region. In the present embodiment, as an example, the first region 32a having the highest average lightness is excluded as a light reflection region, and the fifth region 32e having the smallest average lightness and the third region 32c having the third highest average lightness are set as a bright region.

[0045] When region classification is completed and a dark region and a bright region are set, the type determination color value acquisition unit 113 randomly samples from one of the dark region and the bright region to acquire the color values (L, a, b) of the sample 10s (S106). Specifically, as shown in Fig. 8A and Fig. 8B, the type determination color value acquisition unit 113 randomly samples from the third region 32c and the fifth region 32e to acquire the color values (L, a, b) of the sample 10s.

[0046] Then, the type determination color value acquisition unit 113 stores the average value of the color values (L, a, b) acquired in step S106 in the color determination model storage unit 50 as the type determination color value Lab associated with the known color of the sample 10s (S107).

[0047] Next, the color determination model generation unit 114 randomly acquires 6D color determination values ​​(L, a, b, H, S, V) from both the dark region and the bright region of the sample 10s, associates them with the known colors of the sample 10s, and registers the 6D color determination values ​​from the dark region in the dark processing space and the 6-element color determination values ​​from the bright region in the bright processing space (S108).

[0048] The color determination model generation unit 114 determines whether or not there are any unprocessed samples 10s remaining (S109). If the color determination model generation unit 114 determines that there are any unprocessed samples 10s remaining, it returns to S101. On the other hand, if the color determination model generation unit 114 determines that there are no unprocessed samples 10s remaining, it determines that the 6-dimensional color determination color values ​​associated with the known colors of all samples have been registered in the dark processing space and the bright processing space, respectively, and stores the generated color determination model in the color determination model storage unit 50 (S110). An example of a color determination model constructed in this way is shown in Figure 9. The color determination model construction device 100 then terminates the color determination model construction process.

[0049] It goes without saying that this disclosure is not limited in any way to the embodiments described above, and can be implemented in various forms as long as they fall within the technical scope of this disclosure.

[0050] In the embodiment described above, the lightness (L) of the Lab color space was used as the classification color value. However, luminance or saturation may also be used as the classification color value.

[0051] In the embodiment described above, the type determination unit 213 may randomly acquire a color value for type determination from the third region 32c and calculate the color difference ΔEab. In that case, if the type determination unit 213 determines that there is only one color that is the same as the workpiece 10w in which the color difference ΔEab is less than or equal to the reference value TH, it may determine the color determination type of the workpiece 10w to be bright. Alternatively, if the type determination unit 213 determines that there are two or more colors in which the color difference ΔEab is less than or equal to the reference value TH, it may determine the color determination type of the workpiece 10w to be dark.

[0052] In the embodiments described above, the clustering unit 212 may exclude both the region with the highest average brightness (the first region 32a in the embodiments described above) and the region with the lowest average brightness (the fifth region 32e in the embodiments described above) from the regions that cannot be used for color identification. In this case, the clustering unit 32 may set the regions to be used for color identification from the remaining regions (the second region 32b, the third region 32c, and the fourth region 32d in the embodiments described above).

[0053] Furthermore, the color identification device 200 may have a setting unit that allows arbitrary changes to be made regarding which areas are designated as reflection areas, dark areas, or bright areas. For example, if the bright area becomes small due to the color or shooting environment, the color identification device 200 may adjust the areas by making the second area 32b, which has the second highest average brightness, and the third area 32c, which has the third highest average brightness, into bright areas. These area adjustments may also be performed by receiving input from an operator to an input device (not shown) connected to the color identification device 200.

[0054] In the above-described embodiment, the classification unit 213 calculated the color difference ΔEab as the Euclidean distance between the color value for classification of the workpiece 10W and the color value for classification of the color classification model. However, the classification unit 213 may also calculate it as the Manhattan distance between both color differences ΔEab. Furthermore, using a kd tree in the neighbor search during color classification in the kNN method can speed up the search time.

[0055] In the embodiment described above, the color identification unit 214 identified the color of the color determination color value using the kNN method. However, the color identification unit 214 may also identify the color of the color determination color value using other classification algorithms such as support vector machines (SVM) or random forests.

[0056] In the embodiments described above, the clustering unit 112 and the clustering unit 212 performed clustering using the K-means method. However, the clustering unit 112 and the clustering unit 212 may also perform clustering using other classification algorithms other than the K-means method, such as the mini-batch K-means method or the Gaussian mixture model (GMM).

[0057] In the embodiment described above, the color determination model generation unit 114 may calculate the distribution and / or change trend of the color values ​​for color determination in the processing space of the color determination model generated as described above, and store it in the color determination model storage unit 50 together with the color determination model. By considering the distribution and / or change trend of the color values ​​for color determination in the color determination model when identifying the color of a workpiece, even colors that are difficult to distinguish can be identified with high accuracy.

[0058] Here, "trend of change" refers to the trend of change (specifically, variance) of color space values ​​(specifically, L values) between adjacent clusters. For example, in Figure 7, if white solid and pearl white are colors that are difficult to distinguish, then the variance of lightness (L values) C between the bright region 32c and the adjacent regions 32b and 32d is ２ Calculate the following. Then, in the bright region, the variance C is used instead of the L value. ２ Using the features as color values ​​for color determination (C ２ In the dark region, the aforementioned color determination color values ​​(L, a, b, H, S, V) are used for color identification. This improves the accuracy of color identification between white solid and pearl white, which are difficult to distinguish.

[0059] 10 Image target 10s Sample 10w Workpiece (inspection target) 20 Illumination light source 30 Camera 31 Captured image 40 Color recognition system 50 Color judgment model storage unit 100 Color judgment model construction device 110 Data processing unit 111 Image acquisition unit 112 Clustering (region classification) unit 113 Color value acquisition unit for type determination 114 Color judgment model generation unit

Claims

1. A color determination model construction device for use in a color identification device that acquires an image of an object to be inspected, acquires a color determination color value used for color identification of the object to be inspected from the image, applies a color identification algorithm to the acquired color determination color value, and performs color identification of the object to be inspected, comprising: an image acquisition unit that acquires an image of a predetermined portion from each of the imaged images of a plurality of sample objects having different known colors; a region classification unit that applies a classification algorithm to the classification color value acquired from the image of the predetermined portion and classifies the image of the predetermined portion into a plurality of regions of different types; and a color determination model generation unit that, for each of the plurality of sample objects, acquires a color determination color value from each of the plurality of regions, registers the color determination color value associated with the known color of the sample object in each of the plurality of different processing spaces, thereby generating the various processing spaces as the color determination model.

2. A color determination model construction apparatus according to claim 1, wherein the color determination model generation unit obtains a first color determination color value from a first region of a first type included in the plurality of regions for each of the plurality of sample objects, obtains a second color determination color value from a second region of a second type included in the plurality of regions, registers the first color determination color value in the first type processing space in association with the known color of each sample object, and registers the second color determination color value in the second type processing space in association with the known color of each sample object.

3. A color determination model construction apparatus according to claim 1, wherein the color determination model generation unit calculates at least one of the distribution and change trend of color determination values ​​obtained from two or more regions among the plurality of regions, and the color determination model includes at least one of the distribution and change trend of the color determination values.

4. A color determination model construction device according to any one of claims 1 to 3, wherein the color determination model generation unit updates the color determination model each time it receives the color determination color value associated with the known color of the sample object.

5. A color determination model construction apparatus according to any one of claims 1 to 3, wherein the color determination model includes a color value for classification determination obtained from one of the plurality of regions, and the color value for classification determination is used to determine a classification suitable for color determination when identifying the color of the object to be inspected.

6. A color determination model construction device according to claim 5, wherein the color value for type determination is a color space value of one color space, and the color determination color value is a color value that combines the respective color space values ​​of a plurality of different color spaces.

7. A color determination model construction apparatus according to claim 6, wherein the first region and the second region are selected from the remaining regions obtained by excluding the region with the largest color space value of the color value for type determination from the plurality of regions.

8. A color determination model construction apparatus according to claim 6, wherein the first region and the second region are selected from the remaining regions obtained by excluding the region with the smallest color space value of the color value for type determination from the plurality of regions.

Images