Image processing device, image processing method and program

Abstract

To provide processing for accurately evaluating the surface characteristics of an object even if the object has a complex shape.SOLUTION: An image processing apparatus has: first acquisition means that acquires image data obtained by picking up an image of an object; second acquisition means that acquires the three-dimensional shape of the object based on the image data; determination means that, based on the three-dimensional shape of the object, determines a substantially flat area in the object; and evaluation means that evaluates the surface characteristics of the substantially flat area in the object.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to image processing techniques for evaluating the surface characteristics of an object. [Background technology]

[0002] In recent years, digital cameras have become widespread and are used for a variety of purposes. For example, Patent Document 1 discloses a technique for measuring the color of an object having an uneven or curved surface using a camera. Furthermore, Patent Document 2 discloses a technique for measuring the image clarity of a sample and the texture of orange peel, etc., using a spectroscopic camera. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2017-036937 [Patent Document 2] Japanese Patent Application Publication No. 2016-038222 Summary of the Invention [Problem to be solved by the invention]

[0004] When performing measurements such as those described in Patent Documents 1 and 2, the object to be measured may have a complex shape due to its design or functionality. In such cases, depending on the area to be evaluated, the shape of the object may affect the evaluation results of surface characteristics such as uneven color caused by painting or orange peel, making it difficult to obtain highly accurate evaluation results.

[0005] Therefore, an object of the present invention is to provide a process for evaluating the surface characteristics of an object with high accuracy, even if the object has a complex shape. [Means for solving the problem]

[0006] In order to solve the above problem, an image processing device according to the present invention includes an acquisition means for acquiring a depth map that indicates the depth from an imaging means to an object for each position by a grid projection method based on image data obtained by an imaging means capturing an image of the object illuminated with grid pattern light, a calculation means for calculating a variance value of the depth for each region in the depth map, a first determination means for determining a region where the variance value is equal to or less than a predetermined threshold as a flat region in the object, and a first determination means for acquiring information that indicates a correspondence relationship between the image capture region captured by the imaging means and an illuminated region of square wave pattern light for evaluating the degree of orange peel, and calculating the coordinates of the flat region in the illuminated region of the square wave pattern light based on the information, and The irradiation area of ​​the rectangular wave pattern light corresponds to the flat area. A pattern image for irradiating the rectangular wave pattern light Size or position of and an evaluation means for evaluating the degree of orange peel on the basis of image data obtained by capturing an image of the rectangular wave pattern light irradiated onto the irradiation area on the object. [Effects of the Invention]

[0007] According to the present invention, it is possible to evaluate the surface characteristics of an object with high accuracy, even if the object has a complex shape. [Brief explanation of the drawings]

[0008] [Figure 1] Schematic diagram showing the configuration of a color measurement system including an image processing device [Figure 2] Block diagram showing the hardware configuration of an image processing device [Figure 3] Schematic diagram showing the application window [Figure 4] Block diagram showing the functional configuration of an image processing device [Figure 5] Flowchart showing a process for determining an evaluation area [Figure 6] FIG. 10 is a schematic diagram illustrating a process for calculating a variance value of depth information; [Figure 7] Flowchart showing measurement processing [Figure 8] FIG. 10 is a schematic diagram illustrating a process for displaying a color difference map. [Figure 9] 1 is a flowchart showing a process executed by an image processing apparatus; [Figure 10] Flowchart showing measurement processing [Figure 11] FIG. 1 is a schematic diagram illustrating a correspondence relationship between an imaging area and an irradiation area of ​​pattern light. [Figure 12] Schematic diagram for explaining an evaluation area in an irradiation area of ​​pattern light [Figure 13] Schematic diagram for explaining processing for converting a pattern image [Figure 14] Schematic diagram for explaining the process of calculating an orange peel evaluation value [Figure 15] Schematic diagram showing an example of displaying evaluation results DETAILED DESCRIPTION OF THE INVENTION

[0009] The present embodiment will be described below with reference to the drawings. Note that the following embodiment does not necessarily limit the present invention. Furthermore, not all of the combinations of features described in the present embodiment are necessarily essential to the solution of the present invention.

[0010] [First embodiment] When measuring uneven color or orange peel (orange peel) on the surface of an industrial product such as an automobile, it is necessary to specify an approximately flat area without design indentations as the evaluation area. This is because including indentations in the evaluation area would affect the color measurement values ​​not only due to color unevenness caused by the paint but also due to changes in color tone caused by the indentations. Similarly, in orange peel evaluation, if an evaluation area other than an approximately flat area is specified, the degree of light blur caused by fine irregularities cannot be accurately evaluated, and an appropriate orange peel evaluation value cannot be obtained. Therefore, in this embodiment, an approximately flat area on the target object is identified as an appropriate evaluation area, and the surface characteristics of the identified evaluation area are evaluated. Note that in this embodiment, an approximately flat area is defined as an area that does not include shapes such as indentations related to design or functionality, but includes fine irregularities that cause uneven color or orange peel due to the paint.

[0011] <Color measurement system configuration> 1 is a diagram showing the configuration of a color measurement system according to this embodiment. The color measurement system includes an image processing device 101, an imaging device 102, and a projection device 103. The image processing device 101 controls the projection device 103 to irradiate a target object 104 with pattern light 105. The image processing device 101 controls the imaging device 102 to capture an image of the target object 104 irradiated with the pattern light 105, and determines a substantially flat region of the target object 104 as a region suitable for generating a color difference map based on the image data obtained by capturing the image.

[0012] <Hardware configuration of image processing device> FIG. 2 is a block diagram showing the hardware configuration of the image processing device 101. The CPU 201 uses the RAM 203 as a work memory and executes an operating system (OS) and various programs stored in the ROM 202 and a hard disk drive (HDD) 205. The CPU 201 controls each component via a system bus 210 such as a peripheral component interconnect (PCI) bus. The CPU 201 accesses the HDD 205 via the system bus 210 and an HDD interface (I / F) 204. The HDD I / F 204 is an interface for connecting a secondary storage device such as the HDD 205 or an optical disk drive, and is, for example, a serial ATA (SATA) interface. The CPU 201 can read data from and write data to the HDD 205 via the HDD I / F 204. The CPU 201 also displays a user interface (UI) and processing results on a display 209 via a graphics accelerator 208. Furthermore, the CPU 201 receives instructions from the user via an input device 207 such as a mouse or keyboard connected to the USB I / F 206 .

[0013] <Operation of image processing device> First, the image processing program stored in the HDD 205 is executed by the CPU 201 in response to a user instruction via the input device 207, and an application window 301 shown in FIG. 3 is displayed on the display 209. The image display window 302 displays an image captured by the imaging device 102. The evaluation area determination button 303 is a button used by the user to instruct the start of processing to determine an evaluation area. After the evaluation area determination button 303 is pressed, the three-dimensional shape of the target object is acquired using the imaging device 102 and the projection device 103, and an evaluation area is determined based on the three-dimensional shape of the target object. The measurement button 304 is a button used by the user to instruct the start of processing to determine the evaluation area. After the measurement button 304 is pressed, a color difference map is generated for the determined evaluation area, and the color difference map is displayed in the image display window 302 as an evaluation result of the surface characteristics.

[0014] <Functional configuration of image processing device> 4 is a diagram showing the functional configuration of the image processing device 101. The image processing device 101 has an image acquisition unit 401, a shape acquisition unit 402, a region determination unit 403, and an evaluation unit 404. The image acquisition unit 401 functions as a control unit that controls the imaging device 102 and the projection device 103, and causes the projection device 103 to irradiate a target object with patterned light and causes the imaging device 102 to capture an image of the target object irradiated with the patterned light. The image acquisition unit 401 acquires image data obtained by imaging from the imaging device 102. Note that if the irradiation of patterned light and the imaging have been performed in advance, the image acquisition unit 401 may simply acquire image data from a storage device such as the HDD 205.

[0015] The shape acquisition unit 402 acquires the three-dimensional shape of the target object based on the image data acquired by the image acquisition unit 401. In this embodiment, a known grid projection method is used to acquire the three-dimensional shape. Therefore, the pattern light for acquiring the three-dimensional shape that the projection device 103 irradiates onto the target object is a grid pattern light. The method for acquiring the three-dimensional shape is not limited to the grid projection method, and other known methods, such as a phase shift method using a stripe pattern light, may also be used. The region determination unit 403 determines a substantially flat region of the target object based on the three-dimensional shape of the target object acquired by the shape acquisition unit 402. The evaluation unit 404 evaluates the surface characteristics of the target object using the substantially flat region determined by the region determination unit 403 as an evaluation region. In this embodiment, the surface characteristic of the target object to be evaluated is color unevenness due to painting, and the evaluation unit 404 functions as a color difference map generator that generates a color difference map in which each pixel has a color difference between a predetermined reference value and the color value of the evaluation region.

[0016] <Process for determining evaluation area> The following describes the flow of processing for determining an evaluation area after the evaluation area determination button 303 is pressed, using the flowcharts in Figures 5(a) and 5(b). The processing shown in the flowchart in Figure 5(a) begins when the user inputs an instruction via the input device 207 and the CPU 201 accepts the input instruction. Below, each step (process) is represented by adding an S before the reference number.

[0017] In S501, the image acquisition unit 401 controls the projection device 103 to irradiate the target object with pattern light for acquiring a three-dimensional shape. In this embodiment, the pattern light is a grid pattern light. In S502, the image acquisition unit 401 controls the imaging device 102 to capture an image of the target object irradiated with the pattern light. In S503, the shape acquisition unit 402 acquires the three-dimensional shape of the target object based on image data obtained by imaging. In this embodiment, the three-dimensional shape of the target object is acquired by estimating the phase distribution of the grid according to a known grid projection method.

[0018] In S504, the region determination unit 403 determines a substantially flat region as the evaluation region based on the three-dimensional shape of the target object. Details of the processing of S504 will be described below using the flowchart in FIG. 5(b). In S5041, the region determination unit 403 acquires a depth map for identifying the three-dimensional shape of the target object. The depth map is data having depth information z(x, y) for each pixel, and is generated in S503. Here, (x, y) is the pixel position in the captured image, and z(x, y) is the depth information for that pixel position.

[0019] In S5042, the region determination unit 403 applies low-pass filtering to the depth map. This low-pass filtering reduces noise contained in the depth information, making it possible to identify a substantially flat region with higher accuracy. In this embodiment, a 10x10 average filter is used as the low-pass filter, but other low-pass filters such as a Gaussian filter may also be used. The average filter in this embodiment is expressed by the following equation (1):

[0020]

number

[0021] Furthermore, if the depth information after the low-pass filter processing is zl(x, y), zl(x, y) is expressed by equation (2).

[0022]

number

[0023] In S5043, the region determination unit 403 calculates the variance of the depth information zl(x,y) for each rectangular region in the depth map. The process of calculating the variance of the depth information zl(x,y) will be described using the schematic diagram of FIG. 6. FIG. 6(a) is a schematic diagram that visualizes the depth map. In FIG. 6(a), the door part of a car and bulges due to design are acquired as the depth map, and approximately flat areas and non-approximately flat areas are mixed. In S5043, the region determination unit 403 divides the depth map into 16 regions (4x4) and calculates the variance of the depth information zl(x,y) in each divided region. If the variance of the divided region in the i-th row and j-th column is V(i,j), V(i,j) is expressed by Equation (3).

[0024]

number

[0025] Here, W is the number of horizontal pixels in the depth map, and H is the number of vertical pixels. N is the total number of pixels in the divided region, and N = H × (W / 16). ave(i,j) is the average value of the depth information zl(x,y) in the divided region (i,j). Figure 6(b) is a schematic diagram visualizing the variance value of each divided region. Darker areas indicate higher variance values, and lighter areas indicate lower variance values.

[0026] In S5044, the region determination unit 403 determines a substantially flat region based on the variance value V(i,j). In this embodiment, a region where the variance value V(i,j) of the depth information is equal to or less than a predetermined threshold is determined to be a substantially flat region. In the example of FIG. 6(a), the regions corresponding to the lower and upper left corners of the depth map have low variance values ​​V(i,j) because there are no car doors or linear bulges in the regions. Therefore, by the above process of S504, the lower and upper left corners of the image are determined to be substantially flat regions.

[0027] <Measurement processing> The flow of the measurement process after the measurement button 304 is pressed will be described below with reference to the flowcharts of Figures 7(a) and 7(b). The process shown in the flowchart of Figure 7(a) starts when the user inputs an instruction via the input device 207 and the CPU 201 accepts the input instruction. In S701, the image acquisition unit 401 controls the imaging device 102 to capture an image of a target object that is not irradiated with patterned light.

[0028] In S702, the evaluation unit 404 cuts out the region determined to be a substantially flat region in S504 from the captured image. In the following, the pixel values ​​(R, G, B) of the xth pixel horizontally and the yth pixel vertically from the top left in the cut-out image are defined as R(x, y), G(x, y), and B(x, y), respectively.

[0029] In S703, the evaluation unit 404 generates a color difference map based on the cut-out image corresponding to the substantially flat region. * a * b * Convert to value, L * a * b *The color difference is calculated based on the values. First, the evaluation unit 404 normalizes each of the 8-bit values ​​R(x,y), G(x,y), and B(x,y) to a value between 0 and 1. The normalized values ​​are corrected according to the gamma value of the imaging device 102 to calculate R'(x,y), G'(x,y), and B'(x,y), which are linear luminance values. R'(x,y), G'(x,y), and B'(x,y) are expressed as in equation (4).

[0030]

number

[0031] Next, R'(x,y), G'(x,y), and B'(x,y) are converted into tristimulus values ​​X, Y, and Z using equation (5).

[0032]

number

[0033] Here, the color transformation matrix M is a 3x3 color transformation matrix for converting colors from (R, G, B) to (X, Y, Z). For example, the color transformation matrix M defined in ITU-R BT.709 is expressed by the following equation (6).

[0034]

number

[0035] The color transformation matrix M may be the matrix shown in equation (6), or may be calculated for each imaging device and imaging conditions based on the imaging results of the color chart. * a * b * If the tristimulus values ​​X, Y, and Z of the reference white are (Xw, Yw, Zw), then L * a * b * The values ​​are calculated using the following equations (7), (8), (9), and (10).

[0036]

number

[0037] Next, the evaluation unit 404 evaluates the calculated L * a * b * In this embodiment, the reference value is a general L value of the body of an automobile, which is the target object. * a * b * The reference value L is set in advance. * a * b * The value (L * 0,a * 0,b * 0), and L at any pixel position (x, y) in the image * a * b * The value (L * (x,y) ,a * (x,y) ,b * (x,y) ) The color difference ΔE at (x, y) (x,y) is expressed by equation (11).

[0038]

number

[0039] In S704, the evaluation unit 404 displays the evaluation results on the display 209. Details of the processing in S704 will be described below with reference to the flowchart in FIG. 7(b). In S7041, the evaluation unit 404 acquires image data captured in S701. In S7042, the evaluation unit 404 replaces the evaluation area in the image represented by the image data with a color difference map. FIG. 8(a) is a schematic diagram showing the captured image, and the area surrounded by a dotted line represents the area determined to be a substantially flat area in S504. By replacing this substantially flat area with the color difference map, the composite image shown in FIG. 8(b) is obtained. Here, the color difference map is expressed so that areas with smaller color differences are darker and areas with larger color differences are brighter. In S7043, the evaluation unit 404 presents the composite image generated in S7042 to the user. Specifically, the evaluation unit 404 displays the composite image in the image display window 302.

[0040] <Effects of the first embodiment> As described above, the image processing device in this embodiment acquires image data obtained by capturing an image of an object. Based on the image data, the three-dimensional shape of the object is acquired. Based on the three-dimensional shape of the object, a substantially flat region of the object is determined. The surface characteristics of the substantially flat region of the object are evaluated. By identifying a substantially flat region suitable for evaluating surface characteristics, the influence of shapes such as depressions and bulges related to design and functionality on the evaluation of surface characteristics can be reduced. Therefore, even if the object to be evaluated for surface characteristics has a complex shape, the surface characteristics of the object can be evaluated with high accuracy.

[0041] <Modification> In this embodiment, the process of determining the evaluation area and the measurement process are performed separately in response to a user's instruction, but steps S501 to S704 may be performed as a series of processes. Fig. 9 is a flowchart showing the series of processes. By performing step S701 before step S501, the imaging process without patterned light irradiation and the imaging process with patterned light irradiation can be performed together.

[0042] [Second embodiment] In the first embodiment, color unevenness due to painting was evaluated as the surface characteristic of the target object. In this embodiment, orange peel is evaluated as the surface characteristic of the target object. Note that the hardware configuration and functional configuration of the image processing device 101 in this embodiment are the same as those in the first embodiment, so a description thereof will be omitted. Below, differences between this embodiment and the first embodiment will be mainly described. Note that the same components as those in the first embodiment will be described using the same reference numerals.

[0043] <Measurement processing> The following describes the flow of measurement processing after pressing the measurement button 304, using the flowchart in Fig. 10(a). The processing shown in the flowchart in Fig. 10(a) starts when the user inputs an instruction via the input device 207 and the CPU 201 accepts the input instruction.

[0044] In S1001, the image acquisition unit 401 determines a pattern light for orange skin evaluation based on the evaluation area determined in S504. The process of determining the pattern light for orange skin evaluation will be described in detail later. In S1002, the image acquisition unit 401 controls the projection device 103 to irradiate the target object with the pattern light for orange skin evaluation. In S1003, the image acquisition unit 401 controls the imaging device 102 to capture an image of the target object irradiated with the pattern light for orange skin evaluation. The image acquisition unit 401 also captures an image of the target object not irradiated with the pattern light. In S1004, the evaluation unit 404 cuts out the evaluation area determined in S504 from the captured image for orange skin evaluation. In S1005, the evaluation unit 404 calculates an orange skin evaluation value based on the cut-out image corresponding to the approximately flat area. The process of calculating the orange skin evaluation value will be described in detail later. In S1006, the evaluation unit 404 displays the evaluation result on the display 209. The process of displaying the evaluation result will be described in detail later.

[0045] <Process for determining pattern light for orange skin evaluation> The following describes the process flow for determining the pattern light for orange peel evaluation using the flowchart in FIG. 10(b). In S10011, the image acquisition unit 401 acquires the correspondence relationship between the image capture area and the area illuminated by the pattern light. FIG. 11 is a schematic diagram showing the correspondence relationship between the image capture area of ​​the target object and the area illuminated by the pattern light. The image acquisition unit 401 acquires the coordinates (TLx, TLy) and coordinates (BRx, BRy) in FIG. 11 as the correspondence relationship. In FIG. 11, the image capture area has coordinates (0, 0) at the top left and coordinates (Wc, Hc) at the bottom right. The area illuminated by the pattern light has coordinates (0, 0) at the top left and coordinates (Wp, Hp) at the bottom right. The area surrounded by the dotted line represents the position of the image capture area in the area illuminated by the pattern light. The coordinates (0, 0) at the top left of the image capture area correspond to coordinates (TLx, TLy) in the area illuminated by the pattern light. The coordinates (Wc, Hc) at the bottom right of the image capture area correspond to the coordinates (BRx, BRy) in the area illuminated by the patterned light. The coordinates (TLx, TLy) and (BRx, BRy) are stored in advance in the ROM 202 as information representing the correspondence between the image capture area and the area illuminated by the patterned light.

[0046] In S10012, the image acquisition unit 401 calculates the coordinates of the evaluation area determined in S504 in the area irradiated with the patterned light based on the correspondence between the image capture area and the area irradiated with the patterned light. sp ,y sp ) and the bottom right coordinate (x ep ,y ep ) is expressed by the following equation (12).

[0047]

number

[0048] Here, the coordinate (x s ,y s ) is the upper left coordinate of the evaluation area in the imaging area, and the coordinate (x e ,y e) represents the bottom right coordinate. sp ,y sp ) and coordinates (x ep ,y ep ) The area surrounded by the dotted line is the evaluation area.

[0049] In S10013, the image acquisition unit 401 acquires the coordinates (x sp ,y sp ) and coordinates (x ep ,y ep ) based on the evaluation area defined by the equation (x, y, y, y) and (y, y, y) respectively. Fig. 13 is a schematic diagram for explaining the process of converting a pattern image. Fig. 13(a) shows the pattern image before conversion. In this embodiment, an image of a square wave pattern is used to measure orange peel. Fig. 13(b) shows the pattern image converted based on the evaluation area. First, the image acquisition unit 401 calculates the size of the pattern image by calculating the height Hc and width Wc from the height (y, y) using the known nearest neighbor method. ep -y sp ), width (x ep -x sp Next, the image acquisition unit 401 converts the coordinates of the top left corner of the converted pattern image into (x sp ,y sp 13(b) is generated. In this embodiment, image transformation is performed by enlargement and reduction processing, but if the pattern image is tilted with respect to the evaluation area, rotation correction by affine transformation or projective transformation may be performed.

[0050] <Process for calculating the Yuzu Skin evaluation value> The following describes the flow of processing for calculating the orange skin evaluation value with reference to the flowchart in FIG. 10(c). In S10051, the evaluation unit 404 acquires the image cropped in S1004. In S10052, the evaluation unit 404 determines the calculation target range for the orange skin evaluation value by specifying a single straight line in the cropped image. FIG. 14(a) schematically shows the calculation target range for the orange skin evaluation value in the cropped image, which is displayed in the image display window 302. Based on a user instruction, the evaluation unit 404 specifies a start point 1402 and an end point 1403 of the line for determining the calculation target range for the orange skin evaluation value in the cropped image 1401. Using a preset value w for the line connecting the start point 1402 and the end point 1403, the evaluation unit 404 determines a range 1404 expanded by a width of ±w pixels on both sides as the calculation target range. Note that in this embodiment, the user is prompted to specify the start point and end point of the line, but the user may also be prompted to specify four vertices of the calculation target range. Alternatively, the position may be determined automatically according to the relative position between the evaluation area and the pattern image without receiving a user instruction.

[0051] In S10053, the evaluation unit 404 calculates the average brightness value of the calculation target range in the cut-out image. In S10054, the evaluation unit 404 uses the calculated average brightness value as a threshold to binarize the pixel values ​​of the cut-out image. In S10055, the evaluation unit 404 detects edges in the image whose pixel values ​​have been binarized. FIG. 14(b) shows the image after the binarization process. Here, the coordinates of the starting point 1402 are (x st ,y st ), and the coordinates of the end point 1403 are (x ed ,y ed The evaluation unit 404 calculates the value of y as y st From y edThe x value is changed for each y value, and the point where the value differs from that of the adjacent pixel is determined to be an edge point. However, if there are multiple edge points for the same y value, the edge point is determined to be the one with the shortest distance to the line connecting the start point and end point. Note that in this embodiment, pixels in a binary image whose value differs from that of adjacent pixels are determined to be edge points, but the extracted image may also be quadratically differentiated and points where the output value changes from positive to negative, i.e., inflection points, may be detected as edge points.

[0052] In S10056, the evaluation unit 404 derives an approximate line for the detected edge. In S10057, the evaluation unit 404 calculates the distance between the edge detected in S10055 and the approximate line derived in S10056.

[0053] In S10058, the evaluation unit 404 calculates an orange skin evaluation value based on the distance between the edge and the approximation line. In this embodiment, when the orange skin degree of the target object is small, the spatial variation of the edge points is small. Therefore, the evaluation unit 404 calculates the orange skin evaluation value E so that the orange skin evaluation value is 0 when a set of edge points is located on a straight line, and the orange skin evaluation value increases as the deviation from the straight line increases. Specifically, the orange skin evaluation value E is calculated using the following equation (13):

[0054]

number

[0055] where d j is the distance between the approximate line obtained by linearly approximating the set of edge points using the least squares method and the edge point detected when the y coordinate is j.

[0056] <Process to display evaluation results> The flow of processing for displaying the evaluation results will be described below with reference to the flowchart in FIG. 10(d). In S10061, the evaluation unit 404 acquires image data obtained by capturing an image of the target object not irradiated with patterned light in S1003. In S10062, the evaluation unit 404 superimposes a dotted line representing the evaluation area and an orange peel evaluation value on the image represented by the image data. In S10063, the evaluation unit 404 presents the composite image generated in S10062 to the user. Specifically, the composite image is displayed in the image display window 302. FIG. 15 is a diagram showing an example of displaying the evaluation results.

[0057] <Effects of the second embodiment> As described above, the image processing device in this embodiment changes the pattern light for evaluating orange peel according to the size and position of the evaluation area determined based on the three-dimensional shape of the target object. As a result, even if the target object for evaluating orange peel has a complex shape, the pattern light for evaluating orange peel can be irradiated at an appropriate position, making it possible to evaluate the orange peel of the target object with high accuracy.

[0058] <Modification> In this embodiment, the degree of orange peel of the target object is evaluated, but the evaluation of color unevenness as in the first embodiment may also be performed at the same time. In this case, the color difference map and the orange peel evaluation value are displayed on the display 209 together as the evaluation results.

[0059] [Other embodiments] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. [Explanation of symbols]

[0060] 101 Image processing device 401 Image acquisition unit 402 Shape acquisition section 403 Area determination section 404 Evaluation Department

Claims

1. an acquisition means for acquiring a depth map that indicates a depth from the imaging means to the object for each position by a grid projection method based on image data obtained by imaging the object illuminated with grid pattern light; a calculation means for calculating a variance value of the depth for each region in the depth map; a first determination means for determining a region in which the variance value is equal to or less than a predetermined threshold as a flat region in the object; a second determination means for obtaining information indicating a correspondence relationship between an image capture area captured by the image capture means and an irradiation area of ​​square wave pattern light for evaluating the degree of orange peel, calculating coordinates of the flat area in the irradiation area of ​​the square wave pattern light based on the information, and converting a size or position of a pattern image for irradiating the square wave pattern light based on the coordinates of the flat area so that the irradiation area of ​​the square wave pattern light corresponds to the flat area; and an evaluation means for evaluating the degree of orange peel on the basis of image data obtained by capturing an image of the rectangular wave pattern light irradiated onto the irradiation area on the object; 1. An image processing device comprising:

2. 2. The image processing apparatus according to claim 1, wherein the evaluation means calculates a color difference between a color value of the flat area and a reference value.

3. 3. The image processing apparatus according to claim 2, wherein said evaluation means displays a color difference map having said color difference for each pixel.

4. 4. The image processing apparatus according to claim 3, wherein the evaluation means replaces a region corresponding to the flat region in the image represented by the image data with the color difference map, and displays the image.

5. 5. The image processing device according to claim 1, wherein the evaluation means displays the image represented by the image data together with an orange peel evaluation value indicating the degree of the orange peel effect.

6. the acquisition means acquires the depth map based on the image data obtained by capturing an image of the object before painting; 6. The image processing apparatus according to claim 1, wherein the evaluation means evaluates the surface characteristics of the flat area of ​​the object after painting.

7. The object is an object composed of a plurality of parts, 7. The image processing apparatus according to claim 1, wherein the flat region is a region that does not include a boundary between parts.

8. an acquisition means for acquiring a depth map representing a depth from the imaging means to the object for each position by a phase shift method based on image data obtained by imaging the object illuminated with stripe pattern light; a calculation means for calculating a variance value of the depth for each region in the depth map; a first determination means for determining a region in which the variance value is equal to or less than a predetermined threshold as a flat region in the object; a second determination means for obtaining information indicating a correspondence relationship between an image capture area captured by the image capture means and an irradiation area of ​​square wave pattern light for evaluating the degree of orange peel, calculating coordinates of the flat area in the irradiation area of ​​the square wave pattern light based on the information, and converting a size or position of a pattern image for irradiating the square wave pattern light based on the coordinates of the flat area so that the irradiation area of ​​the square wave pattern light corresponds to the flat area; and an evaluation means for evaluating the degree of orange peel on the basis of image data obtained by capturing an image of the rectangular wave pattern light irradiated onto the irradiation area on the object; 1. An image processing device comprising:

9. A program for causing a computer to function as each of the means of the image processing apparatus according to any one of claims 1 to 8.

10. an acquisition step of acquiring a depth map representing a depth from the imaging means to the object for each position by a grid projection method based on image data obtained by imaging the object illuminated with grid pattern light by the imaging means; a calculation step of calculating a variance of the depth for each region in the depth map; a first determination step of determining a region in which the variance value is equal to or less than a predetermined threshold as a flat region in the object; a second determination step of acquiring information representing a correspondence relationship between an image capture area captured by the image capture means and an irradiation area of ​​square wave pattern light for evaluating the degree of orange peel, calculating coordinates of the flat area in the irradiation area of ​​the square wave pattern light based on the information, and converting a size or position of a pattern image for irradiating the square wave pattern light based on the coordinates of the flat area so that the irradiation area of ​​the square wave pattern light corresponds to the flat area; an evaluation step of evaluating the degree of the orange peel effect based on image data obtained by capturing an image of the rectangular wave pattern light irradiated onto the irradiation area on the object; An image processing method comprising:

11. an acquisition step of acquiring a depth map representing a depth from the imaging means to the object for each position by a phase shift method based on image data obtained by imaging the object illuminated with stripe pattern light by the imaging means; a calculation step of calculating a variance of the depth for each region in the depth map; a first determination step of determining a region in which the variance value is equal to or less than a predetermined threshold as a flat region in the object; a second determination step of acquiring information representing a correspondence relationship between an image capture area captured by the image capture means and an irradiation area of ​​square wave pattern light for evaluating the degree of orange peel, calculating coordinates of the flat area in the irradiation area of ​​the square wave pattern light based on the information, and converting a size or position of a pattern image for irradiating the square wave pattern light based on the coordinates of the flat area so that the irradiation area of ​​the square wave pattern light corresponds to the flat area; an evaluation step of evaluating the degree of the orange peel effect based on image data obtained by capturing an image of the rectangular wave pattern light irradiated onto the irradiation area on the object; An image processing method comprising:

Images