Image processing device, image processing method, and program

Abstract

To accurately evaluate a surface state of an object having a curved surface structure.SOLUTION: An image processing apparatus acquires images obtained by imaging an object to be evaluated while changing the focal position of an imaging apparatus, and acquires, from these images obtained by imaging the object while changing the focal position, an optical profile representing the attribute of an image reflected on a surface of the object. The image processing apparatus composites the optical profiles acquired for the respective images obtained by imaging the object while changing the focal position to create a composite profile, and acquires an evaluation value representing the state of the surface of the object based on the composite profile.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a technique for evaluating the surface condition of an object. [Background technology]

[0002] In the field of industrial design, it is important to investigate the surface condition of an object because the surface condition contributes to the design. Patent Document 1 discloses a method for evaluating the surface condition of an object by capturing an image of a projection pattern reflected on the surface of the object with a camera and using the amount of shape variation of the projection pattern from the captured image and sensory evaluation information obtained in advance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2017-173300 Summary of the Invention [Problem to be solved by the invention]

[0004] However, when the surface of an object is not flat but has a curved structure, the depth of the object surface exceeds the depth of field of the camera, causing part of the captured image to be blurred. When part of the captured image is blurred, the amount of shape variation of the projection pattern cannot be correctly obtained, and an accurate evaluation value cannot be calculated. In such a situation, conventional techniques such as those disclosed in Patent Document 1 cannot adequately evaluate the condition of the object's surface.

[0005] Therefore, an object of the present invention is to enable accurate evaluation of the surface condition of an object having a curved surface structure. [Means for solving the problem]

[0006] The image processing device of the present invention comprises: multiple Focus position in The object to be evaluated is photographed. The multiple image acquisition means for acquiring an image; Based on multiple images , which represent the attributes of the image reflected on the surface of the object. Multiple a profile acquisition means for acquiring an optical profile; Multiple The optical imaging system is characterized by comprising a synthesis means for synthesizing optical profiles to generate a synthetic profile, and an evaluation value acquisition means for acquiring an evaluation value representing the surface condition of the object based on the synthetic profile. [Effects of the Invention]

[0007] According to the present invention, it is possible to accurately evaluate the surface condition of an object having a curved surface structure. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 illustrates an example of the configuration of an image processing device. [Figure 2] 4 is a flowchart of image processing according to the first embodiment. [Figure 3] FIG. 10 is a diagram illustrating an example of a method for capturing an image of an object to be evaluated. [Figure 4] FIG. 10 is a diagram illustrating an example of a GUI. [Figure 5] FIG. 10 is a diagram showing state transitions in a GUI. [Figure 6] FIG. 10 is a diagram used to explain the range of evaluation targets. [Figure 7] 10 is a flowchart of an optical profile calculation process. [Figure 8] 10 is a flowchart of an optical profile synthesis process. [Figure 9] FIG. 10 is a schematic diagram illustrating an example of optical profile synthesis. [Figure 10] 10 is a flowchart of an evaluation value calculation process. [Figure 11] 10 is a block diagram showing the functional configuration of an image processing apparatus according to a second embodiment. [Figure 12] 10 is a flowchart of image processing according to the second embodiment. [Figure 13] 10 is a flowchart of a synthesis determination process according to the second embodiment. [Figure 14]FIG. 10 is a schematic diagram illustrating an example of a maximum difference in variance in optical profile synthesis determination. [Figure 15] FIG. 10 is a diagram illustrating an example of a threshold value for determining optical profile synthesis. [Figure 16] FIG. 10 is a block diagram showing the functional configuration of an image processing apparatus according to a third embodiment. [Figure 17] 10 is a flowchart of image processing according to the third embodiment. [Figure 18] 10 is a flowchart of a synthesis determination process according to the third embodiment. [Figure 19] FIG. 10 is a diagram showing an example of the relationship between imaging conditions and depth of field. [Figure 20] 10 is a flowchart of image processing according to the fourth embodiment. [Figure 21] FIG. 10 is a diagram showing an example of a GUI that notifies a user of a synthesis determination result. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments will be described with reference to the drawings. Note that the following embodiments do not limit the present invention, and not all of the combinations of features described in the embodiments are necessarily essential to the present invention. The configurations shown in the following embodiments are merely examples, and the present invention is not limited to the configurations shown in the drawings.

[0010] <Explanation of evaluation examples of object surface conditions> This embodiment describes image processing for evaluating the so-called "yellow peel" effect, an example of an object surface condition that contributes to aesthetic appeal in the field of industrial design. Also known as orange peel or gun peel, this effect occurs when fine irregularities appear on the surface of an object. For example, when painting an automobile body, paint applied using a spray gun or the like may harden before becoming smooth on the body surface. This results in fine irregularities on the body surface. This condition, known as "yellow peel," deviates from the intended smooth, glossy surface and reduces the aesthetic appeal of the automobile. In this embodiment, an evaluation value is calculated to evaluate the degree of orange peel on the object surface and presented to the user. This allows the user to be informed of the extent to which unintended orange peel has occurred. Note that orange peel can occur in a variety of different patterns, ranging from large to small irregularities. In the following description, the period of irregularities in the orange peel pattern will be referred to as the wavelength, with large orange peel wavelengths referred to as low frequencies and small orange peel wavelengths referred to as high frequencies. In this embodiment, the surface of an object to be evaluated is imaged using an imaging device, and an evaluation value for the surface condition of the object is calculated from the image. In this embodiment, multiple evaluation values ​​are calculated according to the wavelength of, for example, orange peel. As an example, an evaluation value of "1" is used to evaluate an orange peel wavelength of 0.1 mm to 0.3 mm, and an evaluation value of "2" is used to evaluate an orange peel wavelength of, for example, 0.3 mm to 1.0 mm.

[0011] For example, if the object surface has a curved structure rather than a flat surface and the depth of the object surface relative to the imaging device exceeds the depth of field of the imaging device, a portion of the object will be blurred in the captured image, making it impossible to calculate an accurate evaluation value. Japanese Patent Application Laid-Open Publication No. 2005-277813 discloses a method of combining in-focus portions of multiple images captured while shifting the focus position. By using this method to combine multiple images captured while shifting the focus position, it is possible to obtain a blur-free image even for a three-dimensional object. Therefore, it is conceivable to calculate an evaluation value for the surface condition of an object using a composite image obtained by combining multiple images captured while shifting the focus position. However, this method requires a large amount of calculation for the image combination process.

[0012] In this embodiment, too, a plurality of captured images of the projection pattern reflected on the surface of the object to be evaluated are obtained by shifting the focus position of the imaging device. In this embodiment, the image processing device does not combine the plurality of captured images, but instead performs a profile acquisition process to acquire an optical profile representing the attributes of the projection pattern image reflected on the surface of the object to be evaluated from each captured image.

[0013] In this embodiment, the optical profile is a luminance profile obtained by one-dimensionally profiling the luminance fluctuations of a projected pattern image reflected on the surface of an object to be evaluated. In this embodiment, for example, an illumination image reflected on the surface of an illuminated object is converted into a projected pattern image, and an optical profile of the projected pattern image is calculated. Furthermore, the image processing device of this embodiment generates a composite profile by combining multiple optical profiles calculated from multiple images captured at different focus positions, using in-focus portions corresponding to each focus position. In other words, a composite profile synthesized using in-focus portions of multiple optical profiles for each focus position corresponds to an optical profile focused on the entire surface of an object having a curved structure. The image processing device of this embodiment then calculates an evaluation value for the object surface based on the composite profile. This allows the image processing device of this embodiment to calculate a highly accurate evaluation value even when the object to be evaluated has a curved structure. Furthermore, in this embodiment, multiple optical profiles are combined rather than multiple captured images, thereby enabling the evaluation value to be calculated with a small amount of computation.

[0014] <Hardware configuration of image processing device> FIG. 1(a) is a diagram showing an example of the hardware configuration of an image processing device 1 according to this embodiment. The image processing device 1 includes a CPU 101, a ROM 102, and a RAM 103. The image processing device 1 also includes a VC (video card) 104, a general-purpose I / F (interface) 105, a SATA (serial ATA) I / F 106, and a NIC (network interface card) 107. The CPU 101 uses the RAM 103 as a work memory to execute an OS (operating system) and various programs stored in the ROM 102, a HDD (hard disk drive) 113, etc. The CPU 101 also controls each component via a system bus 108. Note that, in the processing of the flowcharts described below, program codes stored in the ROM 102, the HDD 113, etc. are loaded into the RAM 103 and executed by the CPU 101. A display 115 is connected to the VC 104. An input device 110 such as a mouse or a keyboard and an imaging device 111 are connected to the general-purpose I / F 105 via a serial bus 109. A general-purpose drive 114 that reads from and writes to an HDD 113 and various recording media is connected to the SATA I / F 106 via a serial bus 112. The NIC 107 inputs and outputs information to and from external devices. The CPU 101 uses the HDD 113 and various recording media mounted on the general-purpose drive 114 as storage locations for various data. The CPU 101 displays a GUI (Graphical User Interface) provided by a program on a display 115, and receives inputs such as user instructions via an input device 110.

[0015] <Functional configuration of the image processing apparatus according to the first embodiment> Fig. 1(b) is a functional block diagram showing the functional configuration of the image processing device 1 according to the first embodiment. The CPU 101 functions as each functional unit shown in Fig. 1(b) by reading and executing a program stored in the ROM 102 or the HDD 113 using the RAM 103 as a work memory. Note that it is not necessary for the CPU 101 to execute all of the processing of each functional unit shown below, and the image processing device may be configured so that part or all of the processing is performed by one or more processing circuits other than the CPU 101.

[0016] The image processing device 1 includes an image acquisition unit 151 , an optical profile calculation unit 152 , a synthesis unit 153 , an evaluation value calculation unit 154 , and an output unit 155 . The image acquisition unit 151 acquires an image data group consisting of image data of a plurality of captured images of an object to be evaluated while changing the focus position by the imaging device 111. Note that, hereinafter, image data will be simply referred to as "images" unless a distinction is particularly required.

[0017] The optical profile calculation unit 152 calculates an optical profile for each image in the image data group acquired by the image acquisition unit 151 . The synthesis unit 153 integrates (synthesizes) the multiple optical profiles calculated for each image by the optical profile calculation unit 152, thereby calculating one synthetic profile. The evaluation value calculation unit 154 calculates an evaluation value from the composite profile. The output unit 155 outputs the evaluation value calculated by the evaluation value calculation unit 154 .

[0018] <Operation of image processing device and image processing> Figure 2 is a flowchart showing the operation and image processing flow of the image processing device shown in Figures 1(a) and 1(b). In the following explanations of each flowchart, the notation of each process (step) will be omitted by adding the letter "S" to the beginning of each process (step). However, the image processing device does not necessarily have to perform all of the processes described in each of the following flowcharts. Furthermore, the order of each process in each of the following flowcharts may be changed as appropriate.

[0019] In S201, the CPU 101 executes a program stored in the HDD 113 to display a GUI shown in Fig. 4, which will be described later, on the display 115. The GUI shown in Fig. 4 will be described in detail later.

[0020] In S202, the image acquisition unit 151 controls the operation of the imaging device 111 based on an imaging instruction from the user via a GUI in Fig. 4 (described later), and causes the imaging device 111 to capture an image of the object to be evaluated multiple times while shifting the focus position of the imaging device 111. Note that control over the imaging device 111 may be performed by the CPU 101. In this way, the image acquisition unit 151 acquires multiple images captured by the imaging device 111. The imaging conditions and the like when the imaging device 111 captures an image of the object to be evaluated will be described in detail later with reference to Fig. 3.

[0021] In S203, the optical profile calculation unit 152 and the evaluation value calculation unit 154 acquire evaluation conditions set by the user via the GUI in Fig. 4, which will be described later. As will be described in detail later, the evaluation conditions in this embodiment include conditions related to the evaluation target range and conditions related to the evaluation value to be calculated. As will be described in detail later, these conditions related to the evaluation target range and conditions related to the evaluation value are set by the user inputting instructions via the input device 110 to an evaluation range designation area 403 and a calculated evaluation value designation box 404 in Fig. 4, which will be described later.

[0022] In S204, the optical profile calculation unit 152 calculates an optical profile from the captured image using a calculation method described later. Similarly, as will be described in detail later, an optical profile is calculated for each captured image. In S205, the synthesis unit 153 synthesizes the optical profiles calculated for each captured image in S205 to generate a synthetic profile. Details of the synthesis process of the optical profiles will be described later.

[0023] In S206, the evaluation value calculation unit 154 performs an evaluation value acquisition process to acquire an evaluation value based on the composite profile, that is, calculates an evaluation value for orange skin in this embodiment. The calculation process for the orange skin evaluation value using the composite profile will be described in detail later. In S207, the output unit 155 outputs the evaluation value to the display 115.

[0024] <Geometric conditions when imaging an object> The geometric imaging conditions when imaging an object to be evaluated for orange peel texture as a subject will be described below. FIG. 3 is a schematic diagram showing an example of imaging geometric conditions when capturing an image of an object to be evaluated. FIG. 3(a) is a schematic diagram showing an object 301 to be evaluated as viewed from the front under the imaging geometric conditions, and FIG. 3(b) is a schematic diagram showing the object 301 and the imaging device 111 as viewed from above under the imaging geometric conditions. The object 301 is a subject to be evaluated. Furthermore, the object 301 is not necessarily flat. In this embodiment, as shown in FIG. 3(a), for example, the object 301 has a concave curved shape in which the distance from the imaging device 111 increases toward the center. Furthermore, the illumination light source 302 is a linear light that illuminates the object 301. The illumination light source 302 is installed at a certain distance from the object 301 so as not to come into contact with the object 301.

[0025] 3, a portion of light from the illumination light source 302 is projected onto the surface of the object 301, and therefore, the light from the illumination light source 302 is reflected on the surface of the object 301 as an illumination image 303. Furthermore, under these imaging geometric conditions, for example, a user may focus the imaging device 111 on the foreground of the object 301, i.e., on the focal plane SF1 in FIG. 3(b). However, under these imaging geometric conditions, the object 301 has a curved structure rather than a flat surface, and therefore an out-of-focus region occurs in the image captured of the object 301. For example, if the focal plane of the imaging device 111 is aligned with the focal plane FS1 in FIG. 3(b), the edge of the illumination image 303 is in focus, but the central portion of the illumination image 303, which is outside the depth of field of the imaging device 111, is blurred. On the other hand, when the focal plane of the imaging device 111 is aligned with the focal plane FS2 in Figure 3(b), the center of the illumination image 303 is in focus, but the edges of the illumination image 303 that are outside the depth of field will be blurred.

[0026] <gui> FIG. 4 is a diagram showing an example of GUI 4 displayed on display 115. The image display window 401 is a window in which an image captured by the imaging device 111 is displayed. The imaging button 402 is a button that the user presses or otherwise operates to send an imaging instruction to the imaging device 111. The evaluation range designation area 403 is a rectangular area within the image displayed in the image display window 401 that indicates the evaluation range for evaluating orange peel. The evaluation range designation area 403 can be arbitrarily designated by the user. The calculated evaluation value designation box 404 is a checkbox for the user to check one of the boxes to designate the evaluation value to be calculated. In other words, by checking the calculated evaluation value designation box 404, the user sets the evaluation accuracy for the period of the irregularities (orange peel wavelength) on the surface of the object to be evaluated. In the example of FIG. 4, the user can select three evaluation values: evaluation value "1" (0.1 mm to 0.3 mm), evaluation value "2" (0.3 mm to 1.0 mm), and evaluation value "3" (1.0 mm to 3.0 mm). The evaluation value calculation button 405 is a button that the user presses or otherwise operates to instruct execution of evaluation value calculation processing for the evaluation range specified in the calculated evaluation value specification box 404. The evaluation value display text box 406 is a box that displays the evaluation value calculated for the evaluation range specified in the calculated evaluation value specification box 404. The exit button 407 is a button that the user operates to exit the image processing program (hereinafter referred to as the image processing application) according to this embodiment.

[0027] <Image acquisition unit operation> The following describes the operation of the image acquisition unit 151 under the imaging geometric conditions shown in Fig. 3. When the user presses the imaging button 402 on the GUI shown in Fig. 4, the image acquisition unit 151 acquires a captured image from the imaging device 111.

[0028] The image acquisition unit 151 acquires a plurality of images of the object 301 in which the illumination image 303 is reflected, captured by the imaging device 111 while shifting the focus position. It is assumed that when the imaging button 402 is pressed, the focus is on the front surface (focus plane FS1) of the object 301 shown in FIG. 3, for example. In this embodiment, it is assumed that the depth of the object 301 is about 150 mm, for example, and the depth of field of the lens is 80 mm. Therefore, in this case, the imaging device 111 captures the object 301 twice while shifting the focus plane by 80 mm in the depth direction.

[0029] The focus shift amount and the number of times of imaging may be predetermined fixed values ​​as in the first embodiment, or may be automatically set based on imaging geometric conditions, etc. A method for automatically setting the focus shift amount and the number of times of imaging will be described later in the third embodiment. In the first embodiment, the image acquisition unit 151 acquires each captured image obtained by imaging the object 301 while shifting the focus position with the imaging device 111, but this is not limited to this. For example, a plurality of captured images obtained by imaging the object 301 while shifting the focus position by operating the imaging device 111 may be stored in advance in the HDD 113 or the like, and the image acquisition unit 151 may acquire the captured images by reading them from the HDD 113.

[0030] FIG. 5 is a diagram showing state transitions of the image processing device in an example of the GUI of this embodiment. When an image processing application according to this embodiment is executed in response to a user instruction, the image processing device enters state ST501, displays the GUI described above, and then transitions to state ST502. After transitioning to state ST502, the image processing device waits for input from the user. If, for example, the capture button 402 is pressed in state ST502, the image processing device transitions to state ST503. If, in state ST502, the user operates the evaluation range designation area 403 and the calculated evaluation value designation box 404, the image processing device transitions to state ST504. If, in state ST502, the evaluation value calculation button 405 is pressed, the image processing device transitions to state ST505. If, in state ST502, the end button 407 is pressed, the image processing device transitions to state ST506.

[0031] When the image processing device moves to state ST503, it controls the imaging device 111 to capture an image of the object to be evaluated, displays the image obtained by capturing the image in the image display window 401, and then moves to state ST502. When the image processing device transitions to state ST504, if the user specifies the evaluation range and the evaluation value to be calculated as evaluation conditions, the image processing device updates the display of the evaluation range specification area 403 and the calculated evaluation value specification box 404, and then transitions to state ST502. When the image processing apparatus moves to state ST505, it calculates an evaluation value and displays the evaluation value in the evaluation value display text box 406, and then moves to state ST502. When the image processing apparatus moves to state ST506, it performs an operation related to the termination of the image processing application.

[0032] FIG. 6A is a diagram schematically illustrating an evaluation target range 604 for an image 601 acquired in S203. The evaluation target range 604 is set by receiving rectangular information of an evaluation target range designation area 403 designated by a user for an image displayed in the image display window 401. Specifically, the image processing apparatus receives coordinate values ​​of a start point 602 and an end point 603 of the rectangle designated by the user, and sets the evaluation target range 604 with the start point 602 set as the upper left pixel of the rectangle and the end point 603 set as the lower right pixel of the rectangle. The start point 602 and the end point 603 are designated, for example, by inputting coordinate values ​​for the image via the input device 110. In this way, the image processing apparatus of this embodiment designates the rectangular area (evaluation target range designation area 403) designated by the start point 602 and the end point 603 as the evaluation target range 604, and generates range designation data representing the evaluation target range 604.

[0033] <Optical profile calculation process> The optical profile calculation process performed by the optical profile calculation unit 152 in S204 will be described below with reference to the flowchart in Fig. 7. Note that this optical profile calculation process is applied to all captured images. In other words, if there are four images captured while shifting the focus position, an optical profile is calculated for each of the four images, resulting in four optical profiles.

[0034] In S701, the optical profile calculation unit 152 acquires an image from the image acquisition unit 151. Next, in S702, the optical profile calculation unit 152 binarizes each pixel value of the image using a predetermined binarization threshold. In this embodiment, for example, the binarization threshold is the average luminance calculated for each pixel within the evaluation range. The binarization process in S702 is a process in which if a pixel value is equal to or greater than the threshold, the pixel value is set to 1 (black pixel), and if the pixel value is less than the threshold, the pixel value is set to 0 (white pixel). FIG. 6(b) shows an image 605 after the binarization process for the image in FIG. 6(a), and is composed of black pixels 606 with a pixel value of 1 and white pixels 607 with a pixel value of 0.

[0035] Next, in S703, the optical profile calculation unit 152 detects edges of the image 605 after the binarization process. Here, the (x, y) coordinates of the starting point 602 are (x st ,y st ), and the (x, y) coordinates of the end point 603 are (x ed ,y ed Specifically, the optical profile calculation unit 152 calculates x st From x ed For each x value up to y, the y value is st From y ed The optical profile calculation unit 152 searches for pixels up to x, and determines pixels whose pixel values ​​differ from those of adjacent pixels as points that form an edge (hereinafter referred to as edge points). Adjacent pixels are pixels that are adjacent in the y direction. For example, the pixels adjacent to a pixel with coordinates (X, Y) are the pixel with coordinates (X, Y-1) and the pixel with coordinates (X, Y+1). If there are multiple edge points for the same x value, the optical profile calculation unit 152 determines the one with the shortest distance from the straight line connecting the start point and end point as the edge point. This makes it possible to detect edges formed by each edge point.

[0036] Next, in S704, the optical profile calculation unit 152 obtains luminance information on an approximate line obtained by approximating the edge (a set of edge points) detected in S703 with a linear function. In this embodiment, the approximate line is calculated using the well-known least squares method. Next, in S705, the optical profile calculation unit 152 calculates the luminance at the pixel position on the approximate straight line calculated in S704 as an optical profile (luminance profile). 6(c) is a diagram schematically illustrating an approximate line calculated by the optical profile calculation unit 152. The line segments made up of pixels indicated by diagonal lines in FIG.

[0037] <Optical profile synthesis processing> The optical profile synthesis process performed by the synthesis unit 153 in S205 will be described below with reference to the flowchart in FIG. In S801, the synthesis unit 153 synthesizes the optical profile P calculated for each image in S204. m where the optical profile P m "m" in the above expression is the index of the profile calculated by the optical profile calculation unit 152. If the number of optical profiles calculated in S204 is M, then 0≦m≦M−1.

[0038] Next, in S802, the synthesis unit 153 calculates contrast information corresponding to the optical profiles acquired in S801 by applying predetermined dispersion filters as shown in the following formulas (1) and (2), respectively. Note that in these formulas, w is the window width for calculating the dispersion, and in this embodiment, w=2.

[0039]

number

[0040] Next, in S803, the synthesis unit 153 determines whether the processes of S801 and S802 have been performed on all optical profiles. If the processes of S801 and S802 have been performed on all optical profiles, the synthesis unit 153 proceeds to the process of S804, and if not, returns to the process of S801.

[0041] In S804, the synthesis unit 153 synthesizes the above-mentioned plurality of optical profiles P m In this embodiment, a composite profile P is calculated based on the plurality of optical profiles P m The variance V calculated using equation (1) is m The composite profile is calculated by adopting, for each pixel position, the value of the optical profile that maximizes the contrast information calculated in S802. That is, in S804, the composition unit 153 calculates the composite profile by adopting, for each pixel position, the value of the optical profile that maximizes the contrast information calculated in S802. Specifically, the processing in S804 is given by the following equations (3) and (4).

[0042]

number

[0043] The operation of the optical profile synthesis process will be described in detail below with reference to the schematic diagram shown in FIG. Figure 9(a) and Figure 9(b) show the optical profile P obtained by S801. m 9(a) and 9(b) are schematic diagrams showing an example of an optical profile calculated from an image of a portion of the illumination image 303 when the object 301 is imaged at the focal plane FS1 in FIG. 3(b). When the object 301 is imaged at the focal plane FS1 in FIG. 3(b), the edges of the illumination image 303 are in focus. In this case, the optical profile is acquired as fine luminance changes at the edges, whereas the central portion, where the image is blurred, is acquired as smooth luminance changes. On the other hand, P1(x) in FIG. 9(b) is an optical profile calculated from an image of a portion of the illumination image 303 when the object 301 is imaged at the focal plane FS2 in FIG. 3(b). When the object 301 is imaged at the focal plane FS2 in FIG. 3(b), the central portion of the illumination image 303 is in focus. In this case, the optical profile is acquired as fine luminance changes at the central portion, whereas the central portion is acquired as smooth luminance changes.

[0044] FIG. 9(c) is a schematic diagram showing the result V0(x) of applying a dispersion filter to the optical profile P0(x) in FIG. 9(a) in S802, i.e., the change in contrast. Similarly, FIG. 9(d) is a schematic diagram showing the result V1(x) of applying a dispersion filter to the optical profile P1(x) in FIG. 9(b) in S802, i.e., the change in contrast. Because a large dispersion value is calculated in the in-focus portion of the optical profile, the result V0(x) of applying the dispersion filter to the optical profile P0(x) in FIG. 9(a) takes larger values ​​at the edges and smaller values ​​at the center. On the other hand, the result V1(x) of applying the dispersion filter to the optical profile P1(x) in FIG. 9(b) shows the opposite tendency, taking smaller values ​​at the edges and larger values ​​at the center.

[0045] 9(e) is a schematic diagram showing the composite profile P(x) calculated in S804. In S804, the composite profile P(x) is generated by using the optical profile P0(x) or P1(x), whichever has the higher variance value. Therefore, the composite profile P(x) is calculated as an optical profile equivalent to when the entire region is in focus. FIG. 6(d) is a diagram that schematically shows the composite profile after the composition process by the composition unit 153, and portions 609 and 610 made up of pixels indicated by diagonal lines in FIG. 6(d) correspond to the composite profile.

[0046] In this embodiment, the composite profile is generated based on the variance value of the optical profiles, but the present invention is not limited to this example. For example, the optical profiles may be synthesized based on the contrast ratio or the power spectrum of the Fourier transform instead of the variance value. In this embodiment, the optical profile is synthesized for each pixel, but the present invention is not limited to this. For example, the dispersion filter may have a step width, and a composite profile may be generated by determining which optical profile to adopt for each region (for example, every 10 pixels).

[0047] <Evaluation value calculation process> The evaluation value calculation process performed by the evaluation value calculation unit 154 in S206 will be described below with reference to the flowchart in FIG. In S1001, the evaluation value calculation unit 154 acquires the composite profile calculated in S205.

[0048] Next, in S1002, the evaluation value calculation unit 154 performs frequency conversion on the composite profile. Furthermore, in S1003, the evaluation value calculation unit 154 performs frequency conversion corresponding to the actual size of the object using the converted sampling interval and the converted actual size of the object per unit interval using the following equation (5). Here, the frequency before conversion is f pix , the frequency corresponding to the actual size of the object after conversion is f mm Let's say.

[0049]

number

[0050] Next, in S1004, the evaluation value calculation unit 154 performs integration for a predetermined frequency interval and calculates an evaluation value. Here, if the evaluation value is Peva, the frequency amplitude is amp, the lower limit frequency of the integration interval is f1, and the upper limit frequency is f2, then this can be expressed as in equation (6).

[0051]

number

[0052] As an example of a predetermined frequency interval, for example, if the evaluation value is "1", frequencies corresponding to the actual size of the object can be used, such as f1 = 0.1 cycle / mm and f2 = 0.3 cycle / mm. It goes without saying that an interval can be set by specifying any frequency, and multiple intervals can also be set.

[0053] Next, in S1005, the evaluation value calculation unit 154 determines whether or not integration has been performed for all frequency intervals, and if not, returns the process to S1002, and if not, performs processing related to termination.

[0054] As described above, the image processing device of the first embodiment calculates optical profiles from multiple images of an object surface captured while shifting the focus position. The image processing device then generates a composite profile by combining the multiple optical profiles calculated for each image, using only the optical profile portions of the in-focus regions corresponding to the respective focus positions, thereby generating a composite profile that is in focus across the entire image. For example, the image processing device calculates contrast information for each of the multiple optical profiles and generates a composite profile by combining the multiple optical profiles, using the regions corresponding to the maximum calculated contrast information. The image processing device of this embodiment then calculates an orange skin evaluation value using the combined profile, thereby enabling calculation of an orange skin evaluation value even when the shape of the object has a depth that exceeds the depth of field of the image capture device 111. In other words, the image processing device of the first embodiment can calculate an accurate orange skin evaluation value even for an object surface having a curved structure in the depth direction.

[0055] <Second embodiment> In the first embodiment, an object is imaged while shifting the focus position to obtain multiple images, optical profiles are calculated from these images, and these multiple optical profiles are then combined to calculate a highly accurate orange peel evaluation value even for the surface of an object with depth. However, the process of imaging multiple images of an object surface while shifting the focus position is somewhat time-consuming and laborious.

[0056] Therefore, the image processing device of the second embodiment determines whether optical profile synthesis is necessary depending on the wavelength of the orange peel evaluation value to be evaluated. That is, the image processing device of the second embodiment determines whether optical profile synthesis is necessary depending on the evaluation accuracy for the period of the unevenness on the surface of the object to be evaluated. In other words, in this embodiment, whether optical profile synthesis is necessary can be said to refer to whether or not to capture multiple images of the object while shifting the focus position and calculate an optical profile from those images. In this embodiment, only if it is determined that optical profile synthesis is necessary, multiple images of the subject are captured while shifting the focus position. On the other hand, if it is determined that optical profile synthesis is not necessary, the orange peel evaluation value is calculated based on the optical profile acquired from a single captured image. Below, with regard to the image processing device of the second embodiment, only the differences from the first embodiment will be described.

[0057] <Functional configuration of image processing device according to second embodiment> FIG. 11 is a block diagram showing the functional configuration of an image processing device 1 according to the second embodiment. The image processing device 1 of the second embodiment has an image acquisition unit 1101, an optical profile calculation unit 1102, a determination unit 1103, a synthesis unit 1104, an evaluation value calculation unit 1105, and an output unit 1106. The image acquisition unit 1101 acquires an image of an object to be evaluated. The optical profile calculation unit 1102 calculates an optical profile from the image acquired by the image acquisition unit 1101. The determination unit 1103 determines whether or not optical profile synthesis is necessary for the optical profiles calculated by the optical profile calculation unit 1102.

[0058] If the determination unit 1103 determines that optical profile merging is necessary, the image acquisition unit 1101 acquires multiple images captured while shifting the focus position, and the optical profile calculation unit 1102 calculates an optical profile from these multiple images. The merging unit 1104 then acquires multiple optical profiles from the optical profile calculation unit 1102 and integrates them to calculate a single merged profile. On the other hand, if the determination unit 1103 determines that optical profile merging is not necessary, the image acquisition unit 1101 acquires a single image captured without shifting the focus position, and the optical profile calculation unit 1102 calculates an optical profile from this image.

[0059] If a composite profile has been calculated, the evaluation value calculation unit 1105 calculates an evaluation value based on the composite profile, and if a composite profile has not been calculated, the evaluation value calculation unit 1105 calculates an evaluation value based on an optical profile calculated from one image. The output unit 1106 outputs the evaluation value calculated by the evaluation value calculation unit 1105.

[0060] <Image processing in the image processing device of the second embodiment> FIG. 12 is a flowchart showing the flow of image processing executed by the image processing apparatus of the second embodiment.

[0061] In S1201, the CPU 101 of the image processing apparatus executes the program (image processing application) according to the second embodiment stored in the HDD 113, thereby displaying a GUI such as that shown in Fig. 4 on the display 115. Note that the processing in S1201 is similar to the processing in S201 in the first embodiment.

[0062] Next, in S1202, the image acquisition unit 1101 acquires an image of the object to be evaluated, captured by the imaging device 111 in accordance with the user's settings via the GUI. In the second embodiment, in S1202, the image acquisition unit 1101 acquires only one captured image captured without shifting the focus position, unlike S202 in the first embodiment.

[0063] Next, in S1203, the optical profile calculation unit 1102 and the evaluation value calculation unit 1105 acquire the evaluation conditions set by the user via the GUI in Fig. 4. The processing in S1203 is similar to the processing in S203 in the first embodiment, and the evaluation target range and conditions related to the evaluation value specified by the user are acquired via the above-mentioned GUI.

[0064] Next, in S1204, the optical profile calculation unit 1102 calculates an optical profile. The processing in S1204 is similar to the processing in S204 in the first embodiment, but the optical profile in this case is calculated from the single image acquired in S1202.

[0065] Next, in S1205, the determination unit 1103 determines whether or not combining of optical profiles is necessary. Details of the determination process by the determination unit 1103 will be described later. If it is determined in S1205 that combining is necessary, the process of the image processing apparatus proceeds to S1206, and if not, the process proceeds to S1209.

[0066] When the process proceeds to S1206, the image acquisition unit 1101 acquires a plurality of images captured while shifting the focus position, similarly to the first embodiment described above. Furthermore, in S1207, the optical profile calculation unit 1102 calculates optical profiles for the multiple images acquired in S1206 in the same manner as in the first embodiment.

[0067] Next, in S1208, the combining unit 1104 combines the optical profile acquired in S1204 and the multiple optical profiles acquired in step S1207. The optical profile combining process here is similar to the process of S205 in the first embodiment.

[0068] Next, when the process proceeds to S1209, if it is determined in S1205 that compositing is not necessary, the evaluation value calculation unit 1105 calculates an orange skin evaluation value using the optical profile calculated in S1204. On the other hand, if it is determined in S1205 that compositing is necessary, the evaluation value calculation unit 1105 calculates an orange skin evaluation value using the composite profile calculated in S1208. The evaluation value calculation process in S1209 is the same as the process in S206 in the first embodiment.

[0069] Thereafter, in S1210, the output unit 1107 outputs the evaluation value to the display 115. The evaluation value output process in S1210 is the same as the process in S207 in the first embodiment.

[0070] <Optical profile synthesis judgment processing> The optical profile synthesis determination process performed by the determination unit 1103 in step S1205 will be described below with reference to the flowchart in FIG. In S1301, the determination unit 1103 applies a dispersion filter to the optical profile P0 calculated in S1204 and acquires the dispersion filter application result V0. The dispersion filter application process in S1301 is the same as the process in S802 in the first embodiment.

[0071] Next, in S1302, the determination unit 1103 calculates the maximum difference in variance value, that is, the maximum difference in contrast information, for the dispersion filter application result V0 obtained in S1301. The process of S1302 is specifically given by the following equation (7).

[0072]

number

[0073] FIG. 14 is a diagram showing a schematic diagram of the processing operation in S1302. As described in the first embodiment, in-focus areas in the optical profile P0 exhibit high dispersion values, while out-of-focus areas exhibit low dispersion values. In the case of the dispersion filter application result V0(x) for the optical profile illustrated in FIG. 14, the maximum value is 0.5 and the minimum value is 0.2. Therefore, according to equation (7), the maximum difference in dispersion values ​​is calculated as maxDiff=0.5-0.2=0.3. The value calculated by equation (7) indicates how much the dispersion values ​​vary across the entire optical profile, and is a value that indicates the degree of focus blurring that occurs within the optical profile.

[0074] Next, in S1303, the determination unit 1103 acquires information on a predetermined threshold value for determination from the HDD 113 in order to determine whether or not to combine optical profiles. FIG. 15 is a diagram showing an example of the judgment threshold stored in the HDD 113. In this embodiment, as shown in FIG. 15, different judgment thresholds are set for different orange peel evaluation values ​​to be calculated. For example, when evaluating orange peel at a short wavelength (high frequency), such as an evaluation value of "1" (wavelength 0.1 to 0.3 mm), the evaluation value is significantly affected by out-of-focus blur. Therefore, the judgment threshold for the evaluation value of "1" is set to a low value so that an optical profile is synthesized even if even a small amount of out-of-focus blur is observed. On the other hand, when evaluating orange peel at a long wavelength (low frequency), such as an evaluation value of "3" (wavelength 1.0 to 3.0 mm), out-of-focus blur does not significantly affect the evaluation value. Therefore, the judgment threshold for the evaluation value of "3" is set to a high value. In other words, in this embodiment, the threshold used to determine whether or not optical profile synthesis is necessary can be set according to the required evaluation accuracy for the period of the irregularities (orange peel wavelength) on the surface of the object to be evaluated.

[0075] 12, if multiple evaluation values ​​are specified as calculation targets, the judgment threshold value acquired in S1303 is the smallest judgment threshold value among the evaluation values ​​corresponding to the calculation targets. For example, in the GUI example in FIG. 4, if evaluation values ​​"1" and "3" are checked as calculation targets, the judgment threshold value acquired in S1303 is 0.1.

[0076] Next, in S1304, the determination unit 1103 compares the maxDiff calculated as described above with the determination threshold value. If maxDiff is equal to or greater than the threshold value, the determination unit 1103 determines that optical profiles need to be combined, and if not, determines that optical profile combination is not necessary.

[0077] As described above, in the second embodiment, the necessity of optical profile synthesis is determined based on the variance value calculated from the optical profile acquired from a single captured image and the calculated orange peel evaluation value. Then, only when it is determined that profile synthesis is necessary, multiple images of the object to be evaluated are captured while shifting the focus position. On the other hand, when it is determined that profile synthesis is not necessary, the orange peel evaluation value is calculated based on the optical profile obtained from a single image. According to the second embodiment, it is possible to reduce the frequency with which images are captured while shifting the focus position, thereby enabling highly accurate orange peel evaluation of the surface of an object with depth with little burden on the user.

[0078] <Third embodiment> In the second embodiment, the necessity of optical profile synthesis was determined after the object to be evaluated was photographed once. Here, for example, if three-dimensional information of the object to be evaluated, such as three-dimensional information based on design information such as CAD data, is available, the necessity of optical profile synthesis can be determined before the object to be evaluated is photographed. In the third embodiment, depth information of the object to be evaluated is calculated from three-dimensional information such as CAD data of the object to be evaluated, and the necessity of optical profile synthesis is determined based on the depth information, thereby evaluating orange peel. Note that in the third embodiment, as in the above, the necessity of optical profile synthesis can be rephrased as whether or not to photograph the object while shifting the focus position to obtain multiple images and calculate an optical profile from those images. The image processing apparatus of the third embodiment will be described below only in terms of differences from the second embodiment described above.

[0079] <Functional configuration of image processing device according to third embodiment> FIG. 16 is a block diagram showing the functional configuration of an image processing device 1 according to the third embodiment. The image processing apparatus 1 of the third embodiment includes an image acquisition unit 1601 , an optical profile calculation unit 1602 , a synthesis unit 1603 , an evaluation value calculation unit 1604 , an output unit 1605 , and a number acquisition unit 1606 .

[0080] The number acquisition unit 1606 calculates the number of images required to synthesize an optical profile using depth information obtained from three-dimensional information such as CAD data. The image acquisition unit 1601 acquires images obtained by capturing images of the object to be evaluated while changing the focus position, based on the number of images calculated by the number acquisition unit 1606. That is, when the number of images calculated by the number acquisition unit 1606 is one, the image acquisition unit 1601 acquires one image of the object to be evaluated without changing the focus position. On the other hand, when the number of images calculated by the number acquisition unit 1606 is multiple, the image acquisition unit 1601 acquires multiple images of the object to be evaluated while changing the focus position.

[0081] The optical profile calculation unit 1602 calculates an optical profile from the image acquired by the image acquisition unit 1601. That is, if only one image is acquired by the image acquisition unit 1601, the optical profile calculation unit 1602 calculates an optical profile from that one image, and if multiple images are acquired by the image acquisition unit 1601, the optical profile calculation unit 1602 calculates an optical profile for each of those images.

[0082] When the optical profile calculation unit 1602 calculates multiple optical profiles, the synthesis unit 1603 synthesizes the multiple optical profiles to calculate a synthetic profile. Then, when the synthesis unit 1603 calculates a synthetic profile, the evaluation value calculation unit 1604 calculates an orange skin evaluation value using the synthetic profile. On the other hand, when the optical profile calculation unit 162 calculates an optical profile from only one image, the evaluation value calculation unit 1604 calculates an orange skin evaluation value using the optical profile. The output unit 1605 outputs the evaluation value calculated by the evaluation value calculation unit 1604.

[0083] <GUI of the third embodiment> The GUI in the third embodiment is the same as that in the first embodiment except for the image display window 401 in Fig. 4 described above. In the first embodiment, a captured image was displayed in the image display window 401, but in the third embodiment, depth information of an object to be evaluated and to be imaged is also displayed as an image. Therefore, the user specifies the evaluation range by operating the evaluation range specification area 403 while viewing the depth image before capturing the image.

[0084] <Image Processing According to the Third Embodiment> FIG. 17 is a flowchart showing the flow of processing executed by the image processing device in the third embodiment. In S1701, the CPU 101 of the image processing device executes a program (image processing application) according to the third embodiment stored in the HDD 113, thereby displaying a GUI such as that shown in Fig. 4 on the display 115. In the case of the third embodiment, depth information of the object to be evaluated is calculated from CAD data, and the depth information of the object is also displayed in the image display window 401.

[0085] In S1702, the optical profile calculation unit 1102 and the evaluation value calculation unit 1105 acquire the evaluation conditions set by the user via the GUI in Fig. 4. The processing in S1702 is similar to the processing in S203 in the first embodiment, and the evaluation target range and conditions related to the evaluation value specified by the user are acquired via the above-mentioned GUI.

[0086] In S1703, the number acquisition unit 1606 calculates the number of optical profiles required to calculate the orange peel evaluation value (hereinafter referred to as the required number of images n) based on the depth information. Details of the required number of images acquisition process in S1703 will be described later.

[0087] Next, in S1704, the image acquisition unit 1601 acquires images of the object to be evaluated. Here, if the required number of images n calculated by the number acquisition unit 1606 is n=1, the imaging device 111 captures images of the object without shifting the focus position. Therefore, in this case, the image acquisition unit 1601 acquires one captured image. On the other hand, if the required number of images n is not 1 (n is 2 or more), the imaging device 111 captures images of the object while shifting the focus position. Therefore, in this case, the image acquisition unit 1601 acquires multiple images (n) captured by shifting the focus position. Note that the method of capturing an object is the same as in S1201 of the second embodiment when acquiring one image, and is the same as in S1206 when acquiring n images.

[0088] Then, in S1705, the optical profile calculation unit 1602 calculates an optical profile from the image acquired in S1705. In the third embodiment, the optical profile calculation unit 1602 performs alignment processing using the captured image and depth information, and calculates an optical profile using an image area of ​​an evaluation range designation area (evaluation target range) designated by the user on the GUI based on the depth information in S1702. Note that the optical profile calculation processing is the same as S1205 in the second embodiment.

[0089] Next, in S1706, the optical profile calculation unit 1602 determines whether the required number of images n calculated in S1703 is 1. If it is determined that n=1, the processing in the image processing device proceeds to S1708. On the other hand, if it is determined that n=1 is not true (n is 2 or greater), the processing in the image processing device proceeds to S1707.

[0090] When the process proceeds to S1707, the combining unit 1603 combines the multiple optical profiles acquired in S1705. The optical profile combining process here is the same as the process of S205 in the first embodiment described above. After S1705, the process of the image processing apparatus proceeds to S1708.

[0091] In S1708, the evaluation value calculation unit 1604 calculates an orange skin evaluation value. In the third embodiment, if the required number of images n is 1, the orange skin evaluation value is calculated based on an optical profile calculated from one captured image. On the other hand, if the required number of images n is 2 or more, the orange skin evaluation value is calculated based on a composite profile calculated from n captured images and generated in the composition process of S1707.

[0092] Thereafter, in S1709, the output unit 1605 outputs the orange peel evaluation value calculated in S1708 to the display 115. The evaluation value output process in S1709 is the same as S207 in the first embodiment described above.

[0093] <Operation of the image count acquisition unit> The operation of the number obtaining unit 1606 will be described in detail below with reference to the flowchart of FIG. In S1801, the number acquisition unit 1606 acquires the imaging conditions of the imaging device 111 based on the evaluation conditions specified in S1702, and acquires the depth of field dof for the imaging conditions. The depth of field dof is determined by the focal length and F-number of the imaging device 111, and the distance to the object to be evaluated.

[0094] In the third embodiment, information indicating the relationship between imaging conditions and depth of field is stored in advance as a database in the HDD 113. The number of images acquiring unit 1606 acquires information on the depth of field (dof) by searching the database based on the imaging conditions at the time of image capture. FIG. 19 is a diagram showing an example of the relationship between imaging conditions and depth of field, and it is assumed that table information indicating the correspondence relationship shown in FIG. 19 is stored in the database. For example, if imaging is performed under imaging conditions where the focal length is 50 mm, the F-number is 10, and the distance from the imaging device 111 to the object to be evaluated is 1 m, the depth of field (dof) is 80 mm.

[0095] Next, in S1802, the number acquisition unit 1606 calculates the maximum depth difference within the evaluation target range from the depth information. Specifically, the number acquisition unit 1606 calculates the maximum depth difference maxDepth by performing the calculation shown in the following equation (8). Note that in equation (8), D is the depth information of the object to be evaluated calculated from CAD data or the like, and R is the evaluation target range specified in S1702.

[0096]

number

[0097] Next, in S1803, the number acquisition unit 1606 calculates the number of optical profiles (required number of images n) required to calculate the orange peel evaluation value. Specifically, the number acquisition unit 1606 calculates the required number of images n by performing a comparison between the depth of field dof and the maximum depth difference maxDepth as shown in the following equation (9).

[0098]

number

[0099] As described above, the image processing device of the third embodiment calculates the number of optical profiles (required number of images n) required to calculate the orange peel evaluation value based on depth information acquired from three-dimensional information such as CAD data. In the third embodiment, before actually capturing an image of the object to be evaluated, it is possible to determine how many optical profiles are required, that is, how many images with shifted focus positions are required. As a result, according to the third embodiment, a highly accurate orange peel evaluation value can be calculated with the minimum number of images required, thereby reducing the burden on the user.

[0100] <Fourth embodiment> In the second and third embodiments described above, when it is determined that optical profile synthesis is necessary, no feedback is given to the user that optical profile synthesis is necessary, that is, that multiple images need to be captured with different focus positions. However, there are cases where it is difficult to calculate an orange skin evaluation value even when multiple images are captured. For example, this occurs when capturing images with a handheld camera (image capture device 111). This is because, in handheld image capture, the geometric conditions change slightly each time an image is captured due to the effects of camera shake and the like.

[0101] Therefore, in the fourth embodiment, when it is determined that optical profile synthesis is necessary, that is, when multiple images need to be captured, the user is notified of this fact and is then given the option of performing an orange peel evaluation on the evaluation target range on the object to be evaluated while capturing multiple images, or re-setting an evaluation target range that does not require capturing multiple images and performing evaluation using a single image.

[0102] The image processing device of the fourth embodiment will be described below. The block diagram of the functional configuration of the image processing device 1 of the fourth embodiment is the same as that of the second embodiment shown in FIG. 11, and only the differences from the second embodiment will be described below. <Image Processing According to the Fourth Embodiment>

[0103] FIG. 20 is a flowchart showing the flow of image processing executed by the image processing apparatus of the fourth embodiment. In S2001, the CPU 101 of the image processing apparatus executes the program (image processing application) according to the fourth embodiment stored in the HDD 113, thereby displaying a GUI such as that shown in Fig. 4 on the display 115. Note that the processing in S2001 is similar to the processing in S1201 in the second embodiment.

[0104] Next, in S1702, the image acquisition unit 1101 acquires an image of the object to be evaluated, captured by the imaging device 111 in accordance with the user's settings via the GUI described above. In the fourth embodiment, in S2002, the image acquisition unit 1101 acquires only one captured image captured without shifting the focus position, similar to S1202 in the second embodiment.

[0105] Next, in S2003, the optical profile calculation unit 1102 and the evaluation value calculation unit 1105 acquire the evaluation conditions set by the user via the GUI in Fig. 4. The processing in S2003 is similar to the processing in S1203 in the second embodiment, and the evaluation target range and conditions related to the evaluation value specified by the user are acquired via the above-mentioned GUI.

[0106] Next, in S2004, the optical profile calculation unit 1102 calculates an optical profile. The process in S2004 is the same as the process in S1204 in the second embodiment. Next, in S2005, the determination unit 1103 determines whether or not it is necessary to combine multiple optical profiles. If it is determined in S2005 that it is necessary to combine, the process of the image processing apparatus proceeds to S2006, and if not, the process proceeds to S2010.

[0107] When the process proceeds to S2006, the image acquisition unit 1101 displays a GUI on the display 115 to prompt the user to confirm whether or not to continue the process. Specifically, the image acquisition unit 1101 displays a GUI such as that shown in FIG. 21 on the display 115. Note that the process of displaying the GUI on the display 115 may be performed separately by the CPU 101. The GUI shown in FIG. 21 includes a text message and buttons 2101 and 2102 to prompt the user to confirm whether or not to continue the process. Button 2101 is pressed when the user determines that the process should be continued, and button 2102 is pressed when the user determines that the process should not be continued and instead determines that the evaluation target range should be reset. When the user presses button 2101, the process of the image processing device proceeds to S2007. On the other hand, when the user presses button 2102, the process of the image processing device returns to S2003.

[0108] When the process proceeds to S2007, the image acquisition unit 1101 acquires a plurality of images captured while shifting the focus position, similarly to the first embodiment described above. Furthermore, in S2008, the optical profile calculation unit 1102 calculates optical profiles for the multiple images acquired in S2006 in the same manner as in the first embodiment.

[0109] Next, in S2009, the combining unit 1104 combines the multiple optical profiles acquired in S2008. The optical profile combining process here is the same as the process in S205 in the first embodiment. Next, in S2009, the synthesis unit 1104 synthesizes the optical profile acquired in S2004 and the multiple optical profiles acquired in step S2008. The synthesis process of the optical profiles here is the same as the process of S1208 in the second embodiment.

[0110] Then, in S2010, if it is determined in S2005 that combining is not necessary, the evaluation value calculation unit 1105 calculates an orange skin evaluation value using the optical profile calculated in S2004. On the other hand, if it is determined in S2005 that combining is necessary, the evaluation value calculation unit 1105 calculates an orange skin evaluation value using the combined profile calculated in S2009. The evaluation value calculation process in S2010 is the same as the process in S1209 in the second embodiment. Thereafter, in S2011, the output unit 1107 outputs the evaluation value to the display 115. This evaluation value output process in S2011 is similar to the process in S1210 in the second embodiment.

[0111] As described above, the image processing device of the fourth embodiment notifies the user when it is necessary to combine multiple optical profiles, allowing the user to determine whether or not to continue the process. As a result, according to the fourth embodiment, when it is difficult to capture multiple images, the evaluation value for orange peel can be flexibly calculated according to the user's imaging situation, for example, by capturing an image of a flat part of the object to be evaluated and calculating the evaluation value again.

[0112] The disclosure of each embodiment includes the following configurations, methods, and programs. (Configuration 1) an image acquisition means for acquiring images of an object to be evaluated by changing the focus position of an imaging device; a profile acquisition means for acquiring an optical profile representing attributes of an image reflected on a surface of the object from images of the object captured while changing the focus position; a synthesis unit that synthesizes the optical profiles acquired for each image of the object captured while changing the focus position to generate a synthetic profile; an evaluation value acquisition means for acquiring an evaluation value representing a surface state of the object based on the composite profile; 1. An image processing device comprising: (Configuration 2) The image processing device described in configuration 1 is characterized in that the synthesis means generates the synthetic profile by synthesizing a plurality of optical profiles obtained for each image of the object captured by changing the focus position so as to use each in-focus area corresponding to the focus position of the imaging device. (Configuration 3) The synthesis means Calculating contrast information for each of a plurality of optical profiles acquired for each image obtained by capturing an image of the object while changing the focus position; 3. The image processing device according to configuration 1 or 2, wherein the composite profile is generated by combining the plurality of optical profiles so as to use regions corresponding to portions where the calculated contrast information is maximum. (Configuration 4) 4. The image processing device according to configuration 3, wherein the contrast information is a variance value obtained by applying a predetermined variance filter to the optical profile. (Configuration 5) 5. The image processing device according to any one of configurations 1 to 4, wherein the optical profile is information that represents a one-dimensional profile of luminance fluctuations of a projected pattern image that is projected onto the surface of the object. (Configuration 6) 6. The image processing device according to configuration 5, wherein the optical profile is luminance information on an approximate straight line of an edge of the projection pattern image projected onto the surface of the object. (Configuration 7) 7. The image processing device according to any one of configurations 1 to 6, wherein the evaluation value is a value for evaluating the state of unevenness occurring on the surface of the object. (Configuration 8) the image acquisition means acquires an image of the illuminated object; 8. The image processing device according to any one of configurations 1 to 7, wherein the profile acquisition means acquires the optical profile that represents attributes of an image of the illumination projected onto the surface of the object. (Configuration 9) The method further includes a determination unit for determining whether or not the optical profiles need to be combined, When the determination means determines that the optical profile needs to be synthesized, the image acquisition means acquires a plurality of images of the object by changing the focus position of an imaging device; 9. The image processing device according to any one of configurations 1 to 8, wherein the combining means performs the combining using the plurality of optical profiles acquired for each of the plurality of images. (Configuration 10) The determination means Calculating a maximum difference in contrast information from an optical profile acquired from an image of the object captured without changing the focus position of an imaging device; 10. The image processing device according to configuration 9, wherein the necessity of combining the optical profiles is determined based on a comparison between the maximum difference of the contrast information and a predetermined threshold value. (Configuration 11) 11. The image processing device according to configuration 10, wherein the determining means sets the threshold value in accordance with the evaluation accuracy required for the surface state of the object. (Configuration 12) a determination means for determining whether or not the optical profiles need to be combined; The image processing device according to any one of configurations 1 to 8, further comprising: a notification unit that, when the determination unit determines that the optical profiles need to be combined, notifies a user that it is necessary to take multiple images with different focus positions of the imaging device. (Configuration 13) 13. The image processing device according to any one of configurations 1 to 12, further comprising: a number acquisition means for acquiring depth information of the object to be evaluated from three-dimensional information of the object, and calculating the number of images to be captured of the object by changing the focus position based on the depth information. (Configuration 14) The image processing device described in configuration 13, characterized in that the number acquisition means acquires the depth of field of the imaging device, acquires the maximum depth difference of the object relative to the imaging device based on the depth information, and acquires the number of images based on a comparison between the depth of field and the maximum depth difference. (Method 1) An image processing method executed by an image processing device, an image acquisition step of acquiring images of the object to be evaluated by changing the focus position of an imaging device; a profile acquisition step of acquiring an optical profile representing attributes of an image reflected on a surface of the object from images of the object captured while changing the focus position; a synthesis step of generating a synthetic profile by synthesizing the optical profiles acquired for each image obtained by capturing the object while changing the focus position; an evaluation value acquisition step of acquiring an evaluation value representing a surface state of the object based on the composite profile; An image processing method comprising: (Program 1) A program for causing a computer to function as the image processing device according to any one of the first to fourteenth configurations.

[0113] In each of the above-described embodiments, an example has been given in which the orange peel on the surface of an object to be evaluated is evaluated, but the evaluation is not limited to the orange peel, and each embodiment can be applied to other evaluations. The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. The above-described embodiments are merely examples of specific implementations of the present invention, and the technical scope of the present invention should not be construed as being limited by these embodiments. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features. [Explanation of symbols]

[0114] 1: image processing device, 151: image acquisition unit, 152: optical profile calculation unit, 153: synthesis unit, 154: evaluation value calculation unit, 155: output unit< / gui>

Claims

1. An image acquisition means for acquiring a plurality of images obtained by capturing an object to be evaluated at a plurality of focus positions; a profile acquisition means for acquiring a plurality of optical profiles representing attributes of images reflected on the surface of the object based on the plurality of images; a synthesis means for synthesizing the plurality of optical profiles to generate a synthetic profile; an evaluation value acquisition means for acquiring an evaluation value representing a surface state of the object based on the composite profile; 1. An image processing device comprising:

2. The image processing device according to claim 1 , wherein the combining unit combines the plurality of optical profiles so as to use a focus region corresponding to a focus position of an imaging device, and generates the combined profile.

3. The synthesis means Calculating contrast information based on the plurality of optical profiles; 2. The image processing apparatus according to claim 1, wherein the composite profile is generated by combining the plurality of optical profiles so as to use a region corresponding to a portion where the calculated contrast information is maximum among the plurality of optical profiles.

4. 4. The image processing apparatus according to claim 3, wherein the contrast information is a variance value obtained by applying a predetermined variance filter to the optical profile.

5. 2. The image processing apparatus according to claim 1, wherein the optical profile is information representing a one-dimensional profile of luminance fluctuations of a projected pattern image projected onto the surface of the object.

6. 6. The image processing apparatus according to claim 5, wherein the optical profile is luminance information on an approximate straight line of an edge of the projection pattern image projected onto the surface of the object.

7. 2. The image processing apparatus according to claim 1, wherein the evaluation value is a value for evaluating the state of irregularities occurring on the surface of the object.

8. the image acquisition means acquires an image obtained by capturing an image of the illuminated object, 2. The image processing apparatus according to claim 1, wherein the profile acquisition means acquires the optical profile that represents attributes of an image of the illumination projected on the surface of the object.

9. The method further includes a determination unit for determining whether or not the optical profiles need to be combined, When the determination means determines that the optical profile needs to be synthesized, the image acquisition means acquires the plurality of images; 9. The image processing apparatus according to claim 1, wherein the combining means performs the combining.

10. The determination means calculating a maximum difference in contrast information based on an optical profile corresponding to an image obtained by capturing an image of the object; The image processing apparatus according to claim 9 , wherein the necessity of combining the optical profiles is determined based on a comparison between the maximum difference of the contrast information and a predetermined threshold value.

11. 11. The image processing apparatus according to claim 10, wherein the determining means sets the threshold value in accordance with a required evaluation accuracy for the surface condition of the object.

12. a determination means for determining whether or not the optical profiles need to be combined; 9. The image processing device according to claim 1, further comprising: a notification unit that, when the determination unit determines that combining the optical profiles is necessary, notifies a user that multiple images need to be taken with different focus positions of the imaging device.

13. 9. The image processing device according to claim 1, further comprising: a number acquisition means for acquiring depth information of the object to be evaluated based on three-dimensional information of the object; and calculating, based on the depth information, the number of images to be taken of the object by changing the focus position.

14. The image processing device described in claim 13, characterized in that the number acquisition means acquires the depth of field of the imaging device, acquires the maximum depth difference of the object relative to the imaging device based on the depth information, and acquires the number of images based on a comparison between the depth of field and the maximum depth difference.

15. An image processing method executed by an image processing device, an image acquisition step of acquiring a plurality of images obtained by capturing an image of an object to be evaluated at a plurality of focus positions; a profile acquisition step of acquiring a plurality of optical profiles representing attributes of images reflected on the surface of the object based on the plurality of images; a combining step of combining the plurality of optical profiles to generate a combined profile; an evaluation value acquisition step of acquiring an evaluation value representing a surface state of the object based on the composite profile; An image processing method comprising:

16. A program for causing a computer to function as the image processing device according to claim 1.

Images