Information processing device, information processing method, and program

Abstract

To provide an information processing device, an information processing method, and a program that enable generation of a color difference map that can be applied to various uses and needs.SOLUTION: An information processing device sets a first line segment in an image acquired by imaging and sets a second line segment in the image. Then, the information processing device acquires a color difference between a point on the first line segment and a point on the second line segment, and generates a color difference map on the basis of the acquired color difference.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to a technique for generating a color difference map based on an image.

Background Art

[0002] In recent years, there has been an increasing number of cases where a camera is used instead of a color measuring device, and color inspection is performed by referring to color distribution information obtained from pixel values of a captured image. As a technique for handling color data extracted from such a captured image, there is a technique described in Patent Document 1. Patent Document 1 discloses a technique for displaying, as a map, the color difference between a specified reference point and each position in an image. According to this technique, a two-dimensional distribution of the color difference from the reference point can be displayed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, since the color difference map generated by the above prior art is generated based on the color difference from a single reference point, its applications are somewhat limited, and it is difficult to say that it is a color difference map applicable to various applications and needs.

[0005] Therefore, an object of the present invention is to enable the generation of a color difference map applicable to various applications and needs.

Means for Solving the Problems

[0006] The information processing apparatus of the present invention includes a first setting means for setting a first line segment in an image obtained by imaging, a second setting means for setting a second line segment in the image, and the first line segment The intersection point of the above second line segment and [[ID=^48]]a color difference acquisition means for acquiring a color difference between a point on the second line segment and the point on the first line segment The plurality of the second line segmentsThe present invention is characterized by having a generation means for generating a color difference map based on color differences. [Effects of the Invention]

[0007] According to the present invention, it becomes possible to generate color difference maps that can be applied to a variety of uses and needs. [Brief explanation of the drawing]

[0008] [Figure 1] This diagram shows the hardware configuration of an information processing device. [Figure 2] This figure shows the logical configuration of the information processing device according to the first embodiment. [Figure 3] This is a flowchart showing the entire information processing process in the first embodiment. [Figure 4] This figure shows an example of the UI in the first embodiment. [Figure 5] This is a state transition diagram of the information processing device in response to UI input in the first embodiment. [Figure 6] This is a flowchart of the evaluation line calculation process in the first embodiment. [Figure 7] This is a flowchart of the color difference calculation process in the first embodiment. [Figure 8] This flowchart shows the color difference map generation process of the first embodiment. [Figure 9] This figure shows the logical configuration of the information processing device according to the second embodiment. [Figure 10] This is a flowchart showing the overall information processing in the second embodiment. [Figure 11] This figure shows an example of the UI in the second embodiment. [Figure 12] This is a state transition diagram of the information processing device in response to UI input in the second embodiment. [Figure 13] This figure shows the logical configuration of the information processing device according to the third embodiment. [Figure 14] This is a flowchart showing the overall information processing in the third embodiment. [Figure 15]This figure shows an example of the UI in the third embodiment. [Figure 16] This is a state transition diagram of the information processing device in response to UI input in the third embodiment. [Figure 17] This is a flowchart of the evaluation line calculation process in the third embodiment. [Figure 18] This is a flowchart showing the overall information processing in the fourth embodiment. [Figure 19] This figure shows an example of the UI in the fourth embodiment. [Figure 20] This is a state transition diagram of the information processing device in response to UI input in the fourth embodiment. [Figure 21] This is a flowchart of the process involved in adding a new node. [Figure 22] This is a flowchart of the evaluation line calculation process in the fourth embodiment. [Figure 23] This figure shows the logical configuration of the information processing device according to the fifth embodiment. [Figure 24] This is a flowchart showing the overall information processing in the fifth embodiment. [Figure 25] This figure shows an example of the UI in the fifth embodiment. [Figure 26] This is a state transition diagram of the information processing device in response to UI input in the fifth embodiment. [Figure 27] This is a flowchart of the reference line calculation process in the fifth embodiment. [Figure 28] This diagram explains the reference line calculated based on the differences in options. [Figure 29] This figure shows the list table in the fifth embodiment. [Figure 30] This is an explanatory diagram of the process of dividing boundary lines. [Figure 31] This is a flowchart of the evaluation line calculation process in the fifth embodiment. [Figure 32] This is a flowchart of the color difference map generation process in the fifth embodiment. [Modes for carrying out the invention]

[0009] The embodiments will be described below with reference to the drawings. Note that the following embodiments do not limit the present invention, and not all combinations of features described in these embodiments are essential to the present invention. The configurations shown in the following embodiments are merely examples, and the present invention is not limited to the illustrated configurations. Furthermore, in the embodiments described below, the same components will be denoted by the same reference numerals.

[0010] <Hardware configuration of the information processing device> Figure 1 is a diagram showing an example of the hardware configuration of an information processing device according to an embodiment. The CPU 101 uses the main memory 102 as work memory, executes the operating system (OS) and various programs stored in the hard disk drive (HDD) 103, and controls each configuration via the system bus 106. The CPU 101 also uses the HDD 103 and other various recording media for reading and writing data. The programs executed by the CPU 101 include programs that execute the information processing flows according to the embodiments described later.

[0011] The CPU 101 also displays the user interface (hereinafter referred to as UI) provided by the program on the monitor 105 and receives instructions and other information input from the user via the instruction input unit 107. In this embodiment, user instructions are input to the information processing device via the instruction input unit 107, but in order to simplify the description from here on, the description that user instructions are input via the instruction input unit 107 will be omitted as appropriate. The general-purpose interface (I / F) 104 is a serial bus interface such as USB, and the instruction input unit 107, such as a mouse or keyboard, and the image forming unit 108 are connected via the serial bus. The following describes the information processing performed by the CPU 101 in the configuration shown in Figure 1, which is carried out by the execution of an information processing application.

[0012] <Logical configuration of the information processing device in the first embodiment> Figure 2 is a block diagram showing the logical configuration of the information processing device in the first embodiment. The information processing device of this embodiment has the logical configuration shown in Figure 2, which allows setting a plurality of reference points and evaluation points within an image acquired by imaging, and generating and displaying the color difference between these plurality of reference points and evaluation points as a color difference map. Specifically, the information processing device of this embodiment includes an image acquisition unit 21, a reference line setting unit 22, an evaluation line calculation unit 23, a color difference calculation unit 24, a color difference map generation unit 25, and an output unit 26.

[0013] The image acquisition unit 21 acquires image data input in response to instructions from the user. The image data acquired by the image acquisition unit 21 may be data sent directly from a digital camera, data acquired by a digital camera and stored in the HDD 103, or data acquired via a network (not shown). The reference line setting unit 22 sets a reference line consisting of a set of reference points as a first line segment in the image acquired by the image acquisition unit 21. In this embodiment, the setting of the reference line as the first line segment is performed by specifying the start and end points according to the user's instructions. Details of the reference line and reference points will be described later.

[0014] The evaluation line calculation unit 23 calculates a second line segment passing through a reference point on the line segment of the reference line set by the reference line setting unit 22 as the evaluation line, and further calculates evaluation points on the line segment of that evaluation line. Details of the evaluation line and evaluation points will be described later.

[0015] The color difference calculation unit 24 performs a color difference acquisition process to calculate the color difference between a reference point on the reference line and an evaluation point on the evaluation line. Details of the color difference acquisition (color difference calculation) process will be described later. The color difference map generation unit 25 generates a color difference map representing the two-dimensional distribution of color differences based on the color differences calculated by the color difference calculation unit 24. Details of color difference map generation will be described later. The output unit 26 displays the color difference map generated by the color difference map generation unit 25 on the monitor 105.

[0016] Figure 3 is a flowchart showing the overall flow of information processing performed by the information processing device of the first embodiment. The operations described below are initiated when the user launches an information processing application via the instruction input unit 107 and then inputs predetermined instructions to the UI (user interface) described later.

[0017] In step S301, the image acquisition unit 21 acquires captured image data. In step S302, the reference line setting unit 22 sets the reference line. In step S303, the evaluation line calculation unit 23 calculates an evaluation line for a reference point on the reference line and calculates evaluation points on the evaluation line by an evaluation line calculation process described later. The evaluation points on the evaluation line are comparison points that are used to compare the color with respect to the reference point on the reference line. In step S304, the color difference calculation unit 24 acquires color difference information by a color difference calculation process described later, using a reference point and a comparison point (evaluation point). In step S305, the color difference map generation unit 25 generates a color difference map using the color difference information through a color difference map generation process described later. The output unit 26 then displays the generated color difference map on the monitor 105. After that, the information processing device performs processing related to the end of the process.

[0018] (UI in the information processing application of the first embodiment) Figure 4 is a schematic diagram showing the UI of the information processing device in the first embodiment. The input image setting button 41 is a button that the user will press when setting the input image. The color difference map display button 45 is a button that the user presses when they want to display the color difference map. The exit button 49 is the button the user presses to close the application.

[0019] The image display window 42 is a window that displays an image set by the user via instructions to the input image setting button 41. Plot 43 in the image display window 42 is a plot representing the starting point of the reference line and is specified by the user. Plot 44 is a plot representing the ending point of the reference line and is also specified by the user. In the first embodiment, a first line segment with plots 43 and 44 as its endpoints is set as the reference line.

[0020] The color difference map display window 46 is a window that displays the color difference map obtained through the color difference acquisition process (color difference calculation process) and color difference map generation process described later. Plot 47 in the color difference map display window 46 is a plot corresponding to the starting point of the reference line and shows the same position as the user-specified coordinates in plot 43. Plot 48 is a plot corresponding to the ending point of the reference line and shows the same position as the user-specified coordinates in plot 44.

[0021] Figure 5 is a state transition diagram showing the state transitions in the information processing device in response to user input to the UI according to the first embodiment. The control of each state transition shown in the state transition diagram of Figure 5 and the other state transition diagrams described thereafter is performed by the CPU 101.

[0022] When the user inputs an application launch command, the information processing device transitions to state ST501, which displays the aforementioned UI on monitor 105. After state ST501, the information processing device transitions to state ST502.

[0023] In state ST502, when the user presses the input image setting button 41, the information processing device transitions to state ST503, reads the image specified by the user, and displays it in the image display window 42. Image specification in state ST502 and image acquisition in state ST503 correspond to the image acquisition process in step S301 by the image acquisition unit 21. Once the image acquisition process in state ST503 is completed, the information processing device transitions to state ST502.

[0024] Next, in state ST502, when the user sets plot 43 within the image display window 42, the information processing device transitions to state ST504. Then, in state ST504, when the user sets plot 44, the information processing device acquires coordinate information for the areas set by plots 43 and 44. That is, the start and end points of the first line segment, the reference line, are identified by the coordinate information of the areas set by plots 43 and 44, and the reference line is set. The acquisition of coordinate information and setting of the line segment in state ST504 corresponds to the reference line setting process in step S302 by the reference line setting unit 22. When the reference line setting process in state ST504 is completed, the information processing device transitions to state ST505.

[0025] When the system transitions to state ST505, the information processing device calculates the evaluation line through the evaluation line calculation process. The details of this process will be described later, but the calculation of the evaluation line in state ST505 corresponds to the evaluation line calculation process in step S303 by the evaluation line calculation unit 23. After the evaluation line calculation process in state ST505 is completed, the information processing device transitions to state ST502.

[0026] Next, in state ST502, when the user presses the color difference map display button 45, the information processing device transitions to state ST506. In state ST506, the information processing device calculates the color difference, generates a color difference map, and displays the color difference map in the color difference map display window 46. In state ST506, the color difference calculation corresponds to the color difference calculation process in step S304 by the color difference calculation unit 24, and the generation of the color difference map corresponds to the color difference map generation process in step S305 by the color difference map generation unit 25. Once the color difference calculation, color difference map generation, and display processes in state ST506 are completed, the information processing device transitions to state ST502.

[0027] Subsequently, in state ST502, when the user presses the exit button 49, the information processing device transitions to state ST507 and performs the processing associated with termination.

[0028] <Evaluation line calculation processing flow in step S303> Figure 6 is a flowchart showing the flow of the evaluation line calculation process performed in the evaluation line calculation unit 23 of the information processing apparatus according to the first embodiment. First, in step S601, the evaluation line calculation unit 23 acquires input image information and reference line information. The reference line information acquired at this time is the coordinate values ​​(xs, ys) of plot 43 and the coordinate values ​​(xe, ye) of plot 44. The coordinate system here is set with the upper left corner of the input image as the origin. Note that if the input image is used directly, the noise in the image may cause sudden values ​​to be calculated in the color difference calculation, so an image after performing a moving average of multiple pixels may be used.

[0029] Next, in step S602, the evaluation line calculation unit 23 calculates reference points on the reference line. In this embodiment, I is used as the identification information (ID) for the points (reference points) to be calculated on the reference line, and the initial value is set to 0. An example of a calculation formula for determining the total number of reference points N is shown in equation (1). Note that N represents the largest integer value that does not exceed the number inside the Gaussian symbol used in equation (1). The x,y coordinate values ​​(xi,yi) of the reference point I are calculated using equation (2).

[0030]

number

[0031] Next, in step S603, the evaluation line calculation unit 23 calculates the RGB values ​​of the three primary colors at reference point I. The RGB values ​​(Ri, Gi, Bi) of reference point I are calculated using equation (3).

[0032]

number

[0033] Next, in step S604, the evaluation line calculation unit 23 determines whether or not RGB values ​​have been calculated for all reference points I. If they have been calculated, it initializes I to 0 and proceeds to step S605. Otherwise, it adds 1 to I and returns to step S602.

[0034] When the process proceeds to step S605, the evaluation line calculation unit 23 calculates the first evaluation line using equation (4). Here, the first evaluation line is a straight line passing through the reference point I, and as an example, it is a straight line perpendicular to the reference line. However, it is not limited to this example, as long as it is a straight line that intersects the reference line and passes through the reference point I. For example, a line segment that passes through the reference point I and is parallel to the horizontal or vertical direction of the image may be used as the evaluation line.

[0035]

number

[0036] Next, in step S606, the evaluation line calculation unit 23 calculates a comparison point IJ (evaluation point) on the evaluation line. Here, IJ is used to represent the identification information (ID) of the Jth point on the Ith evaluation line, and the initial values ​​of I and J are both set to 0. An example of a calculation formula for determining the total number of comparison points NI on the Ith evaluation line is shown in equation (5). Here, the intersection points of any two sides of the image with the evaluation line are represented as P1(xp1,yp1) and P2(xp2,yp2). The coordinate values ​​(xij,yij) of the comparison point IJ are calculated using equation (6).

[0037]

number

[0038] Next, in step S607, the evaluation line calculation unit 23 calculates the RGB values ​​of comparison point IJ. The RGB values ​​(Rij, Gij, Bij) of comparison point IJ are calculated using equation (7).

[0039]

number

[0040] Next, in step S608, the evaluation line calculation unit 23 determines whether or not processing has been performed for all comparison points IJ. If processing has been performed, it adds 1 to I and proceeds to step S609; otherwise, it adds 1 to J and returns to step S606. When the process proceeds to step S609, the evaluation line calculation unit 23 determines whether processing has been performed for all evaluation lines. If processing has not been performed, it adds 1 to I and returns to step S605. Otherwise, it performs the processing associated with termination.

[0041] <Color difference calculation processing flow in step S304> Figure 7 is a flowchart showing the flow of the color difference calculation process performed in the color difference calculation unit 24 of the information processing apparatus according to the first embodiment. First, in step S701, the color difference calculation unit 24 obtains the coordinate values ​​and RGB values ​​of the reference point I and the comparison point IJ. Next, in step S702, the color difference calculation unit 24 calculates Lab. Lab is calculated from the pixel values ​​RGB using equations (8) and (9).

[0042]

number

[0043] Next, in step S703, the color difference calculation unit 24 calculates the color difference. As an example of calculation, when the reference point I is C1(L1, a1, b1) and the comparison point IJ is C2(L2, a2, b2), an example of calculating ΔE is shown in equation (10).

[0044]

number

[0045] Alternatively, difference information in the color space such as ΔL, Δa, and Δb may be used instead of ΔE. An example of calculating ΔL is shown in equation (11). An example of calculating Δa is shown in equation (12), and an example of calculating Δb is shown in equation (13).

[0046] ΔL = L2 - L1 Equation (11) Δa=a2-a1 formula (12) Δb=b2-b1 formula (13)

[0047] Next, in step S704, the color difference calculation unit 24 determines whether or not processing has been performed for all comparison points on the evaluation line. If processing has been performed, it proceeds to step S705; otherwise, it adds 1 to J and returns to step S702.

[0048] Next, in step S705, the color difference calculation unit 24 determines whether processing has been performed for all evaluation lines. If processing has not been performed, it adds 1 to I and returns to step S702. Otherwise, it performs the processing associated with termination.

[0049] <Color difference map generation process flow in step S305> Figure 8 is a flowchart showing the flow of the color difference map generation process performed in the color difference map generation unit 25 of the information processing apparatus according to the first embodiment. First, in step S801, the color difference map generation unit 25 acquires color difference information (referred to as color difference IJ), which is the color difference of the Jth comparison point IJ on the Ith evaluation line.

[0050] Next, in step S802, the color difference map generation unit 25 compares A, which represents the maximum color difference, with the color difference IJ. If the color difference IJ is greater than the maximum color difference A, the process proceeds to step S803; otherwise, the process proceeds to step S804. The initial value of A, the maximum color difference, is set to 0.

[0051] If the process proceeds to step S803, the color difference map generation unit 25 updates the maximum color difference A to the color difference information of color difference IJ and proceeds to step S804. If the process proceeds to step S804, the color difference map generation unit 25 determines whether or not it has processed all the color difference information for the color difference IJ. If it has processed, it proceeds to step 805; otherwise, it returns to step S802.

[0052] In step S805, the color difference map generation unit 25 calculates the RGB values ​​in the color difference map of the comparison point IJ. Here, as an example of a color difference map, we will explain it as a gray gradient using 8-bit color signal values, where a color difference of 0 is RGB(0,0,0) and the maximum color difference is RGB(255,255,255). The RGB values ​​are calculated using equation (14). In addition, nonlinear transformations such as gamma transformation or log transformation may be used to obtain a visually smooth gradient.

[0053] R=G=B=(ΔE / A)×255 Equation (14)

[0054] Next, in step S806, the color difference map generation unit 25 determines whether or not processing has been performed for all comparison points IJ. If processing has not been performed, it returns to step S805; otherwise, it performs the processing associated with termination.

[0055] As described above, in the information processing device of the first embodiment, a set of reference points is set linearly as a first line segment (reference line) within the image, and a set of evaluation points (comparison points) for which color difference is calculated for each reference point is calculated as a second line segment (evaluation line). The information processing device of the first embodiment then calculates the color difference between these multiple reference points and evaluation points. Thus, according to the first embodiment, it is possible to generate and display the color difference between multiple reference points and evaluation points specified in the image as a color difference map.

[0056] <Second Embodiment> In the first embodiment, a method was described for setting a reference line within the input image and automatically setting evaluation lines that intersect the reference line. On the other hand, in the field of color inspection, it is sometimes necessary to control the vector of the evaluation line independently of the reference line due to the relationship of illuminance distribution due to lighting. Therefore, in the second embodiment, in addition to setting the reference line, evaluation lines are also set, and a color difference map is generated based on the information of the reference points on the set reference line and the evaluation points (comparison points) on the evaluation line. Note that in the second embodiment, only the configuration and processing that differ from the first embodiment will be described. The hardware configuration of the information processing device according to the second embodiment is the same as the configuration shown in Figure 1, so its illustration and description are omitted.

[0057] Figure 9 is a block diagram showing the logical configuration of the information processing device in the second embodiment. The information processing device of the second embodiment includes an image acquisition unit 91, a reference line setting unit 92, an evaluation line setting unit 93, an evaluation line calculation unit 94, a color difference calculation unit 95, a color difference map generation unit 96, and an output unit 97.

[0058] The image acquisition unit 91 acquires the captured image data input in accordance with the user's instructions. The reference line setting unit 92 sets a reference line, which is a set of reference points, as a first line segment in the image acquired by the image acquisition unit 91. The setting of the reference line (first line segment) is performed by specifying the start and end points by user instruction, as in the first embodiment. Details of the reference line setting process in the second embodiment will be described later.

[0059] The evaluation line setting unit 93 sets an evaluation line, which is a set of evaluation points (comparison points), as a second line segment in the image acquired by the image acquisition unit 91. In the second embodiment, the evaluation line (second line segment) is set by specifying the start and end points according to the user's instructions. Details of the evaluation line setting process in the second embodiment will be described later.

[0060] The evaluation line calculation unit 94 calculates a line that passes through a reference point on the reference line and is parallel to the evaluation line, and also calculates the evaluation points on the calculated line. Details of the evaluation line calculation process in the second embodiment will be described later. The color difference calculation unit 95 calculates the color difference between the reference point and the evaluation point on the line. The color difference map generation unit 96 calculates a color difference map based on the calculated color difference. The output unit 97 displays the generated color difference map.

[0061] Figure 10 is a flowchart showing the overall flow of information processing performed in the information processing device of the second embodiment. The operations described below are initiated when the user launches an information processing application and then inputs predetermined instructions to the UI described later.

[0062] In step S1001, the image acquisition unit 91 acquires captured image data. In step S1002, the reference line setting unit 92 sets the reference line. In step S1003, the evaluation line setting unit 93 sets a base evaluation line (referred to as the base evaluation line in this embodiment).

[0063] In step S1004, the evaluation line calculation unit 94 performs an intersection determination between the reference line and the base evaluation line. The determination formula for the intersection determination between the reference line and the base evaluation line is the formula shown in equation (15) below. In step S1004, the evaluation line calculation unit 94 determines that there is an intersection if the determination result E in equation (15) satisfies the conditions, and that there is no intersection otherwise. If there is no intersection, the process returns to step S1003, and the evaluation line setting unit 93 sets the base evaluation line again. However, the coordinates of the starting point of the reference line are (xs, ys) and the coordinates of the ending point are (xe, ye), and the coordinates of the starting point of the base evaluation line are (xp1, yp1) and the coordinates of the ending point are (xp2, yp2).

[0064]

number

[0065] Next, in step S1005, the evaluation line calculation unit 94 calculates a line segment parallel to the base evaluation line set in step S1003 as an evaluation line with respect to a reference point on the reference line. The evaluation line calculated here can be expressed by the following equation (16), where the coordinates of the reference point are (xi, yi). Also in step S1005, the evaluation line calculation unit 94 calculates evaluation points (comparison points) on each evaluation line in the same way as in the first embodiment, at the same time as calculating the evaluation lines.

[0066]

number

[0067] Next, in step S1006, the color difference calculation unit 95 calculates color difference information by performing the same color difference calculation process as described in step S304 of the first embodiment. Subsequently, in step S1007, the color difference map generation unit 96 generates a color difference map by performing the same color difference map generation process as described in step S305 of the first embodiment. Then, the output unit 97 displays the color difference map on the monitor 105. After that, the information processing device performs processing related to the end of the process.

[0068] <UI in the information processing application of the second embodiment> Figure 11 is a schematic diagram representing the UI in the second embodiment. The input image setting button 1101 is a button that the user will press when setting the input image. The color difference map display button 1107 is a button that the user will press when they want to display the color difference map. The exit button 1113 is a button used to close the application and is operated by the user.

[0069] The image display window 1102 is a window that displays the image set by the input image setting button 1101. Plot 1103 in the image display window 1102 is a plot representing the starting point of the reference line, and plot 1104 is a plot representing the ending point of the reference line, both of which are specified by the user. Plot 1105 is a plot representing the starting point of the base evaluation line, and plot 1106 is a plot representing the ending point of the base evaluation line, both of which are specified by the user.

[0070] The color difference map display window 1108 is a window for displaying the calculated color difference map. Plot 1109 in the color difference map display window 1108 is a plot representing the starting point of the reference line and shows the same position as the user-specified coordinates in plot 1103. Plot 1110 is a plot representing the ending point of the reference line and shows the same position as the user-specified coordinates in plot 1104. Plot 1111 is a plot representing the starting point of the evaluation line and shows the same position as the user-specified coordinates in plot 1105. Plot 1112 is a plot representing the ending point of the evaluation line and shows the same position as the user-specified coordinates in plot 1106. Note that although plots 1111 and 1112 are plots that show the same positions as the user-specified plots 1105 and 1106, plots showing the starting and ending points of the evaluation line calculated by the evaluation line calculation unit 94 may also be displayed.

[0071] Figure 12 shows the state transitions of the information processing device in response to user input to the UI in the second embodiment. Upon receiving an application launch command from the user, the information processing device transitions to state ST1201, which displays the UI on monitor 105. After state ST1201, the information processing device transitions to state ST1202.

[0072] When the input image setting button 1101 is pressed in state ST1201, the information processing device transitions to state ST1203, reads the user-specified image, and displays it in the image display window 1102. The image acquisition in state ST1203 corresponds to the image acquisition process in step S1001 by the image acquisition unit 91. After state ST1203, the information processing device transitions to state ST1202.

[0073] Next, when plot 1103 is set in the image display window 1102 in state ST1202, the information processing device transitions to state ST1204. Then, in state ST1204, when plot 1104 is set, the information processing device acquires coordinate information for the area set by plot 1103 and plot 1104. This identifies the start and end points of the first line segment, the reference line, and sets the reference line. This acquisition of coordinate information and setting of the line segment in state ST1202 corresponds to the reference line setting process in step S1002 by the reference line setting unit 92. When this reference line setting process in state ST1204 is completed, the information processing device transitions to state ST1202.

[0074] Furthermore, when plot 1105 is set in the image display window 1102 in state ST1202, the information processing device transitions to state ST1205. Then, in state ST1205, when plot 1106 is set, the information processing device acquires the coordinate information of the area set by plot 1105 and plot 1106 as the coordinate information of the start and end points of the line segment which is the base evaluation line. This sets the base evaluation line. This acquisition of coordinate information and setting of the line segment in state ST1205 corresponds to the base evaluation line setting process in step S1003 by the evaluation line setting unit 93. When the base evaluation line setting process in state ST1205 is completed, the information processing device transitions to state ST1206.

[0075] When the system transitions to state ST1206, the information processing device calculates the intersection point between the reference line and the base evaluation line. This intersection calculation process in state ST1206 corresponds to the aforementioned cross-determination process in step S1004 by the evaluation line calculation unit 94. Once the cross-determination process in state ST206 is completed, the information processing device transitions to state ST1207.

[0076] When the system transitions to state ST1207, the information processing device calculates an evaluation line and then calculates evaluation points (comparison points) on that evaluation line. The calculation of the evaluation line and evaluation points in state ST1207 corresponds to the evaluation line calculation process described above in step S1005 of the evaluation line calculation unit 94. After state ST1207, the information processing device transitions to state ST1202.

[0077] Next, in state ST1202, when the user presses the color difference map display button 1107, the information processing device transitions to state ST1208. In state ST1208, the color difference map is generated and displayed in the color difference map display window 1108. The processing in state ST1208 corresponds to the color difference calculation process in step S1006 and the color difference map generation process in step S1007 by the color difference map generation unit 96, and the color difference map display process by the output unit 97. After state ST1208, the information processing device transitions to state ST1202.

[0078] Subsequently, in state ST1202, when the user presses the exit button 1113, the information processing device transitions to state ST1209 and performs the termination-related processing.

[0079] As described above, in the information processing device of the second embodiment, a reference line and a base evaluation line are set in the image, an intersection determination is made between the reference line and the base evaluation line, and an evaluation line is calculated based on the result of the intersection determination. The information processing device of the second embodiment then calculates the color difference between multiple reference points and evaluation points (comparison points) on the reference line and the calculated evaluation line and generates a color difference map. In other words, according to the information processing device of the second embodiment, it is possible to set the reference line and the evaluation line independently and generate a color difference map for multiple reference points.

[0080] <Third Embodiment> In the second embodiment, an example was described in which a color difference map for multiple reference points is generated by independently setting a reference line and an evaluation line. However, in this case, depending on how the evaluation line is set, it may be possible to generate a color difference map for areas that are not actually needed, which may increase the overall processing time. Therefore, in the third embodiment, instead of setting independent evaluation lines in the image, a method is described in which a rectangle is set in the image and the evaluation direction for color difference calculation is set based on that rectangle, thereby enabling the generation of the minimum necessary color difference map. In the third embodiment as well, only configurations and processes that differ from the first and second embodiments described above will be described. The hardware configuration of the information processing device according to the third embodiment is the same as the configuration shown in Figure 1, so its illustration and description are omitted.

[0081] Figure 13 is a block diagram showing the logical configuration of an information processing device according to the third embodiment. The information processing device of the third embodiment includes an image acquisition unit 1301, a reference line setting unit 1302, a rectangle setting unit 1303, an evaluation line calculation unit 1304, a color difference calculation unit 1305, a color difference map generation unit 1306, and an output unit 1307.

[0082] The image acquisition unit 1301 acquires the captured image data input according to the user's specifications. The reference line setting unit 1302 sets a reference line, which is a set of reference points, within the image acquired by the image acquisition unit 1301. The rectangle setting unit 1303 sets a rectangle within the image acquired by the image acquisition unit 1301 to determine the evaluation direction of the color difference relative to the reference line when generating color difference calculation and color difference map generation. In this embodiment, no independent evaluation line is set, but the evaluation direction of the color difference relative to the reference line is set by the rectangle. The rectangle is set as a rectangle with a first vertex and a second vertex specified by user instruction as diagonal vertices. In the third embodiment, if the vertical and horizontal directions of the rectangle coincide with the vertical and horizontal directions of the image, the evaluation direction of the color difference will be the vertical or horizontal direction of the image, but the evaluation direction of the color difference does not have to be limited to the vertical or horizontal direction of the image. For example, if the evaluation direction of the color difference is to be in a direction different from the vertical or horizontal direction of the image, the desired direction can be set as the evaluation direction of the color difference by rotating the rectangle relative to the image.

[0083] The evaluation line calculation unit 1304 calculates a line that passes through a reference point on the reference line and is parallel to the horizontal or vertical direction of the rectangle, and also calculates points on the calculated line as evaluation points (comparison points). The direction of the line parallel to the horizontal or vertical direction of the rectangle corresponds to the evaluation direction of the color difference, and in this embodiment, the line parallel to the horizontal or vertical direction of the rectangle is used as the evaluation line. The color difference calculation unit 1305 calculates the color difference between the reference point and the evaluation point (comparison point) on the evaluation line. The color difference map generation unit 1306 calculates a color difference map based on the calculated color difference. The output unit 1307 displays the generated color difference map.

[0084] Figure 14 is a flowchart showing the overall flow of information processing performed in the information processing device of the third embodiment. The operations described below are initiated when the user launches an information processing application and then inputs predetermined instructions to the UI described later.

[0085] In step S1401, the image acquisition unit 1301 acquires image data. In step S1402, the reference line setting unit 1302 sets the reference line. In step S1403, the rectangle setting unit 1303 sets a rectangle, and the evaluation line calculation unit 1304 calculates an evaluation line. In this embodiment, the evaluation line calculation unit 1304 calculates an evaluation line for a reference point on a reference line and a comparison point on the evaluation line based on the rectangle set by the rectangle setting unit 1303, through a process described later. In step S1404, the color difference calculation unit 1305 calculates color difference information by the same process as described in step S304 of the first embodiment. In step S1405, the color difference map generation unit 1306 generates a color difference map using the same process as described in step S305 of the first embodiment. The output unit 1307 then displays the color difference map on the monitor 105. After that, the information processing device performs processing related to the end of the process.

[0086] <UI in the information processing application of the third embodiment> Figure 15 is a schematic diagram representing the UI in the third embodiment. The input image setting button 1501 is a button that the user presses when setting the input image. Radio button 1508 is selected by the user when the color difference evaluation direction is set to the horizontal direction of the rectangle. Radio button 1509 is selected by the user when the color difference evaluation direction is set to the vertical direction of the rectangle. The color difference map display button 1510 is a button that the user will press when they want to display the color difference map. The Exit button 1517 is a button that the user will press when they want to exit the application.

[0087] The image display window 1502 is a window that displays the image set by the input image setting button 1501. Plot 1503 is a plot representing the starting point of the reference line, and plot 1504 is a plot representing the ending point of the reference line, both specified by the user. Plot 1505 is a plot representing the first vertex of the rectangle, and plot 1506 is a plot representing the second vertex of the rectangle, both specified by the user. Rectangle 1507 is set as a rectangle represented by a dashed line with plots 1505 and 1506 as diagonal vertices. Plot 1518 represents the third vertex of the rectangle, and plot 1519 represents the fourth vertex of the rectangle. However, since the rectangle is determined by the setting of the diagonal vertices of plot 1505 (first vertex) and plot 1506 (second vertex), plots 1518 (third vertex) and 1519 (fourth vertex) are not specified by the user but are automatically determined according to the rectangle setting. Furthermore, by specifying any of the vertices from the first to the fourth vertex and then issuing instructions such as rotation, the rectangle 1507 can be rotated within the image displayed in the image window 1502.

[0088] The color difference map display window 1511 is a window for displaying the calculated color difference map. Plot 1512 is a plot representing the starting point of the reference line and shows the same position as the user-specified coordinates in plot 1503. Plot 1513 is a plot representing the ending point of the reference line and shows the same position as the user-specified coordinates in plot 1504. Plot 1514 is a plot representing the first vertex of the rectangle and shows the same position as the user-specified coordinates in plot 1505. Plot 1515 is a plot representing the second vertex of the rectangle and shows the same position as the user-specified coordinates in plot 1506. Rectangle 1516 is a rectangle with the same coordinates as rectangle 1507, which has plots 1505 and 1506 as diagonal vertices. Note that if rectangle 1507 is a rotated rectangle in the image as described above, rectangle 1516 will also be a rotated rectangle.

[0089] Figure 16 shows the state transitions of the information processing device in response to user input to the UI in the third embodiment. Upon receiving an application launch command from the user, the information processing device transitions to state ST1601, which displays the UI on monitor 105. After state ST1601, the information processing device transitions to state ST1602.

[0090] In state ST1602, when the input image setting button 1501 is pressed, the information processing device transitions to state ST1603, reads the specified image, and displays it in the image display window 1502. The image acquisition in state ST1603 corresponds to the image acquisition process in step S1401 by the image acquisition unit 1301. After state ST1603, the information processing device transitions to state ST1602.

[0091] Next, in state ST1602, when plot 1503 is set within the image display window 1502, the information processing device transitions to state ST1604. Then, when plot 1504 is set in state ST1604, the information processing device acquires coordinate information for the area set by plot 1503 and plot 1504. This sets the first line segment, which is the reference line. The acquisition of coordinate information and setting of the line segment in state ST1604 corresponds to the reference line setting process in step S1402 by the reference line setting unit 1302. When the reference line setting process in state ST1604 is completed, the information processing device transitions to state ST1602.

[0092] Furthermore, in state ST1602, when plot 1505 is set within the image display window 1502, the information processing device transitions to state ST1605. Then, in state ST1605, when plot 1506 is set, the information processing device obtains coordinate information for a rectangle with plot 1505 as the first vertex and plot 1506 as the second vertex, and then transitions to state ST1606.

[0093] When the system transitions to state ST1606, the information processing device sets the direction of the evaluation line (i.e., the color difference evaluation direction) based on the rectangle. Here, for example, if radio button 1508 is selected, the horizontal direction of the rectangle is set as the direction of the evaluation line, while if radio button 1509 is selected, the vertical direction of the rectangle is set as the direction of the evaluation line. After the evaluation line direction (color difference evaluation direction) is set in state ST1606, the information processing device transitions to state ST1607. Then, in state ST1607, the information processing device calculates the comparison points on the evaluation line and then transitions to state ST1602. These processes from state ST1605 to state ST1607 correspond to the rectangle setting process in the rectangle setting unit 1303 and the evaluation line calculation process in the evaluation line calculation unit 1304 in step S1403.

[0094] Next, in state ST1602, when the color difference map display button 1510 is pressed, the information processing device transitions to state ST1608. In state ST1608, the information processing device generates a color difference map and then displays the color difference map in the color difference map display window 1511. The processing in state ST1608 corresponds to the color difference calculation process in step S1404 by the color difference calculation unit 1305 and the color difference map generation process in step S1405 by the color difference map generation unit 1306. After state ST1608, the information processing device transitions to state ST1602. Subsequently, in state ST1602, when the user presses the exit button 1517, the information processing device transitions to state ST1609 and performs the processing associated with termination.

[0095] <Evaluation line calculation process flow in step S1403> Figure 17 is a flowchart showing the flow of the evaluation line calculation process by the evaluation line calculation unit 1304 in the information processing device of the third embodiment. First, in step S1701, the evaluation line calculation unit 1304 acquires input image information and reference line information. The reference line information acquired at this time is the coordinate values ​​(xs, ys) of plot 1503 and the coordinate values ​​(xe, ye) of plot 1504.

[0096] Next, in step S1702, the evaluation line calculation unit 1304 calculates the reference point on the reference line using the aforementioned equation (1), and further calculates the coordinate values ​​(xi, yi) of the reference point I using the aforementioned equation (2). Next, in step S1703, the evaluation line calculation unit 1304 calculates the RGB values ​​of reference point I. The RGB values ​​(Ri, Gi, Bi) of reference point I are calculated using the formula (3) described above.

[0097] Next, in step S1704, the evaluation line calculation unit 1304 determines whether or not RGB values ​​have been calculated for all reference points I. If they have been calculated, it initializes I to 0 and proceeds to step S1705. Otherwise, it adds 1 to I and returns to step S1702.

[0098] When the process proceeds to step S1705, the evaluation line calculation unit 1304 calculates the first evaluation line. The direction of the evaluation line is obtained as either the horizontal or vertical direction of the rectangle, depending on the user's selection for radio button 1508 or radio button 1509. If the direction of the evaluation line is set to the horizontal direction of the rectangle, the evaluation line is represented by equation (17). If the direction of the evaluation line is set to the vertical direction of the rectangle, the evaluation line is represented by equation (18). Here, the coordinates of plot 1506 are (xp1, yp1), the coordinates of plot 1518 are (xp2, yp2), and the coordinates of plot 1519 are (xp3, yp3).

[0099]

number

[0100] Furthermore, an intersection check is performed between the reference line and the evaluation line. If the set direction does not result in an intersection, the direction of the evaluation line is changed to the direction in which it intersects.

[0101] Next, in step S1706, the evaluation line calculation unit 1304 calculates the comparison point IJ on the evaluation line. The comparison point IJ is represented as the Jth point on the Ith evaluation line. Here, if the direction of the evaluation line is the horizontal direction of the rectangle, the coordinate values ​​(xij, yij) of the comparison point IJ are calculated using equation (19). On the other hand, if the direction of the evaluation line is the vertical direction of the rectangle, the coordinate values ​​(xij, yij) of the comparison point IJ are calculated using equation (20).

[0102]

number

[0103] Next, in step S1707, the evaluation line calculation unit 1304 calculates the RGB values ​​of comparison point IJ. The RGB values ​​(Rij, Gij, Bij) of comparison point IJ are calculated using the formula (7) described above. Next, in step S1708, the evaluation line calculation unit 1304 determines whether or not processing has been performed for all comparison points IJ. If processing has been performed, it adds 1 to I and proceeds to step S1709; otherwise, it adds 1 to J and returns to step S1706. When the process proceeds to step S1709, the evaluation line calculation unit 1304 determines whether processing has been performed for all evaluation lines. If processing has not been performed, it adds 1 to I and returns to step S1705. Otherwise, it performs the processing associated with termination.

[0104] As explained above, in the information processing device of the third embodiment, the evaluation direction of color difference (direction of the evaluation line) is set based on a rectangle. As a result, the information processing device of the third embodiment can generate the minimum necessary color difference map, and thus the processing time can be shortened.

[0105] <Fourth Embodiment> In the embodiments described above, a method was explained for setting a set of reference points using a single straight line. However, if the subject has a three-dimensional shape or the lighting distribution is uneven, the set of reference points may be curved rather than straight. When the set of reference points has a curved distribution in this way, the method used in the embodiments described above may not function effectively. Therefore, in the fourth embodiment, a method for effectively setting a reference line even when the set of reference points has a non-linear distribution will be described. In the fourth embodiment, only the parts that differ from the embodiments described above will be explained. The hardware configuration of the information processing device according to the fourth embodiment is the same as the configuration shown in Figure 1, so its illustration and description will be omitted.

[0106] Figure 18 is a flowchart showing the flow of information processing performed in the information processing device of the fourth embodiment. The logical configuration of the information processing device in the fourth embodiment is the same as that described in Figure 2 of the first embodiment. However, in the case of the information processing device of the fourth embodiment, the processing in the reference line setting unit 22 and the evaluation line calculation unit 23 differs from the processing in the first embodiment. The operation described below is started when the user launches the information processing application and then the user inputs predetermined instructions to the UI described later.

[0107] In step S1801, the image acquisition unit 21 acquires captured image data. In step S1802, the reference line setting unit 22 sets the starting point and the first node of the reference line. Details of setting the first node and each node from the second node onward will be described later. In step S1803, the evaluation line calculation unit 23 calculates a base evaluation line for the line segment composed of the starting point and the first node. In step S1804, the reference line setting unit 22 sets each node from the second node onward by a process described later.

[0108] In step S1805, the evaluation line calculation unit 23 calculates the evaluation line by a process described later. In step S1806, the color difference calculation unit 24 calculates the color difference between the reference point and the evaluation point (comparison point) on the evaluation line by the same process as described in step S304 of the first embodiment. In step S1807, the color difference map generation unit 25 generates a color difference map using the same process as described in step S305 of the first embodiment, and the output unit 26 then displays the color difference map on the monitor 105. After that, the information processing device performs processing related to the end of the process.

[0109] (UI in the information processing application of the fourth embodiment) Figure 19 is a schematic diagram showing the UI of the information processing device in the fourth embodiment. The input image setting button 1901 is a button that the user will press when setting the input image. The color difference map display button 1907 is the button that the user presses when they want to display the color difference map. The Exit button 1913 is the button the user presses to close the application.

[0110] The image display window 1902 is a window that displays the image set by the input image setting button 1901. Plot 1903 is a plot representing the starting point of the reference line and is specified by the user. Plot 1904 is a plot representing the first node of the reference line and is specified by the user. Note that one or more plots representing nodes can be set, and plot 1905 is a plot representing another node that is set separately from plot 1904 of the first node and is specified by the user. Plot 1906 is a plot representing the end point of the reference line and is specified by the user. Note that the latest node may be set as the end point of the reference line when the color difference map display button 1907 is pressed.

[0111] The color difference map display window 1908 is a window that displays the calculated color difference map. Plot 1909 is a plot representing the starting point of the reference line and shows the same position as the user-specified coordinates in plot 1903. Plots 1910 and 1911 are plots representing each node of the reference line, with plot 1910 showing the same position as the user-specified coordinates in plot 1904, and plot 1911 showing the same position as the user-specified coordinates in plot 1905. Plot 1912 is a plot representing the ending point of the reference line and shows the same position as the user-specified coordinates in plot 1906.

[0112] Figure 20 shows the state transitions of the information processing device in response to user input to the UI of the fourth embodiment. Upon receiving an application launch command from the user, the information processing device transitions to state ST2001, which displays the UI on monitor 105. After state ST2001, the information processing device transitions to state ST2002.

[0113] In state ST2002, when the user presses the input image setting button 1901, the information processing device transitions to state ST2003, reads the image specified by the user, and displays that image in the image display window 1902. Image acquisition in states ST2002 and ST2003 corresponds to the image acquisition process in step S1801 by the image acquisition unit 21. After the processing in state ST2003 is completed, the information processing device transitions to state ST2002.

[0114] Next, in state ST2002, when plot 1903 is set by the user in the image display window 1902, the information processing device transitions to state ST2004. Then, in state ST2004, when plot 1904 is further set as the first node, the information processing device acquires the coordinate information of the area set by plot 1903 and plot 1904. This acquisition of coordinate information in state ST2004 corresponds to the setting process of the starting point and the first node of the reference line in step S1802 of the reference line setting unit 22.

[0115] Furthermore, the information processing device transitions to state ST2005 and calculates a base evaluation line for the starting point and first node of the reference line set in state ST2004. This calculation of the base evaluation line in state ST2005 corresponds to the evaluation line calculation process in step S1803 by the evaluation line calculation unit 23. After state ST2005, the information processing device transitions to state ST2002.

[0116] Here, as mentioned above, multiple nodes can be set for the reference line. When a plot is set as a new node in state ST2002, the information processing unit transitions to state ST2006. In state ST2006, the information processing unit determines whether the newly added node is suitable for setting the evaluation line. If the information processing unit determines that the new node is suitable, it adds the node and then transitions to state ST2007. On the other hand, if it determines that the node is not suitable, it does not add the node and transitions to state ST2002. Furthermore, each time a new node is added in state ST2002, the information processing unit repeats the process of adding a new node and making a determination in state ST2006, as described above. The addition of a new node and the determination in state ST2006 correspond to step S1804 in the reference line setting unit 22.

[0117] When the system transitions to state ST2007, the information processing device calculates the evaluation line and the evaluation points (comparison points) on that evaluation line. This processing in state ST2007 corresponds to the evaluation line calculation processing in step S1805 by the evaluation line calculation unit 23. After processing in state ST2007, the information processing device transitions to state ST2002.

[0118] Next, in state ST2002, when the color difference map display button 1907 is pressed, the information processing device transitions to state ST2008. Upon transitioning to state ST2008, the information processing device performs color difference calculation and color difference map generation, and displays the color difference map in the color difference map display window 1908. In state ST2008, the color difference calculation corresponds to the color difference calculation process in step S1806 by the color difference calculation unit 24, and the color difference map generation corresponds to the color difference map generation process in step S1807 by the color difference map generation unit 25. After state ST2008, the information processing device transitions to state ST2002.

[0119] Subsequently, in state ST2002, when the user presses the exit button 1913, the information processing device transitions to state ST2009 and performs the termination-related processing.

[0120] <Processing flow when adding a node in step S1804> Figure 21 is a flowchart showing the processing flow when a node is added in the evaluation line calculation unit 23 of the information processing device of the fourth embodiment.

[0121] First, in step S2101, the evaluation line calculation unit 23 acquires the starting point and each node of the reference line. Next, in step S2102, the evaluation line calculation unit 23 determines whether the line segment connecting the new node and the node immediately preceding it, and the line segment connecting the starting point of the reference line and the first node, satisfy the conditions of equation (21). If the evaluation line calculation unit 23 determines that the conditions of equation (21) are satisfied, it proceeds to step S2103; otherwise, it performs actions associated with termination. Note that the coordinates of the starting point are (x s ,y s ), the coordinates of the first node are (x1, y1), and the coordinates of the new node are (x n ,y n ), the coordinates of the node immediately preceding the new node are (x n-1 ,y n-1 )

[0122]

number

[0123] Furthermore, a straight line that is orthogonal to the line segment connecting the i-th node and the (i-1)th node (the previous adjacent node) and passes through the i-th node is represented by equation (22). Similarly, a straight line that is orthogonal to the line segment connecting the i-th node and the (i-1)th node (the previous adjacent node) and passes through the (i-1)th node is represented by equation (23). When the evaluation line calculation unit 23 proceeds to step S2103, it calculates the intersection points of the two aforementioned straight lines for the nth newly added node and the two aforementioned straight lines for the other nodes, and determines, for example, whether or not an intersection point exists within a predetermined area in the image. This processing in step S2103 corresponds to the appropriateness determination in state ST2006 described above. If the evaluation line calculation unit 23 determines in step S2103 that no intersection point is within the predetermined area, it proceeds to step S2104; otherwise, it performs actions associated with termination. In other words, when calculating the evaluation line, the evaluation line calculation unit 23 ensures that the evaluation line (second line segment) in the section connecting two adjacent nodes does not intersect with the evaluation line (second line segment) in the section connecting any other adjacent nodes within a predetermined area.

[0124]

number

[0125] When the process proceeds to step S2104, the evaluation line calculation unit 23 determines whether processing has been performed for all sections. If processing has not been completed for all sections, the process returns to step S2103; otherwise, the process proceeds to step S2105. When the process proceeds to step S2105, the evaluation line calculation unit 23 records the node designated as a new node as a new node that has been determined to be appropriate in the appropriateness determination. Subsequently, the evaluation line calculation unit 23 performs the operation associated with the completion.

[0126] <Evaluation Line Calculation Processing Flow in Step S1805> FIG. 22 is a flowchart of the evaluation line calculation process executed by the evaluation line calculation unit 23 of the fourth embodiment.

[0127] First, in step S2201, the evaluation line calculation unit 23 acquires a reference line and a base evaluation line. Next, in step S2202, the evaluation line calculation unit 23 projects section I onto the vector formed by the starting point of the reference line and the first node, and obtains the length of the projected line segment. Here, the node section for projection is I, and the initial value is 1. Also, a straight line parallel to the vector formed by the starting point of the reference line and the first node and passing through the node immediately before the new node is shown in Equation (24).

[0128]

Equation

[0129] Furthermore, if the projected coordinates of the new node are (x n ', y n '), then the projected coordinates are as shown in Equation (25).

[0130]

Equation

[0131] Furthermore, the evaluation line calculation unit 23 calculates the number of reference points in section I from Equation (26).

[0132]

Equation

[0133] Next, in step S2203, the evaluation line calculation unit 23 calculates the coordinates of the reference points from Equation (27). Here, the i-th coordinate of section I is represented by (x i , y i ).

[0134]

number

[0135] Next, in step S2204, the evaluation line calculation unit 23 determines the RGB value of reference point Ii using the formula (3) described above. Next, in step S2205, the evaluation line calculation unit 23 determines whether or not processing has been performed for all reference points in section I. If processing has not been performed, it returns to step S2203; otherwise, it proceeds to step S2206. When the process proceeds to step S2206, the evaluation line calculation unit 23 calculates the first evaluation line for reference point Ii using equation (28).

[0136]

number

[0137] Next, in step S2207, the evaluation line calculation unit 23 calculates the comparison point IJ on the evaluation line. The comparison point is calculated using the aforementioned equations (5) and (6). Next, in step S2208, the evaluation line calculation unit 23 calculates the RGB values ​​of comparison point IJ. The RGB values ​​(Rij, Gij, Bij) of comparison point IJ are calculated using the aforementioned formula (7).

[0138] Next, in step S2209, the evaluation line calculation unit 23 determines whether or not processing has been performed for all comparison points IJ. If processing has been performed, it adds 1 to I and proceeds to step S2210; otherwise, it adds 1 to J and returns to step S2207.

[0139] When the process proceeds to step S2210, the evaluation line calculation unit 23 determines whether or not an evaluation line has been calculated for all reference points. If it has been calculated, the process proceeds to step S2211; otherwise, the process returns to step S2206. When the process proceeds to step S2211, the evaluation line calculation unit 23 determines whether processing has been performed for all sections. If processing has not been performed, it returns to step S2202; otherwise, it performs the processing associated with termination.

[0140] As described above, in the information processing device of the fourth embodiment, a reference line distributed in a curve can be obtained by setting a reference line using multiple nodes. As a result, according to the fourth embodiment, a color difference map can be generated even if the set of reference points is not distributed on a straight line.

[0141] <Fifth Embodiment> In the embodiments described above, examples were explained in which a reference line is set based on user specifications, and an evaluation line is also set or calculated. However, in the field of color inspection, it is sometimes necessary to evaluate color differences according to the distance from the boundaries of different parts. Therefore, in this fifth embodiment, an example is described in which the boundaries of various parts (hereinafter referred to as boundary lines) are extracted from image data captured with those parts as subjects, and a reference line and an evaluation line are calculated based on those boundary lines. Note that the hardware configuration of the information processing device in the fifth embodiment is the same as in Figure 1, so its illustration and description are omitted. The following description of the fifth embodiment will focus on the differences from the first embodiment.

[0142] Figure 23 is a block diagram showing the logical configuration of an information processing device according to the fifth embodiment. The information processing device of the fifth embodiment includes an image acquisition unit 2301, a boundary line setting unit 2302, a reference point setting unit 2303, a reference line determination unit 2304, an evaluation line determination unit 2305, a color difference map generation unit 2306, and an output unit 2307.

[0143] The image acquisition unit 2301 acquires the captured image data input in accordance with the user's instructions. The boundary line setting unit 2302 performs boundary setting processing on the captured image data acquired by the image acquisition unit 2301 to set boundary lines for dividing the image into regions. In this embodiment, the boundary line setting unit 2302 detects the boundaries of subjects such as parts that are captured in the image and generates binary image data that divides the detected boundaries into regions of 1 and the other regions into regions of 0. In other words, in this embodiment, the boundary lines that divide the image into regions are the lines of the boundary between 1 and 0 in the binary image data. As an example of such processing, this can be done by extracting the color of the lines of the boundaries between parts using a known thresholding process. However, this method is not limited to this method, and processing to extract the edges of the boundaries using the Canny method or the like may also be used.

[0144] The reference point setting unit 2303 sets a point through which the reference line passes within the image acquired by the image acquisition unit 1301, based on the user's specification. In this embodiment, the point through which the reference line passes is a reference point specified by the user, but it is not limited to a reference point; any information that indicates the distance between the boundary line and the reference line, and on which side of the boundary line the reference line is placed, may be used.

[0145] The reference line determination unit 2304 determines the reference line based on the boundary line and the reference point. Details of this process will be described later. The evaluation line determination unit 2305 determines the evaluation line based on the reference line. Details of this process will be described later. The color difference map generation unit 2306 calculates a color difference map from the reference line and the evaluation line. Details of this process will be described later. The output unit 2307 displays the generated color difference map on the monitor 105.

[0146] Figure 24 is a flowchart showing the overall flow of information processing performed in the information processing device of the fifth embodiment. The operations described below are initiated when the user launches an information processing application and then inputs predetermined instructions to the UI described later.

[0147] In step S2401, the image acquisition unit 2301 acquires captured image data. In step S2402, the boundary line setting unit 2302 sets boundary lines based on the boundaries of components, etc., in the captured image. In step S2403, the reference point setting unit 2303 sets a reference point. In this embodiment, the reference point is specified by the user. In step S2404, the reference line determination unit 2304 calculates a reference line based on the boundary line and the reference point. In step S2405, the evaluation line determination unit 2305 calculates the evaluation line based on the reference line. In step S2406, the color difference map generation unit 2306 generates a color difference map, and the output unit 2307 displays the color difference map on the monitor 105.

[0148] (UI in the information processing application of the fifth embodiment) Figure 25 is a schematic diagram representing the UI of the information processing device in the fifth embodiment. The input image setting button 2501 is a button that the user will press when setting the input image. The reference line setting button 2505 is a button pressed by the user to execute the processing of the reference line determination unit 2304. When the reference line setting button 2505 is pressed, the reference line determination unit 2304 calculates the reference line 2506 based on the reference point 2504, as will be described later. The color difference map display button 2509 is a button pressed by the user when the color difference map generation unit 2306 processes the data. When the color difference map display button 2509 is pressed, the color difference map generation unit 2306 generates a color difference map.

[0149] Options 2507 and 2508 are radio buttons that indicate to the user when selecting options related to the calculation of the reference line and evaluation line. Option 2507 represents the orthogonal direction option described later, and option 2508 represents the translation direction option described later; the user selects one of these two. The shape of the reference line 2506 will differ depending on the selection of the orthogonal direction option 2507 and the translation direction option 2508. The display switching buttons 2513 and 2514 are used by the user to switch between displaying a color difference map for each divided boundary line when the boundary line is divided into multiple sections, as will be described later. The Exit button 2515 is a button that the user presses when they want to close the application.

[0150] The image display window 2502 is a window that displays the image set by the input image setting button 2501. When an input image is set, a line representing the boundary line 2503 is displayed in the image display window 2502. In addition, a reference point 2504 specified by the user is displayed in the image display window 2502. Furthermore, if the reference line setting button 2505 is pressed, the reference line 2506 calculated by the reference line determination unit 2304 as described later is displayed on the image display window 2502.

[0151] The color difference map display window 1108 is a window for displaying the calculated color difference map. When the color difference map display button 2509 is pressed, the output unit 2307 displays the color difference map generated by the color difference map generation unit 2306 in the color difference map display window 1108. Also, for example, when the display switching button 2513 is pressed, the color difference maps calculated corresponding to the different boundary lines that have been divided as described later are displayed in the color difference map display window 2510. On the other hand, when the display switching button 2513 is pressed, the color difference map that was displayed before the display switching button 2514 was pressed is displayed in the color difference map display window 2510.

[0152] Figure 26 shows the state transitions of the information processing device in response to user input to the UI in the fifth embodiment. Upon receiving an application launch command from the user, the information processing device transitions to state ST2601, which displays the UI on monitor 105. After state ST2601, the information processing device transitions to state ST2602.

[0153] In state ST2602, when the user presses the input image setting button 2501, the information processing device transitions to state ST2603, reads the image specified by the user, and displays that image in the image display window 2502. Image acquisition in states ST2602 and ST2603 corresponds to the image acquisition process in step S2401 by the image acquisition unit 2301. After the image acquisition process in state ST2603 is completed, the information processing device transitions to state ST2604.

[0154] In state ST2604, the information processing device sets boundary lines 2503 based on the input captured image and displays these boundary lines 2503 together with the captured image in the image display window 2502. This boundary line setting in state ST2604 corresponds to the boundary line setting process in step S2402 by the boundary line setting unit 2302. After state ST2604, the information processing device transitions to state ST2602.

[0155] Next, in state ST2602, with the boundary line 2503 displayed, the user specifies a reference point, and when the user presses the reference line setting button 2505, the information processing device transitions to state ST2605. In state ST2605, the information processing device sets the reference point 2504 in the image display window 2502, calculates a reference line based on the reference point 2504 and the boundary line 2503, and displays the reference line 2506 in the image display window 2502. The reference point setting corresponds to the reference point setting process in step S2403 of the reference point setting unit 2303, and the reference line calculation corresponds to the reference line calculation process in step S2404 by the reference line determination unit 2304. After state ST2605, the information processing device transitions to state ST2606.

[0156] When the system transitions to state ST2606, the information processing device calculates an evaluation line based on the reference line. This calculation of the evaluation line in state ST2606 corresponds to the evaluation line calculation process in step S2405 by the evaluation line determination unit 2305. After state ST2606, the information processing device transitions to state ST2602.

[0157] Next, in state ST2602, when the user presses the color difference map display button 2509, the information processing device transitions to state ST2607. In state ST2607, the information processing device calculates the color difference map, generates the color difference map, and displays the color difference map using the same processing as in the embodiment described above. The color difference map calculation and color difference map generation in state ST2607 correspond to the processing in step S2406 by the color difference map generation unit 2306. The generated color difference map is then displayed in the color difference map display window 2510 by the output unit 2307. After state ST2607, the information processing device transitions to state ST2602.

[0158] Subsequently, when the user presses the exit button 2515 in state ST2602, the information processing device transitions to state ST2608 and performs the processing associated with the termination of the application.

[0159] <Processing flow for calculating the reference line in step S2404> Figure 27 is a detailed flowchart of the reference line calculation process performed by the reference line determination unit 2304 in step S2404 of the fifth embodiment. Here, the reference line calculation process performed by the reference line determination unit 2304 in step S2404 differs depending on whether the orthogonal direction option 2507 is selected or the parallel translation direction option 2508 is selected.

[0160] Figures 28(a) and 28(b) illustrate the differences in the reference lines calculated due to the differences in these options and represent the image display window 2502. If the orthogonal direction option 2507 is selected, in step S2404, the reference line determination unit 2304 calculates a reference line 2506 as shown by the dashed line in Figure 28(a). That is, when the orthogonal direction option 2507 is selected, the reference line determination unit 2304 generates a reference line 2506 as a set of points that are orthogonal to each point on the boundary line 2503 and are equally far apart from each other. As will be described in more detail later, when the orthogonal direction option 2507 is selected, the evaluation line determination unit 2305 determines an evaluation line that is orthogonal to the reference line, in other words, an evaluation line that is orthogonal to the boundary line.

[0161] On the other hand, if the translation direction option 2508 is selected, in step S2404, the reference line determination unit 2304 calculates a reference line 2506 as shown by the dashed line in Figure 28(b). That is, when the translation direction option 2508 is selected, the reference line determination unit 2304 generates the reference line 2506 as a set of points obtained by translating each point of the boundary line 2503. As will be described in more detail later, when the translation direction option 2508 is selected, the evaluation line determination unit 2305 determines an evaluation line that is parallel to that translation, in other words, an evaluation line that is parallel to the boundary line.

[0162] Figure 27(a) is a flowchart when the orthogonal option 2507 is selected, and Figure 27(b) is a flowchart when the translation option 2508 is selected.

[0163] Let's explain using the flowchart in Figure 27(a). First, in step S2701, the reference line determination unit 2304 acquires the boundary line and the reference point. Here, the binary image data showing the boundary line calculated in step S2402 and the coordinate information of the reference point set in step S2403 are stored, for example, in the main memory 102 or HDD 103. Therefore, the reference line determination unit 2304 acquires the binary image data showing the boundary line and the coordinate information of the reference point from the main memory 102 or HDD 103.

[0164] Next, in step S2702, the reference line determination unit 2304 thins the boundary line. That is, for example, in the binary image data representing the boundary line, the reference line determination unit 2304 thins the boundary line so that it becomes a line with a width of 1 pixel and connected to 4 or 8 neighbors. This process can be carried out by various known methods, such as repeatedly replacing pixels with non-zero pixel values ​​with a pixel value of 0 in the binary image data representing the boundary line according to the pattern of its neighboring pixels.

[0165] Furthermore, in step S2702, the reference line determination unit 2304 extracts boundary lines from the binary image data. For example, the reference line determination unit 2304 searches for the ends of boundary lines having non-zero pixel values ​​in the thinned binary image data and records their coordinates in the first row of the list shown in Figure 29. Next, the reference line determination unit 2304 searches for pixels connected to the pixels at the ends of the boundary lines and records their coordinates in the second row of the list. The reference line determination unit 2304 repeats this process until it reaches the opposite end of the boundary line. The columns for division number and evaluation line parameter in Figure 29 will be explained later, but nothing is recorded in the corresponding columns here.

[0166] Next, in step S2703, the reference line determination unit 2304 detects points on the boundary line where the curvature changes abruptly, such as points where the line changes from a straight line to a curve (hereinafter referred to as discontinuities), and performs a process to divide the boundary line at these discontinuities. Figure 30 is a diagram illustrating this process and represents the image display window 2502. The reference line determination unit 2304 extracts points where the boundary line 2503 is discontinuous, as shown at point 3201 in Figure 30(a). In this process, for example, a point (k) where the Ck value of equation (29), which is the second derivative of the boundary line, becomes greater than a threshold is defined as a discontinuity. Here, point (k) is the number in the list shown in Figure 29, and (xk,yk) are the x,y coordinate values ​​recorded on the same row as the number.

[0167]

number

[0168] The discontinuity point extraction process is not limited to this; for example, the reference line determination unit 2304 may extract Harris corners from the captured image data and detect the extracted pixels as discontinuity points.

[0169] Next, the reference line determination unit 2304 divides the boundary line 2503 at the discontinuity point 3201, as shown in the division line 3202 in Figure 30(b) and the division line 3203 in Figure 30(c). Specifically, the reference line determination unit 2304 performs the process of assigning different numbers to the division number column in the list in Figure 29, with the discontinuity point number k as the boundary.

[0170] Let's return to the explanation in Figure 27(a). In step S2704, the reference line determination unit 2304 calculates the distance D between the reference point and the boundary line. Here, the reference line determination unit 2304 calculates, for example, the Euclidean distance between the coordinate values ​​of each row in the list in Figure 29 and the coordinate values ​​of the reference point, and sets the minimum of these values ​​as the distance D.

[0171] Next, as part of the loop processing in step S2705, the reference line determination unit 2304 sets the division number in the list in Figure 29 as Bi, and performs the subsequent processing from steps S2706 to S2712 for all division numbers while changing Bi.

[0172] Furthermore, as part of the loop processing in step S2706, the reference line determination unit 2304 performs the processing from step S2707 to step S2711 for all pixels on the divided boundary line, in other words, for all rows with the same division number Bi in the list in Figure 29.

[0173] In step S2707, the reference line determination unit 2304 calculates the tangent vector tj for the pixel of the j-th row. This process can be expressed by equation (30).

[0174]

number

[0175] Next, in step S2708, the reference line determination unit 2304 calculates the vector оj in the direction from the j-th row pixel to the reference point using equation (31). However, (x о ,y о ) are the x,y coordinates of the reference point.

[0176]

number

[0177] Next, in step S2709, the reference line determination unit 2304 determines the outer product t j and j The cross product vector g j This is calculated using equation (32).

[0178]

number

[0179] g calculated using formula (32)j becomes [0, 0, 1] or [0, 0, -1], whereby it can be determined on which side (left or right) of the boundary line the reference point is located. Next, in step S2710, the reference line determination unit 2304 calculates the cross product vector m j of g j and t j using Equation (33). This is a process of calculating the direction orthogonal to t j in the xy plane.

[0180]

Number

[0181] Next, in step S2711, the reference line determination unit 2304 calculates, using Equation (34), the position that is at a distance D away in the direction orthogonal to t j .

[0182]

Number

[0183] Also in step SS2711, the reference line determination unit 2304 adds a new row to a list representing a reference line in the format shown in FIG. 29 (a list separate from the list indicating the boundary line), and records the x and y coordinate values of p j , and the value of the division number B j at that time in the corresponding columns.

[0184] The reference line determination unit 2304 generates a list of corresponding reference lines by performing the processes described in steps S2707 to S2711 for all pixels of the divided boundary line. However, the lines in the list of reference lines are not always continuous. In this case, in step S2712, the reference line determination unit 2304 performs interpolation using known methods such as linear interpolation or spline interpolation so that the lines are continuous. Specifically, in the list of reference lines, if the Euclidean distance between the coordinate values ​​of adjacent rows is greater than 1, the reference line determination unit 2304 inserts a new row and records the coordinate values ​​obtained by the interpolation process. The reference line determination unit 2304 repeats this process until there are no more rows with a Euclidean distance greater than 1.

[0185] By performing the processing shown in the flowchart of Figure 27(a) described above, the reference line determination unit 2304 can generate a reference line that is a certain distance away from the boundary line in a direction perpendicular to it.

[0186] Figure 27(b) is a flowchart showing the case when the translation direction option 2508 is selected. In Figure 27(b), the processes from steps S2701 to S2703 are the same as in Figure 27(a), so their explanation is omitted.

[0187] In step S2713, the reference line determination unit 2304 calculates the amount of translation from the boundary line and the reference point. Here, the reference line determination unit 2304 first calculates the Euclidean distance between the coordinate values ​​of each row in the boundary line list and the coordinate values ​​of the reference point, and records the coordinate values ​​of the row with the smallest Euclidean distance. Then, the reference line determination unit 2304 calculates the amount of translation vector s from that coordinate value using equation (35). Note that (x о ,y о ) are the x,y coordinate values ​​of the reference point, and (x d ,y d ) is the coordinate value of the row where the Euclidean distance is minimized.

[0188]

number

[0189] Next, in the loop processing of step S2714, the reference line determination unit 2304 determines the division number of the list of boundary lines shown in Figure 29 as B i Then, the subsequent steps S2715 to S2717 are performed by B i Perform this process for all division numbers, making changes as needed. Furthermore, in the loop processing of step S2715, the reference line determination unit 2304 determines all pixels on the divided boundary line, in other words, the division number B of the boundary line list. i For all rows that are the same, perform the process in step S2716.

[0190] Then, in step S2716, the reference line determination unit 2304 determines the coordinate value p of the translated pixel relative to the j-th row pixel. j This is calculated. This process can be performed using formula (36).

[0191]

number

[0192] Also in step S2716, the reference line determination unit 2304 adds a new row to the list representing the reference line in the format shown in Figure 29, p j The x,y coordinate values ​​and the corresponding division number B. j Record the value in the corresponding column.

[0193] Next, in step S2717, the reference line determination unit 2304 performs interpolation processing on the list of reference lines calculated as described above so that the lines are continuous. This process is the same as in step S2712, so its explanation is omitted here. As described above, the flowchart in Figure 27(b) allows the reference line determination unit 2304 to generate a reference line obtained by translating the boundary line.

[0194] <Evaluation line calculation processing flow in step S2405> Figure 31 is a flowchart showing the detailed flow of the evaluation line calculation performed by the evaluation line determination unit 2305 in step S2405.

[0195] First, in step S2801, the evaluation line determination unit 2305 reads the list of reference lines calculated in step S2404. As mentioned above, this list is stored in the format shown in Figure 29, for example, in the main memory 102 or the HDD 103.

[0196] Next, in the loop processing of step S2802, the evaluation line determination unit 2305 sets the division number of the list of reference lines as Li, and performs the subsequent processing from steps S2803 to S2804 for all division numbers while changing Li. Furthermore, in the loop processing of step S2803, the evaluation line determination unit 2305 performs the processing of step S2804 for all pixels on the divided reference line, i.e., for all rows with the same division number Li.

[0197] In step S2804, the evaluation line determination unit 2305 calculates the parameters of the evaluation line for the j-th row pixel. Here, the parameters of the evaluation line are, for example, the parameters a of the equation of the straight line represented by equation (37). j ,b j ,c j This indicates that.

[0198] a j x+b j y+c j =0 Equation (37)

[0199] If the orthogonal option 2507 is selected, these parameters are calculated using equation (38) so as to be orthogonal to the tangent vector.

[0200] a j =x j+1 -x j-1 Formula (38) b j = y j+1 -y j-1 Equation (39) c j = -a·x j -b·y j Equation (40)

[0201] On the other hand, when the translation direction option 2508 is selected, the evaluation line determination unit 2305 calculates the parameters using Equations (41) to (43) so as to be parallel to the translation vector s. Here, the x component of the translation vector s is x s and the y component is y s .

[0202] a j = -y s Equation (41) b j = x s Equation (41) c j = -a·x j -b·y j Equation (43)

[0203] Also in step S2804, the evaluation line determination unit 2305 sequentially records the parameters calculated as described above in the evaluation parameter column of the reference line list. The evaluation line determination unit 2305 calculates the corresponding evaluation lines for all rows of the reference line by performing the processes from step S2802 to step S2804 described above.

[0204] <Color difference map generation process flow in step S2406> FIG. 32(a) and FIG. 32(b) are flowcharts showing the detailed flow of the color difference map generation process executed by the color difference map generation unit 2306 in step S2406.

[0205] <000099​ If the input image coordinate system option 2511 is selected, the color difference map generation unit 2306 generates a color difference map using the same coordinate system as the input image. Therefore, the user can directly understand the correspondence between the position and color difference of each pixel in the input image. On the other hand, when the orthogonal coordinate system option 2512 is selected, the color difference map generation unit 2306 generates a color difference map in a coordinate system where the reference line and the evaluation line are orthogonal. Therefore, the user can easily understand how the color difference changes depending on the distance from the reference line.

[0206] Figure 32(a) is a flowchart of the color difference map generation process when input image coordinate system option 2511 is selected. In step S2901, the color difference map generation unit 2306 reads the input image and the list of reference lines calculated in the processing up to step S2405.

[0207] Next, in the loop processing of step S2902, the color difference map generation unit 2306 sets the division number of the list of reference lines to Li, and performs the subsequent processing from steps S2903 to S2910 for all division numbers while changing Li. In this process, a different color difference map is generated for each divided reference line.

[0208] Next, in step S2903, the color difference map generation unit 2306 allocates a memory area on the main memory 102 to store the generated color difference map as two-dimensional image data, and initializes it with pixel values ​​of 0. The size of the color difference map is set to be the same as the input image.

[0209] Next, in the loop processing of step S2904, the color difference map generation unit 2306 performs the processing from steps S2905 to S2909 for each pixel of the color difference map secured in the processing described above. The pixel of interest within this loop will be referred to as the k-th pixel, and the following explanation will continue.

[0210] In the loop processing of step S2905, the color difference map generation unit 2306 performs the processing from step S2906 to step S2908 for all pixels on the divided reference line, i.e., all rows with the same division number Li in the list of reference lines.

[0211] In step S2906, the color difference map generation unit 2306 calculates the distance h between the straight line determined by the evaluation line parameter of the j-th row and the k-th pixel of the color difference map. j This is calculated. This process can be done using equation (44). Here, the coordinates of the k-th pixel of the color difference map are (x k ,y k )

[0212]

number

[0213] Next, in step S2907, the color difference map generation unit 2306 determines whether the k-th pixel of the color difference map is on the evaluation line of the j-th row. Here, the color difference map generation unit 2306 determines h j If the value is smaller than the threshold TH, it is determined that the value is on the evaluation line and the process proceeds to step S2908; otherwise, the process in step S2908 is skipped.

[0214] When the process proceeds to step S2908, the color difference map generation unit 2306 generates the color difference map in the input image, using coordinates (x k ,y k The color of the pixel in ) and the coordinates (x) of the jth row of the reference line. j ,y j The color difference between the pixel color in ) and the other pixel is calculated. In this process, for example, ΔE is calculated according to equation (10) already explained, and it is recorded in the main memory 102.

[0215] Here, if the processing from steps S2906 to S2908 is performed for all rows on the reference line Li, multiple evaluation lines may intersect at the pixel k of interest in the color difference map, resulting in the calculation of multiple color differences. Therefore, in step S2909, the color difference map generation unit 2306 calculates the maximum or average value from these color differences and uses it as the pixel value of the pixel k of interest. That is, the color difference at the point where multiple evaluation lines (second line segments) intersect is calculated using the average or maximum value of the multiple color differences calculated for each of those multiple evaluation lines. Note that the user can select whether to use the maximum or average value from a UI (not shown) or the like.

[0216] Next, in step S2910, the color difference map generation unit 2306 records the color difference map generated by the processes from steps S2904 to S2909 in the main memory 102 or HDD 103, linking it to the division number Li of the reference line. Through the series of processes described above, the color difference map generation unit 2306 can generate a color difference map having the same coordinate system as the input image for each division number of the reference line.

[0217] Figure 29(b) is a flowchart showing the flow of the color difference map generation process in step S2406 when the Cartesian coordinate system option 2512 is selected. In step S2901, the color difference map generation unit 2306 reads the input image and the list of reference lines calculated in the processing up to step S2405.

[0218] Next, in the loop processing of step S2911, the color difference map generation unit 2306 sets the division number of the list of reference lines to Li, and performs the subsequent processing from steps S2912 to S2918 for all division numbers while changing Li. In this process as well, a different color difference map is generated for each divided reference line.

[0219] In step S2912, the color difference map generation unit 2306 allocates a memory area on the main memory 102 to store the generated color difference map as two-dimensional image data, and initializes it with a pixel value of 0. The size of the color difference map here is WxH pixels, where the horizontal dimension is the number of rows W belonging to the reference line Li and the vertical dimension is a preset H.

[0220] Next, in the loop processing of step S2913, the color difference map generation unit 2306 performs the processing from step S2906 to step S2908 for all pixels on the divided reference line, that is, for all rows with the same division number Li in the list of reference lines.

[0221] Furthermore, in the loop processing of step S2914, the color difference map generation unit 2306 performs the processing from step S2915 to step S2917 while changing the variable k from -H / 2 to H / 2 with respect to the vertical number of pixels H of the color difference map.

[0222] In step S2915, the color difference map generation unit 2306 calculates the coordinate t of a pixel that is k pixels away from the coordinate value of the jth row on a straight line determined by the evaluation line parameter of the jth row of the reference line. Specifically, the color difference map generation unit 2306 can calculate this using equation (45).

[0223]

number

[0224] Next, in step S2916, the color difference map generation unit 2306 generates the color of the pixel at coordinate t in the input image and the coordinate (x) of the jth row of the reference line. j ,y j The color difference between the pixel color in ) and the other pixel is calculated. In this process, for example, ΔE is calculated according to equation (10) already explained, and it is recorded in the main memory 102.

[0225] Next, in step S2917, the color difference map generation unit 2306 records the color difference calculated in step S2916 as the pixel value at the position where the x-coordinate is j and the y-coordinate is k+H / 2 in the color difference map. Subsequently, in step S2918, the color difference map generation unit 2306 records the color difference map generated by the processing in steps S2913 to S2917 in the main memory 102 or HDD 103, linked to the division number Li of the reference line. Through the series of processes described above, the color difference map generation unit 2306 can generate a color difference map having a coordinate system in which the reference line and the evaluation line are orthogonal to each division number of the reference line.

[0226] In the fifth embodiment, an example was described in which boundary lines are extracted from captured image data, and reference lines and evaluation lines are calculated based on these boundary lines. As a result, the information processing device of the fifth embodiment makes it possible to evaluate color differences according to the distance from the boundary between different parts. Furthermore, manually setting the reference lines and evaluation lines each time an evaluation is performed has the disadvantage that the direction of the evaluation lines changes each time, making it difficult to perform a stable evaluation. In contrast, in this embodiment, since the reference lines and evaluation lines are determined based on the boundary lines, the direction of the evaluation lines does not change each time an evaluation is performed, and a stable evaluation can be performed.

[0227] In the fifth embodiment, an example was described in which boundary lines are extracted and set from captured image data. However, the method is not limited to this, and for example, the user may manually draw boundary lines, or multiple boundary line candidates may be displayed and selected from among them. Furthermore, although this embodiment describes a configuration in which the reference line is determined from the boundary line and the reference point, it is not always necessary to specify the reference point. As long as the distance between the reference line and the boundary line, and whether the reference line is set on the left or right side of the boundary line can be determined, information other than the reference point can be used. For example, multiple candidate reference lines may be presented, and the user may select from among them. In this case, in the flowchart shown in Figure 27(a), multiple values ​​for the distance D between the boundary line and the reference point can be set in advance, and the reference lines calculated for each of these values ​​can be used as candidates. In addition, for each value of the distance D, the cross product vector g j By calculating reference lines for the two cases [0,0,1] and [0,0,-1], it is possible to set reference lines on both sides of the boundary line as candidates.

[0228] <Other Embodiments> In the embodiment described above, ΔE, ΔL, Δa, and Δb were used as the types of color difference information, but ΔC, Δh, etc. may also be used. Furthermore, while ΔE1976 was used as the type of color difference information, other evaluation values ​​representing color differences, such as ΔE1994 or ΔE2000, may also be used. Furthermore, while Lab was used as the color information, psychophysical values ​​such as JCH could also be used. In the second embodiment described above, an example was shown in which a start point and an end point are specified when setting the evaluation line. However, the start point of the reference line may be used as the start point of the evaluation line, and the evaluation line may be set based on a point different from the start point and a point on the reference line. The point different from the point on the reference line may be, for example, the end point of the evaluation line. In other words, the start point of the reference line may be used as the start point of the evaluation line, and the evaluation line may be set by specifying only the end point of the evaluation line.

[0229] The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions. The embodiments described above are merely examples of how the present invention can be implemented, and the technical scope of the invention should not be interpreted as being limited by them. In other words, the present invention can be implemented in various ways without departing from its technical concept or its main features. [Explanation of symbols]

[0230] 21: Image acquisition unit, 22: Reference line setting unit, 23: Evaluation line calculation unit, 24: Color difference calculation unit, 25: Color difference map generation unit, 26: Output unit

Claims

1. A first setting means for setting a first line segment in an image obtained by imaging, A second setting means for setting a second line segment in the aforementioned image, A color difference acquisition means for acquiring the color difference between the intersection point of the first line segment and the second line segment and a point on the second line segment, A generation means for generating a color difference map based on the color differences in a plurality of the second line segments, An information processing device characterized by having the following features.

2. The information processing apparatus according to claim 1, characterized in that the second line segment is a line segment perpendicular to the first line segment.

3. The information processing apparatus according to claim 1, characterized in that the second line segment is a line segment corresponding to the vertical or horizontal direction of the image.

4. The information processing apparatus according to claim 1, characterized in that the second setting means sets the second line segment so as to intersect with the first line segment.

5. The information processing apparatus according to claim 1 or 4, characterized in that the second setting means sets the second line segment based on the set start and end points.

6. The information processing apparatus according to claim 1 or 4, characterized in that the second setting means sets the second line segment based on the starting point of the first line segment and a point different from the point on the first line segment.

7. The system has a rectangle setting means for setting a rectangle for the aforementioned image, The information processing apparatus according to claim 1 or 4, characterized in that the second setting means sets the second line segment based on the rectangle.

8. The information processing apparatus according to claim 7, characterized in that the second setting means sets the second line segment corresponding to the vertical or horizontal direction of the rectangle.

9. The information processing apparatus according to claim 1 or 4, characterized in that the first line segment is a line segment composed of a start point and an end point and one or more nodes between the start point and the end point.

10. The information processing apparatus according to claim 9, characterized in that the second setting means ensures that a second line segment in an interval connecting adjacent nodes does not intersect a second line segment in an interval connecting other nodes within a predetermined area.

11. Boundary setting means for setting boundary lines to divide an image obtained by imaging into regions, A first setting means for setting a first line segment located a predetermined distance from the boundary line, A second setting means for setting a second line segment that determines the direction in which to extract multiple colors from the image to be compared with the colors of each point on the first line segment, An information processing device characterized by having the following features.

12. A color difference acquisition means for acquiring the color difference between a point on the first line segment and a point on the second line segment, A generation means for generating a color difference map based on the acquired color difference, The information processing apparatus according to claim 11, characterized by having the following features.

13. The information processing apparatus according to claim 12, characterized in that the second setting means sets the direction of the second line segment based on the first line segment.

14. The first setting means sets the first line segment in a direction perpendicular to the boundary line, at a predetermined distance from the boundary line, The information processing apparatus according to claim 13, characterized in that the second setting means sets the direction perpendicular to the first line segment to the direction of the second line segment.

15. The first setting means sets the first line segment in a direction that moves the boundary line in parallel, and is located a predetermined distance from the boundary line. The information processing apparatus according to claim 13, characterized in that the second setting means sets the direction of the translation to the direction of the second line segment.

16. The first setting means divides the boundary line at the point of change of the boundary line, and sets a plurality of the first line segments for each of the divided boundary lines. The information processing apparatus according to any one of claims 12 to 15, characterized in that the second setting means sets the second line segment for each of the plurality of first line segments.

17. The information processing apparatus according to claim 16, characterized in that the generation means generates the color difference map corresponding to each of the divided boundary lines.

18. The information processing apparatus according to claim 17, characterized in that, in the color difference map, the color difference at the points where the plurality of second line segments set for each of the plurality of first line segments intersect is the average or maximum value of the plurality of color differences calculated for each of the plurality of second line segments.

19. The information processing apparatus according to claim 17 or 18, characterized in that the generation means generates the color difference map in a coordinate system in which the first line segment and the second line segment are orthogonal.

20. A first setting step involves setting a first line segment in the image acquired by imaging, A second setting step involves setting a second line segment in the aforementioned image, A color difference acquisition step to acquire the color difference between the intersection point of the first line segment and the second line segment and a point on the second line segment, A generation step of generating a color difference map based on the color differences in a plurality of the second line segments, An information processing method characterized by having the following features.

21. A boundary setting step involves setting boundaries to divide the image acquired by imaging into regions, A first setting step of setting a first line segment located a predetermined distance from the boundary line, A second setting step involves setting a second line segment that determines the direction in which to extract multiple colors from the image to be compared with the colors of each point on the first line segment, An information processing method characterized by having the following features.

22. A program for causing a computer to function as an information processing device according to any one of claims 1 to 19.

Images