Image processing device and image processing method

Abstract

To provide a technique that enables determination of the presence or absence of an asperity defect in a boundary portion between an inspection area and non-inspection area while tolerating individual variations in uniformity of asperity in the boundary portion.SOLUTION: An asperity-related map based on multiple captured images of an object under inspection is obtained. The presence or absence of an asperity defect in a boundary portion between an inspection area and non-inspection area is determined on the basis of uniformity of the map in the boundary portion.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a technique for determining the presence or absence of unevenness defects in an object to be inspected. [Background technology]

[0002] As an appearance inspection technique for industrial products, a technique for detecting defects on the surface (inspection surface) of an object to be inspected is known. Patent Document 1 discloses a technique for determining whether an object is good or bad based on whether the angle difference between the normal vector of the object to be inspected and the normal vector of a reference object exceeds a threshold value. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2003-240539 A Summary of the Invention [Problem to be solved by the invention]

[0004] Some industrial products are manufactured by pouring a fluid into a container and solidifying it. In such industrial products, interfacial tension acts between the container and the fluid, so that an uneven shape is formed along the edge of the container in the area close to the wall of the container. However, if there is a defect in the manufacturing process, unevenness defects may occur in the uneven shape along the edge of the container. For this reason, an appearance inspection is performed that focuses on the uneven shape along the edge of the container.

[0005] In such a visual inspection of a surface having an uneven shape along the edge of a container, there is a need to detect uneven defects in the region along the edge of the container while allowing the feature indicating the inclination of the slope along the edge of the container to vary from one individual to another. For example, even if the feature indicating the inclination of the slope along the uneven shape along the edge of a container varies from one individual to another, it may be desired to classify the product as a non-defective product if the feature indicating the inclination of the slope is uniform among the individual products, and to classify the product as defective if the inclination of the slope is non-uniform among the individual products.

[0006] However, the technology described in Patent Document 1 detects uneven defects based on the angle difference between the normal vector of the inspection object and the normal vector of a reference object, so if the characteristics indicating the inclination of the slope along the edge of the container vary from one item to another, the product will be judged as defective even when it is intended to be a good product.

[0007] The present invention provides a technique that enables determination of the presence or absence of unevenness defects in a boundary portion between an inspection area and a non-inspection area, while allowing for individual variations in the uniformity of unevenness in the boundary portion. [Means for solving the problem]

[0008] According to one aspect of the present invention, the present invention is characterized by comprising an acquisition means for acquiring a map relating to unevenness based on a plurality of captured images of the object to be inspected, and a judgment means for judging the presence or absence of unevenness defects at the boundary portion between an inspection area and a non-inspection area based on the uniformity of the map at the boundary portion in the map. Effect of the Invention

[0009] According to the present invention, it is possible to determine the presence or absence of unevenness defects in a boundary portion between an inspection area and a non-inspection area, while allowing for individual variations in the uniformity of unevenness in the boundary portion. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram showing an example of a system configuration. [Diagram 2] FIG. 10A is a diagram showing an example of the exterior appearance of the system, FIG. 10B is an overhead view of the arrangement of the imaging unit 1012 and light sources 0 to 7 of the lighting unit 1013, and FIG. 10C is a block diagram showing an example of the hardware configuration of the system. [Diagram 3] 2 is a conceptual diagram illustrating a concavo-convex defect along the edge of a container in an inspection object 204. FIG. [Figure 4] 4 is a flowchart of a process for determining the presence or absence of a concavo-convex defect. [Diagram 5]5A to 5C are diagrams for explaining a process for determining the presence or absence of a concavo-convex defect in an uneven shape. [Figure 6] FIG. 13 is a diagram for explaining a second modified example. [Figure 7] 4 is a flowchart of a process for determining the presence or absence of a concavo-convex defect. [Figure 8] FIG. 13 is a diagram showing an example of a result of calculating an outlier value for a defective product. [Figure 9] 4 is a flowchart of a process for determining the presence or absence of a concavo-convex defect. [Figure 10] (a) is a top view of the reference object, and (b) is a top view of the test object. [Figure 11] 4 is a flowchart of a process for determining the presence or absence of a concavo-convex defect. [Figure 12] FIG. 13 is a diagram showing how a spatial filter is applied. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0012] [First embodiment] <System configuration example> An example of the configuration of a system according to this embodiment for performing visual inspection of an inspection target will be described with reference to the block diagram of Fig. 1. As shown in Fig. 1, the system according to this embodiment has an imaging device 101 and an image processing device 102.

[0013] First, a description will be given of the imaging device 101. The imaging device 101 is a device configured to capture an image of an inspection target object (inspection object) that is an object to be inspected for appearance, and includes an imaging control unit 1011, an imaging unit 1012, and an illumination unit 1013.

[0014] The imaging unit 1012 is a two-dimensional imaging device, such as a general digital camera, video camera, or industrial camera. The lighting unit 1013 has a plurality of light sources with different arrangement positions and lighting directions. The imaging control unit 1011 controls the operation of the imaging unit 1012 and the lighting unit 1013, such as the timing of imaging by the imaging unit 1012 and the timing of light emission of each light source in the lighting unit 1013.

[0015] Next, a description will be given of the image processing device 102. The image processing device 102 is a device configured to inspect the appearance of an object to be inspected based on a plurality of captured images of the object to be inspected captured by an imaging unit 1012, and includes an image processing unit 1021 and a detection unit 1022.

[0016] The image processing unit 1021 generates a feature map, which is a map relating to unevenness, based on a plurality of captured images of the inspection object captured by the imaging unit 1012. The detection unit 1022 judges the presence or absence of unevenness defects in a boundary portion between an inspection region and a non-inspection region in the feature map generated by the image processing unit 1021, based on the uniformity of the feature map in the boundary portion.

[0017] Next, an example of the appearance of the system according to this embodiment will be described with reference to Fig. 2(a) and (b). Fig. 2(a) is a diagram showing an example of the appearance of the system according to this embodiment, and Fig. 2(b) is an overhead view of the state in which the imaging unit 1012 and light sources 0 to 7 of the illumination unit 1013 are arranged.

[0018] The imaging unit 1012 is disposed directly above the object 204 with the imaging direction facing the object 204, for example, as shown in Fig. 2(a), in order to capture an image of the appearance of the object 204. The light sources in the illumination unit 1013 are disposed so as to surround the object 204, for example, as shown in Fig. 2(a) and (b), in order to irradiate the object 204 with light from various directions, and the illumination unit 1013 thereby constitutes a dome-shaped multi-light illumination.

[0019] In such a configuration, the imaging device 101 performs the following operations for i=0 to 7 in order. The imaging control unit 1011 outputs a light emission instruction (light emission instruction i) to the illumination unit 1013 to cause the light source i to emit light, and the illumination unit 1013 causes the light source i to emit light in response to the light emission instruction i. The imaging control unit 1011 then outputs an imaging instruction to the imaging unit 1012, and the imaging unit 1012 captures a still image (captured image) of the inspection object 204 in response to the imaging instruction, and outputs the captured image to the image processing device 102. As a result, the image processing device 102 acquires a captured image of the inspection object 204 irradiated with light by the light source i. The imaging control unit 1011 then outputs a turn-off instruction (turn-off instruction i) to the illumination unit 1013 to turn off the light source i, and the illumination unit 1013 turns off the light source i in response to the turn-off instruction i.

[0020] The above-described configuration of the imaging device 101 is merely one example of a configuration for acquiring a captured image of the inspection object 204 illuminated with light from a plurality of light sources having different arrangement positions and light illumination directions. Therefore, the arrangement positions, light illumination directions, number, etc. of the light sources are not limited to the above-described configuration (the number of light sources may be three or more).

[0021] Next, an example of the hardware configuration of the system according to this embodiment will be described with reference to the block diagram in Fig. 2(c). The CPU 203 executes various processes using computer programs and data stored in the RAM 202. As a result, the CPU 203 controls the operation of the entire image processing device 102, and executes or controls various processes described as processes performed by the image processing device 102.

[0022] The RAM 202 has an area for storing computer programs and data loaded from the ROM 204 or the storage device 205, and an area for storing various information (captured images, etc.) output from the imaging device 101 via the interface 206. The RAM 202 further has a work area used when the CPU 203 executes various processes. In this way, the RAM 202 can provide various areas as needed.

[0023] The ROM 204 stores setting data for the image processing device 102, computer programs and data related to the startup of the image processing device 102, computer programs and data related to the basic operations of the image processing device 102, and the like.

[0024] The storage device 205 is a non-volatile memory device such as a hard disk drive device. The storage device 205 stores an OS, computer programs and data for causing the CPU 203 to execute or control various processes described as processes performed by the image processing device 102, and the like. The computer programs stored in the storage device 205 also include computer programs for causing the CPU 203 to execute the functions of the image processing unit 1021 and the detection unit 1022. In the following, the image processing unit 1021 and the detection unit 1022 may be described as the subject of processing, but in reality, the CPU 203 executes computer programs corresponding to the image processing unit 1021 and the detection unit 1022 to realize the functions of the corresponding functional units. Note that the image processing unit 1021 and the detection unit 1022 may be implemented in hardware.

[0025] An external bus 208 is connected to the interface 206. To the external bus 208, a display 209, a mouse 210, a keyboard 211, and the imaging device 101 are connected.

[0026] The display 209 is a device having a liquid crystal screen or a touch panel screen, and can display the results of processing by the CPU 203 as images, characters, etc. The display 209 may be a projection device such as a projector that projects images and characters.

[0027] The keyboard 211 and mouse 210 are examples of a user interface, and the user can input various instructions and information to the image processing device 102 by operating them.

[0028] <Confirmation of the problem of this embodiment> In the visual inspection of a surface having an uneven shape along the boundary between an inspection area and a non-inspection area, there is a need to detect uneven defects in the area along the boundary while allowing the inclination of the slope along the boundary to vary depending on the individual. Here, the boundary between the inspection area and the non-inspection area may be an uneven shape formed along the edge of a container by the interfacial tension between the contents and the container on the surface of an industrial product manufactured by putting the contents into a container and solidifying it. In addition, the boundary between the inspection area and the non-inspection area may be an area where a slope occurs in a convex shape formed by embossing or a concave shape formed by debossing. Figure 3 shows an example of an inspection object in a visual inspection having such a need. In Figures 3(a) to (c), the upper row shows a cross-sectional view, the middle row shows a perspective view, and the lower row shows a top view.

[0029] The inspection object shown in FIG. 3 is an industrial product manufactured by pouring a fluid into a container 301 and solidifying it. At the time when the fluid is poured into the container 301, an interfacial tension acts between the container 301 and the fluid, so that an uneven shape 303 is formed along the edge 305 of the container in the region close to the wall of the container 301. For convenience of explanation, the edge 305 of the container is illustrated only in the top view in the lower part of FIG. 3(a). On the other hand, the surface 302 near the center of the container 301 is approximately flat. In this embodiment, as shown in FIG. 3, a surface having an uneven shape 303 along the boundary between the inspection area and the non-inspection area is inspected. In particular, in this embodiment, the purpose is to determine the presence or absence of uneven defects in the uneven shape 303 along the boundary between the inspection area and the non-inspection area.

[0030] At this time, there is a need to allow the inclination of the slope along the boundary between the inspection area and the non-inspection area to vary from one object to another. For example, the inclination of the slope of the uneven shape 303 is different between the inspection object in FIG. 3(a) and the inspection object in FIG. 3(b). However, the inclination of the slope of the uneven shape 303 is uniform along the boundary between the inspection area and the non-inspection area, i.e., the edge of the container 301, between the inspection object in FIG. 3(a) and the inspection object in FIG. 3(b). For this reason, it is desired to determine that both the inspection object in FIG. 3(a) and the inspection object in FIG. 3(b) have no unevenness defects.

[0031] On the other hand, the inspection object in FIG. 3(c) has an uneven slope of the uneven shape 303 along the boundary between the inspection area and the non-inspection area, i.e., the edge of the container 301. The cross-sectional view in the upper part of FIG. 3(c) shows that the height of the uneven shape 303 is higher on the right side than on the left side, and the slope of the uneven shape 303 is larger on the right side than on the left side. The top view in the lower part of FIG. 3(c) shows that such uneven defects 304 appear on the slope of the uneven shape 303 along the edge of the container 301. Therefore, it is desired to determine that the inspection object in FIG. 3(c) has an uneven defect. Note that the uneven defect 304 is shown by a dashed line along the edge of the container for convenience of explanation, but in reality it spreads to the area inside the dashed line (the area of ​​the slope of the uneven shape 303).

[0032] In this case, the technique described in Patent Document 1 detects uneven defects based on the angle difference between the normal vector of the inspection object and the normal vector of the reference object. Therefore, if the inspection object in FIG. 3(a) is set as the reference object in the uneven shape 303 along the boundary between the inspection area and the non-inspection area, it is possible to determine that the inspection object in FIG. 3(c) has an uneven defect. That is, in the inspection object in FIG. 3(a), the inclination of the slope of the uneven shape 303 is uniform along the edge of the container 301. On the other hand, in the inspection object in FIG. 3(c), the inclination of the slope of the uneven shape 303 is non-uniform along the edge of the container 301. Therefore, if the inspection object in FIG. 3(a) is set as the reference object, it is possible to determine that the inspection object in FIG. 3(c) has an uneven defect.

[0033] However, in the technology described in Patent Document 1, if the inclination of the slope along the edge of the container 301 varies from one individual to another, as in the example shown in Figures 3(a) and 3(b), the product will be judged as defective even if it is a non-defective product. In other words, if the inspection object in Figure 3(a) is set as the reference object, the inspection object in Figure 3(b) will be judged as defective because the inclination of the slope of the uneven shape 303 differs from that of the inspection object in Figure 3(a).

[0034] Therefore, in this embodiment, in visual inspection of a surface having an uneven shape along the boundary between an inspection area and a non-inspection area, the objective is to detect uneven defects while allowing the inclination of the slope along the boundary to vary from one individual to another, as shown in Figures 3(a) and 3(b).

[0035] Note that the object of inspection, that is, the object for determining the presence or absence of unevenness defects, is not limited to the uneven shape 303 along the edge of the container 301. For example, uneven shapes such as patterns and letters may be formed on the surface 302 by embossing or debossing. In embossing, a convex shape is formed on the surface. In debossing, a concave shape is formed on the surface. In this case, the uneven shapes such as patterns and letters are composed of a combination of straight lines and curves, so they are uneven shapes along the boundary between the inspection area and the non-inspection area. Even in such uneven shapes along the boundary between the inspection area and the non-inspection area formed by embossing or debossing, there is a need to detect unevenness defects while allowing the inclination of the slope along the edge of the container 301 to vary depending on the individual. Therefore, in this embodiment, the convex shape formed by embossing or the concave shape formed by debossing is also the object of inspection, that is, the object for determining the presence or absence of unevenness defects.

[0036] <System operation example> Next, the process performed by the system according to this embodiment to determine the presence or absence of uneven defects will be described with reference to the flowchart in Fig. 4. In step S401, the imaging control unit 1011 controls the imaging unit 1012 and the illumination unit 1013 as described above. As a result, the imaging device 101 sequentially emits light from each light source to capture images of the inspection surface of the inspection object 204, and outputs a plurality of captured images obtained by the imaging to the image processing device 102.

[0037] The inspection surface of the inspection object 204 is the surface of the inspection object 204, and has an uneven shape along the boundary between the inspection area and the non-inspection area. In the example of Fig. 3, the inspection surface of the inspection object 204 corresponds to the uneven shape 303 and the surface 302. At this time, the captured image acquired by the imaging device 101 is an image like the top view shown in the lower part of Fig. 3(a) to (c).

[0038] Next, in step S402, the image processing unit 1021 generates (calculates) a feature map, which is a map relating to unevenness, from a plurality of captured images acquired from the imaging device 101. The feature map is a map such as a normal map showing the distribution of normal vectors at each position on the inspection surface of the inspection object 204, a height map showing the distribution of heights at each position on the inspection surface, and a brightness map showing the distribution of brightness at each position on the inspection surface.

[0039] In this embodiment, the image processing unit 1021 generates a normal map (normal image) representing a normal vector at each position on the inspection surface of the inspection object 204 as a feature map of the inspection surface of the inspection object 204. At each pixel position of the normal image, component data of the normal vector at a position in real space corresponding to the pixel position is stored. The component data is three-channel data, with the x-component of the normal vector stored in the first channel, the y-component of the normal vector stored in the second channel, and the z-component of the normal vector stored in the third channel.

[0040] The image processing unit 1021 generates a normal image from a plurality of captured images, for example, by using a photometric stereo method. For example, the image processing unit 1021 generates a normal image by using a method disclosed in JP 2016-186421 A. Note that the method for generating the normal image is not limited to a specific method, and for example, the image processing unit 1021 may acquire shape information of the inspection surface by a light section method and generate a normal image based on the acquired shape information.

[0041] Next, in step S403, the image processing unit 1021 specifies an inspection area from the normal image. In the example of Fig. 3, the image processing unit 1021 specifies an area in the normal image corresponding to the inspection surface (surface 302 and uneven shape 303) as the inspection area. The method for specifying the inspection area in the normal image is not limited to a specific method.

[0042] For example, the image processing unit 1021 first specifies a region corresponding to the inspection surface (surface 302 and uneven shape 303) in the captured image as the target region. Various methods are possible for specifying the target region in the captured image. For example, the image processing unit 1021 specifies a region having the hue, saturation, and brightness of the contents as the target region in a captured image in which the inspection object 204 is captured from near the zenith. Then, the image processing unit 1021 specifies a corresponding region in the normal image that corresponds to the target region as the inspection region. In addition, the image processing unit 1021 may specify the region of the contents using semantic segmentation using deep learning, or may detect the boundary between the contents and the container as an edge by image processing.

[0043] Furthermore, the image processing unit 1021 specifies the remaining area other than the inspection area in the normal image as a non-inspection area.

[0044] Next, in step S404, the detection unit 1022 sets an area (boundary portion) having a specified width (width of one pixel or more) along the boundary between the inspection area and the non-inspection area in the normal image as the first inspection area. For example, the detection unit 1022 identifies a difference area between a reduced target area obtained by reducing a target area in an image obtained by capturing an image of the inspection object 204 from near the zenith at a specified reduction ratio and the target area, and sets a corresponding area in the normal image corresponding to the difference area as the first inspection area. The detection unit 1022 may set a difference area between a reduced inspection area obtained by reducing an inspection area in the normal image at a specified reduction ratio and the inspection area as the first inspection area. By setting the above-mentioned "specified width" to "one pixel", it is possible to set a group of pixels on the outermost shell in the inspection area as the first inspection area. Then, the detection unit 1022 aligns the group of pixels in the first inspection area according to the extracted shape. The detection unit 1022 can also extract an area (pixel group) having a specified width by the filter size in the reduction process. In this case, the closer to the inside of the inspection area, the smaller the inspection area becomes, so it is necessary to resize it. The method of setting the first inspection area is not limited to a specific setting method.

[0045] In step S405 described later, the detection unit 1022 determines whether or not there is a concave-convex defect in the concave-convex shape based on the inclination of the slope of the concave-convex shape. In the example of FIG. 3, the detection unit 1022 determines whether or not there is a concave-convex defect in the concave-convex shape 303. Hereinafter, a case will be described in which a linear region 306 along the edge 305 of the container is set as the first inspection region. In this case, the detection unit 1022 detects the edge 305 of the container by applying a known edge detection process to a captured image of the inspection object 204 captured from near the zenith, and sets the linear region 306 located a certain distance inward from the edge 305 of the container as the first inspection region. The linear region 306 is included in the slope of the concave-convex shape 303. In step S405 described later, the detection unit 1022 determines whether or not there is a concave-convex defect in the concave-convex shape based on the inclination of the slope in the linear region 306. Specifically, the detection unit 1022 determines that there is no defect if the slope of the slope of the uneven shape is uniform in the first inspection area (linear area 306), and determines that there is a defect if the slope of the slope of the uneven shape is non-uniform in the first inspection area (linear area 306).

[0046] If the position of the inspection object 204 does not change or if the position of the inspection object has been specified, the detection unit 1022 may set a predefined area as the first inspection area without performing edge detection. The first inspection area is not limited to a linear area, and may be set as a band-shaped area. In addition, when detecting unevenness defects in a linear convex shape formed by embossing or when detecting unevenness defects in a linear concave shape formed by debossing, an area along the boundary between the inspection area and the non-inspection area is set as the first inspection area by the same method.

[0047] In step S404, the detection unit 1022 further acquires component data of each pixel constituting the first inspection area in the normal image. As shown in the top view in the lower part of FIG. 3(a), the x-axis and y-axis are set on the inspection surface of the inspection object 204, the inspection surface is set as an xy plane, and the z-axis is set in a direction perpendicular to the inspection surface. The inspection surface of the inspection object 204 is a surface based on the inspection object 204, and may be, for example, a surface on which the inspection object 204 is placed or may be the surface 302. At this time, each position in the linear area 306 can be expressed by an angle θ with respect to the x-axis. Therefore, when the component data at the position (x, y) is n(x, y), the component data can be expressed as n(θ) with the angle θ as an argument.

[0048] Next, in step S405, the detection unit 1022 determines the uniformity of the slope in the first inspection area based on each component data acquired in step S404, and determines the presence or absence of a concavo-convex defect in the concavo-convex shape based on the result of the determination.

[0049] The process for determining the presence or absence of unevenness defects in the uneven shape based on the result of the determination of the uniformity of the slope in the first inspection area will be described with reference to Fig. 5. In Fig. 5, the dotted arrows indicate normal vectors n represented by the component data n(θ) of each of the three pixels in the first inspection area. The length of the normal vector n is normalized to 1.

[0050] In this embodiment, the detection unit 1022 judges the presence or absence of a concave-convex defect in the concave-convex shape based on the outlier of the normal vector. Specifically, the detection unit 1022 judges the presence or absence of a concave-convex defect in the concave-convex shape based on the presence or absence of an outlier of the direction of the normal vector. For example, the angle φ of the normal vector n with respect to the xy plane is used as the direction of the normal vector. In FIG. 5, nxy is a projection vector obtained by projecting the normal vector n onto the xy plane. The angle φ is the angle of the normal vector n with respect to the vector nxy. This angle φ can be calculated from the normal vector n and is a function of the position θ in the first inspection area, so it is expressed as φ(θ). This function φ(θ) is a function indicating the inclination (angle of inclination) of the slope at each position θ in the first inspection area. As shown in FIG. 5(b) and FIG. 5(c), if the inclination of the slope is uniform according to the change in the position θ, the value of the function φ(θ) is constant with respect to the position θ. Note that Fig. 5(b) corresponds to Fig. 3(a), and Fig. 5(c) corresponds to Fig. 3(b). On the other hand, as shown in Fig. 5(a), when the inclination of the slope is not uniform (uneven) depending on the position θ, the value of the function φ(θ) is not constant with respect to the position θ, but fluctuates. Note that Fig. 5(a) corresponds to Fig. 3(c). Therefore, the detection unit 1022 detects outliers of the function φ(θ) in the first inspection area to determine the presence or absence of unevenness defects in the uneven shape.

[0051] For example, the detection unit 1022 removes the DC component of the value of the function φ(θ) by calculating the average value of the value of the function φ(θ) at each position θ and subtracting the average value from the value of the function φ(θ) at each position θ. Note that, in order to remove the DC component, the detection unit 1022 may apply a one-dimensional spatial filter (e.g., a DoG filter) whose coefficient sum is zero to the value of the function φ(θ) at each position θ.

[0052] Then, the detection unit 1022 compares the value at each position θ of the function φ(θ) after the DC component is removed with the threshold value. In this case, a positive threshold value and a negative threshold value are set as the threshold value. Then, the detection unit 1022 determines that there is an outlier when there is a value that exceeds the positive threshold value among the values ​​at each position θ of the function φ(θ) after the DC component is removed. Similarly, the detection unit 1022 determines that there is an outlier when there is a value that falls below the negative threshold value among the values ​​at each position θ of the function φ(θ) after the DC component is removed. Note that the detection unit 1022 determines that there is no outlier when there is neither a value that exceeds the positive threshold value nor a value that falls below the negative threshold value among the values ​​at each position θ of the function φ(θ) after the DC component is removed. Note that various methods can be applied to the method for determining the presence or absence of an outlier for the value at each position θ of the function φ(θ) after the DC component is removed, and is not limited to the above method.

[0053] If the detection unit 1022 determines that there is an outlier, it determines that there is a unevenness defect in the uneven shape. If the detection unit 1022 determines that there is no outlier, it determines that there is no unevenness defect in the uneven shape.

[0054] Then, the detection unit 1022 outputs the result of the judgment. There are various output methods for the result of the judgment, and the output method is not limited to a specific output method. For example, the detection unit 1022 may display a message expressing the presence or absence of a concave-convex defect in the concave-convex shape by an image and / or text on the display 209, or may transmit such a message to an external device via a network. In addition, for example, when an outlier is present, the detection unit 1022 may superimpose a marker or the like at a position corresponding to the position θ at which the outlier is present in an image captured by capturing the inspection object 204 from near the zenith. In this way, as long as the presence or absence of a concave-convex defect in the concave-convex shape can be notified to a user, the method is not limited to a specific method.

[0055] Furthermore, in this embodiment, if any one of the values ​​at each position θ of the function φ(θ) after removal of the DC component exceeds a threshold on the positive side or falls below a threshold on the negative side, it is determined that "there is a defect in the uneven shape". However, if the number of values ​​at each position θ of the function φ(θ) after removal of the DC component that exceeds a threshold on the positive side or falls below a threshold on the negative side is equal to or greater than a specified number, it may be determined that "there is a defect in the uneven shape", and if the number is less than the specified number, it may be determined that "there is no defect in the uneven shape".

[0056] <Advantages of the First Embodiment> In this manner, in this embodiment, if the function φ(θ) indicating the inclination of the slope in the uneven shape is uniform in the first inspection region (linear region 306), it is determined that there is no defect. Therefore, even if the inclination of the slope along the first inspection region differs from one individual to another as shown in FIG. 3(a) and FIG. 3(b), it can be determined that there is no defect in either case. On the other hand, in this embodiment, if the value of the function φ(θ) indicating the inclination of the slope in the uneven shape is non-uniform in the first inspection region (linear region 306), it is determined that there is a defect. Therefore, if the inclination of the slope changes depending on the position θ as shown in FIG. 3(c), it can be determined that there is a defect.

[0057] In other words, according to this embodiment, in visual inspection of a surface having an uneven shape along the boundary between an inspection area and a non-inspection area, it is possible to detect uneven defects in the area along the boundary while allowing the inclination of the slope along the boundary to vary from one individual to another.

[0058] Below, several modified examples of the first embodiment will be described. In these modified examples, the differences from the first embodiment will be described, and unless otherwise specified, the modified examples will be considered to be similar to the first embodiment.

[0059] <Variation 1> As can be seen from FIG. 5, the angle φ indicating the inclination of the slope of the uneven shape 303 is cos -1(|nxy| / |n|). nxy is the projection vector of the normal vector n projected onto the inspection surface (xy plane). |nxy| indicates the length of nxy. |n| indicates the length of the normal vector n. As mentioned above, the length of the normal vector n is normalized to 1. Therefore, the angle φ is expressed as cos -1 (|nxy|). That is, the angle φ is uniquely determined by |nxy|. More specifically, the angle φ decreases monotonically with an increase in |nxy|. Therefore, when determining whether or not there is a concave-convex defect in the concave-convex shape 303, instead of detecting an outlier of the angle φ, an outlier of the length |nxy| of the projection vector obtained by projecting the normal vector n onto the inspection surface (xy plane) may be detected. In that case, as shown in FIG. 5(b) and FIG. 5(c), if the inclination of the slope is uniform depending on the position θ, the length |nxy| becomes uniform regardless of the location, and no outlier is detected, so that it can be determined that there is no defect. On the other hand, as shown in FIG. 5(a), if the inclination of the slope is non-uniform depending on the position θ, the length |nxy| becomes non-uniform depending on the location, and an outlier is detected, so that it can be determined that there is a defect.

[0060] <Variation 2> Figures 6(a) to (c) correspond to Figures 5(a) to (c), respectively. In Figure 6, nz indicates a projection vector obtained by projecting the normal vector n onto the direction perpendicular to the inspection surface (z-axis). In Figure 6, the normal vector nz is indicated by a solid arrow. Meanwhile, the normal vector n and the projection vector nxy obtained by projecting the normal vector n onto the inspection surface (xy plane) are indicated by dotted arrows. As shown in Figure 6, the angle φ indicating the inclination of the slope of the uneven shape 303 is expressed as sin -1 (|nz| / |n|). |nz| indicates the length of the normal vector nz and is the value of the z component of the normal vector n. As mentioned above, the normal vector n is normalized to a length of 1. Therefore, the angle φ is sin -1(|nz|). That is, the angle φ is uniquely determined by |nz|. More specifically, the angle φ increases monotonically as |nz| increases. Therefore, when determining whether or not there is a concave-convex defect in the concave-convex shape 303, instead of detecting outliers in the angle φ, it is also possible to detect outliers in the length |nz| of the projection vector obtained by projecting the normal vector n in the direction perpendicular to the inspection surface (z-axis). In that case, as shown in FIG. 6(b) and FIG. 6(c), if the inclination of the slope is uniform depending on the position θ, |nz| becomes uniform regardless of the position θ, and no outlier is detected, so that it can be determined that there is no defect. On the other hand, as shown in FIG. 6(a), if the inclination of the slope is non-uniform depending on the position θ, |nz| becomes non-uniform depending on the position θ, and an outlier is detected, so that it can be determined that there is a defect.

[0061] <Modification 3> In this modification, a luminance map is used as the feature map. The luminance map is an image showing the distribution of luminance on the inspection surface of the inspection object 204. Since the distribution of luminance on the inspection surface reflects the distribution of normal vectors on the inspection surface, the presence or absence of unevenness defects can be determined by detecting outliers of pixel values ​​in the luminance map instead of detecting outliers of normal vectors.

[0062] The process performed by the system according to this modified example to determine the presence or absence of unevenness defects will be described with reference to the flowchart in Fig. 7. In Fig. 7, the same process steps as in Fig. 4 are assigned the same step numbers, and the description of these process steps will be omitted.

[0063] In step S702, the image processing unit 1021 acquires a luminance map as a feature map from a plurality of captured images acquired from the imaging device 101. In this embodiment, the image processing unit 1021 acquires a reflected image as a luminance map. The reflected image is a maximum value image generated by extracting the maximum pixel value for each pixel position from a plurality of captured images obtained by sequentially emitting light from light sources at different positions. When the captured image group is an RGB image, for example, a reflected image is generated using a pixel value of the G channel. That is, the pixel value at a pixel position (x, y) in the reflected image is the maximum pixel value among the pixel values ​​of the G channel at each pixel position (x, y) of the plurality of captured images.

[0064] The image processing unit 1021 may generate (calculate) an average value image as the luminance map. The average value image is an image generated by calculating the average value of pixel values ​​for each pixel position from a plurality of captured images obtained by sequentially emitting light from a light source at different positions. In other words, the pixel value at a pixel position (x, y) in the average value image is the average value of the pixel values ​​at each pixel position (x, y) of the plurality of captured images.

[0065] Furthermore, the image processing unit 1021 may select (obtain) as the luminance map an image that contains the most specularly reflected light from the light source from among a plurality of captured images obtained by sequentially emitting light from light sources at different positions. For example, as shown in FIG. 2(b), when the imaging unit 1012 is installed at the apex of a dome-shaped multi-lamp lighting, an image captured by turning on the light source located closest to the imaging unit 1012 contains the most specularly reflected light from the light source. Therefore, in this case, the image processing unit 1021 selects as the luminance map an image captured by turning on the light source located closest to the imaging unit 1012.

[0066] In step S403 according to this embodiment, the image processing unit 1021 identifies an inspection area from the reflected image, for example, using a method similar to that of the first embodiment, and identifies the remaining area in the reflected image other than the inspection area as a non-inspection area.

[0067] In step S704, the detection unit 1022 sets a first inspection area in the reflected image in the same manner as in step S404 described above, and obtains pixel values ​​(brightness values) of each pixel constituting the first inspection area in the reflected image. When the pixel value at a pixel position (x, y) in the reflected image is i(x, y), the pixel value can be expressed as i(θ) using the angle θ as an argument.

[0068] In step S705, the detection unit 1022 determines whether or not there is an outlier for the pixel values ​​acquired in step S704. For example, the detection unit 1022 calculates the average value of the pixel values ​​acquired in step S704. The detection unit 1022 then determines that there is an outlier if there is any pixel value among the pixel values ​​acquired in step S704 that is deviated from the average value by a specified value or more. On the other hand, the detection unit 1022 determines that there is no outlier if there is no pixel value among the pixel values ​​acquired in step S704 that is deviated from the average value by a specified value or more.

[0069] If the detection unit 1022 determines that there is an outlier, it determines that there is a unevenness defect in the uneven shape. If the detection unit 1022 determines that there is no outlier, it determines that there is no unevenness defect in the uneven shape.

[0070] As described above, even when outliers in a brightness map are used instead of outliers in a normal image, uneven defects in the area along the first inspection area can be detected while allowing the inclination of the slope along the first inspection area to vary from one individual to another.

[0071] Figure 8 shows an example of the results of calculating outliers for defective products. When a value above a certain threshold is displayed in the reflected image, an area 801 where a concave-convex defect occurs remains. Since the coordinate value of the remaining area 801 with the regular reflection component is the outlier itself, it is possible to determine whether the product is good or bad by finding the sum of the above area and checking whether this sum exceeds the threshold. Through the above process, the location and size of the concave-convex defect can be identified.

[0072] <Variation 4> In this modification, in addition to inspecting the uneven shape along the first inspection area, other areas on the inspection surface are inspected. As described above, in the uneven shape along the boundary between the inspection area and the non-inspection area (e.g., uneven shape 303), there is a need to allow the inclination of the slope along the first inspection area to vary from one object to another. On the other hand, in other areas on the inspection surface (e.g., surface 302), there is a need to not allow the variation in shape or slope from one object to another, and to detect unevenness outside the reference range as an unevenness defect. In this modification, the presence or absence of an unevenness defect is determined using a different method for each area. This modification differs from the first embodiment in that the following processes are performed in steps S404 and S405 in the flowchart of FIG. 4.

[0073] In step S404, the detection unit 1022 sets a first inspection area for the normal image in the same manner as in the first embodiment, and sets an area in the normal image that is not included in the first inspection area (for example, the surface 302) as a second inspection area.

[0074] In step S405, the detection unit 1022 judges the presence or absence of unevenness defects in the first inspection area, as in the first embodiment. Furthermore, the detection unit 1022 judges the uniformity of the inclination in the second inspection area, and judges the presence or absence of unevenness defects in the second inspection area based on the result of the judgment. Specifically, the detection unit 1022 judges that there is no defect if the inclination of the slope in the second inspection area is within a reference range, and judges that there is a defect if the inclination of the slope in the second inspection area is not within the reference range. For this judgment method, for example, the method described in JP 2003-240539 A is used. Specifically, the detection unit 1022 obtains an angle difference between the normal vector of the inspection object and the normal vector of the reference object in the second inspection area. Then, if the sum of the angle differences is equal to or greater than a threshold, the detection unit 1022 judges that "there is an unevenness defect in the second inspection area", and if the sum of the angle differences is less than the threshold, the detection unit 1022 judges that "there is no unevenness defect in the second inspection area".

[0075] As a result, in the second inspection area, individual variations in shape or slope are not tolerated, and unevenness outside the standard range can be detected as unevenness defects. In this way, according to the fourth modification, unevenness inspection can be performed from different perspectives for each area.

[0076] [Second embodiment] In this embodiment, the difference from the first embodiment will be described, and unless otherwise specified, it is assumed that the present embodiment is the same as the first embodiment. In this embodiment, a reference object for comparison with an inspection object is used to detect uneven defects. In this embodiment, uneven defects in an area along the first inspection area are also detected while allowing the inclination of the slope along the first inspection area to vary from individual to individual. In other words, when determining the reference object, it is necessary to incorporate a mechanism that does not reflect differences in individual differences between non-defective products.

[0077] The process performed by the system according to this embodiment to determine the presence or absence of a concave-convex defect will be described with reference to the flowchart of FIG. 9. In this embodiment, the presence or absence of a concave-convex defect is determined based on a first projection vector nxy obtained by projecting a normal vector n of an object to be inspected onto an xy plane, and a second projection vector n'xy obtained by projecting a normal vector n' of a reference object onto an xy plane. Here, the reason for using the length of the projection vector projected onto the xy plane is to allow for differences in the inclination of the slope of the concave-convex shape. By ignoring the z component of the normal vector, for example, when the inclination of the slope of the concave-convex shape is the same for adjacent normal vectors, the lengths of the normal vectors projected onto the xy plane will be the same, and no difference will occur.

[0078] FIG. 10(a) is a top view of the reference object, and shows a projection vector n'xy1001 obtained by projecting a normal vector n' at the edge of the container of the reference object onto the xy plane. Assuming that the object used as the reference object is a non-defective product, all of the projection vectors n'xy1001 point toward the center of the container. This is because the edge shape is not wavy. In this example, a circular container is used, so all of the projection vectors intersect at the center. If the container is rectangular, when considering the projection vectors toward the center that are perpendicular to each side of the container, they intersect like a lattice. As described above, the direction perpendicular to the container edge is determined regardless of the shape of the container, so the shape of the container is not limited.

[0079] FIG. 10(b) is a top view of the inspection object, and shows the projection vector nxy 1002 obtained by projecting the normal vector n at the edge of the container onto the xy plane. If the object used as the inspection object is a non-defective product, the projection vectors nxy are perpendicular to the tangent of the boundary with the non-inspection area, so all of them point toward the center of the container. On the other hand, if the inspection object is a defective product, the projection vectors nxy will point in various directions. FIG. 10(b) shows the case where the inspection object is a defective product as an example. The direction of the projection vector is determined due to the wavy shape of the edge.

[0080] In this embodiment, the image processing device 102 calculates an angle 1003 between a projection vector n'xy obtained by projecting a normal vector n' at the container edge of the reference object onto the xy plane and a projection vector nxy obtained by projecting a normal vector n of the edge of the inspection object onto the xy plane. It can be said that the angle 1003 is the difference between the direction in which the projection vector obtained by projecting the normal vector of the inspection object onto the xy plane should be directed. Therefore, before inspecting the lot, it is necessary to calculate the projection vector n'xy of the reference object and hold it as advance data. Therefore, in this embodiment, the image processing device 102 performs the same process as in the first embodiment on the reference object to calculate a projection vector n'xy obtained by projecting a normal vector n' in the first inspection area of ​​the reference object onto the xy plane. The image processing device 102 then stores the projection vector n'xy calculated in advance in a memory device such as the storage device 205.

[0081] The direction of the projection vector n'xy does not depend on the way the fluid poured into the reference object solidifies, and even if the inclination of the slope of the uneven shape varies, it will point in the same direction as long as it is constant. This has the effect of making it possible to detect unevenness defects while allowing for individual variations.

[0082] Steps S901 to S903 are the same processes as steps S401 to S403, respectively, and in step S904, the image processing device 102 acquires a normal vector n in the first inspection region of the object in the same manner as in step S404 described above. Furthermore, in step S904, a projection vector nxy obtained by projecting the normal vector n onto the xy plane is calculated in the same manner as in step S405 described above.

[0083] In step S905, the image processing unit 1021 loads (acquires) into the RAM 202 the projection vector n′xy that has been calculated in advance and stored in a memory device such as the storage device 205.

[0084] In step S906, the image processing unit 1021 calculates an angle 1003 between the projection vector n'xy acquired in step S905 and the projection vector nxy calculated in the processes of steps S901 to S904 for each corresponding position θ.

[0085] In step S907, the detection unit 1022 calculates the sum of the angles 1003 calculated for each position θ in step S906, and compares the sum with a threshold value. If the sum is equal to or greater than the threshold value as a result of this comparison, the detection unit 1022 determines that there is a "non-uniform defect (the inspection object 204 is a defective product)," and if the sum is less than the threshold value, the detection unit 1022 determines that there is no non-uniform defect (the inspection object 204 is a non-defective product). The detection unit 1022 then outputs this determination result. The output form is not limited to a specific output form.

[0086] Note that the inner product value of the projection vector n'xy and the projection vector nxy may be used instead of the angle 1003. In this way, any information may be used instead of the angle 1003 as long as it represents the difference between the projection vector n'xy and the projection vector nxy.

[0087] <Effects of the second embodiment> In this manner, in this embodiment, if a projection vector n'xy obtained by projecting the normal vector n' of the reference object onto the xy plane is obtained in advance, uneven defects on the container edge can be inspected by comparing the normal vector n of the inspection object with the projection vector nxy obtained by projecting the normal vector n onto the xy plane. This makes it possible to detect uneven defects along the edge of the container while allowing for individual variations in the inclination of the slope along the edge.

[0088] <Modification> A modified example of the second embodiment will be described below, but in this modified example, the differences from the second embodiment will be described, and unless otherwise specified, it will be assumed that the modified example is the same as the second embodiment. In this modified example, the coefficient of the spatial filter to be applied to the edge of the inspection object is determined according to the direction of the projection vector n'xy obtained by projecting the normal vector n' of the reference object onto the xy plane. Therefore, as in the second embodiment, it is necessary to obtain in advance the projection vector n'xy obtained by projecting the normal vector n' of the reference object onto the xy plane. Then, the presence or absence of unevenness defects is determined by applying a different spatial filter to each position.

[0089] The process performed by the system according to this embodiment to determine the presence or absence of unevenness defects will be described with reference to the flowchart in Fig. 11. In Fig. 11, the same process steps as those in Fig. 9 are given the same step numbers, and the description of these process steps will be omitted.

[0090] In step S1104, the detection unit 1022 sets a first inspection area for the normal image generated in step S902 in the same manner as in the first embodiment. In step S1106, the image processing unit 1021 selects a "filter coefficient for edge detection" corresponding to the direction of the projection vector n'xy at each position θ (for example, the angle from the x-axis) at that position θ on the first inspection area set in step S1104.

[0091] Filter coefficients of a spatial filter corresponding to various angles from the x-axis are stored in advance in a memory device such as the storage device 205. Therefore, before the process of step S1106 starts, the image processing unit 1021 loads the filter coefficients of the spatial filter corresponding to various angles from the x-axis from a memory device such as the storage device 205 into the RAM 202. Therefore, in step S1106, the image processing unit 1021 selects a filter coefficient corresponding to the direction of the projection vector n'xy at each position θ on the first inspection area set in step S1104 from the "filter coefficients of the spatial filter corresponding to various angles from the x-axis" loaded into the RAM 202.

[0092] This spatial filter is a spatial filter for detecting edges in a direction perpendicular to the edge of the container. This is because if a spatial filter for detecting edges in a direction along the edge of the container is used, it will react to the inclination of the slope of the uneven shape of the container edge itself, and the defect phenomenon that should be detected will be buried in the output result. Therefore, it is necessary to change the direction in which the weight is concentrated depending on the direction of the tangent of the boundary between the inspection area and the non-inspection area. For example, by keeping the weight in the direction perpendicular to the tangent of the boundary approximately constant, it is not affected by the inclination of the slope.

[0093] In step S1107, the detection unit 1022 generates a map in which the component data in the first inspection region in the normal image of the object is converted into a scalar value (for example, the length of the normal vector obtained from each of the x, y, and z components of the normal vector represented by the component data). Then, the detection unit 1022 performs convolution (filter processing) of a spatial filter having the filter coefficient selected in step S1106 for a position θ in the generated map. That is, the convolution of the spatial filter is performed along the first inspection region. In the example of FIG. 3, the detection unit 1022 performs a product-sum operation between an area consisting of a pixel of interest and pixels surrounding the pixel of interest in the linear region 306, and a filter coefficient corresponding to the position θ of the pixel of interest. Then, the detection unit 1022 regards the value obtained by the product-sum operation as the result of the filter processing at the pixel of interest.

[0094] The manner in which a spatial filter is applied is shown in Fig. 12. As shown in Fig. 12, when the angle of the center direction in the coordinates on the reference object is 180° (to the left), a spatial filter that detects edges in the y direction is applied as shown by reference number 1201. When the angle of the center direction in the coordinates on the reference object is 270° (downward), a spatial filter that detects edges in the x direction is applied as shown by reference number 1202.

[0095] Here, the edges are detected using a DoG filter adjusted to fit the uneven shape to be detected. There are no restrictions on the shape or coefficients of the filter as long as uneven defects can be detected. The above process is performed once around the first inspection area at the edge of the inspection object.

[0096] In step S1108, the detection unit 1022 calculates the sum of the values ​​(the values ​​obtained by the above product-sum calculation) of each pixel position in the map obtained by the above filter processing, and compares the magnitude of this sum with a threshold value. If the result of this comparison shows that the sum is equal to or greater than the threshold value, the detection unit 1022 judges that "there is a unevenness defect (the inspection object 204 is a defective product)," and if the sum is less than the threshold value, the detection unit 1022 judges that "there is no unevenness defect (the inspection object 204 is a non-defective product)." The detection unit 1022 then outputs this judgment result. The output form is not limited to a specific output form.

[0097] In this way, according to this modified example, the presence or absence of unevenness defects can be determined by performing convolution processing using a spatial filter determined using a projection vector n'xy obtained by projecting the normal vector n' of the reference object onto the xy plane.

[0098] The numerical values, processing timing, processing order, processing subject, data (information) acquisition method / destination / source / storage location, etc. used in the above embodiments and variant examples are given as examples to provide a concrete explanation, and are not intended to be limited to these examples.

[0099] In addition, any part or all of the embodiments and modifications described above may be used in appropriate combination. In addition, any part or all of the embodiments and modifications described above may be used selectively.

[0100] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0101] The invention of this specification includes the following image processing device, image processing method, and computer program. (Item 1) an acquisition means for acquiring a map relating to unevenness based on a plurality of captured images of the inspection object; a determination means for determining the presence or absence of unevenness defects in a boundary portion between an inspection area and a non-inspection area based on the uniformity of the map in the boundary portion; An image processing device comprising: (Item 2) 2. The image processing device according to item 1, wherein the acquisition means generates a first map representing a distribution of normal vectors from the plurality of captured images. (Item 3) The image processing device according to item 2, characterized in that the determination means determines whether or not there is an outlier in the angle of inclination based on the normal vector of the boundary portion in the first map, and determines whether or not there is a unevenness defect in the boundary portion based on the result of the determination. (Item 4) The image processing device according to item 3, characterized in that the determination means determines whether or not there is an outlier in the length of a projection vector obtained by projecting a normal vector of the boundary portion in the first map onto a plane based on the inspection object, and determines whether or not there is a unevenness defect in the boundary portion based on the result of the determination. (Item 5) The image processing device according to item 3, characterized in that the determination means determines the presence or absence of an outlier in the length of a projection vector obtained by projecting a normal vector of the boundary portion in the first map onto an axis perpendicular to a plane based on the inspection object, and determines the presence or absence of a concavo-convex defect in the boundary portion based on a result of the determination. (Item 6) The acquisition means generates a second map representing a distribution of normal vectors based on a plurality of captured images of the reference object; The image processing device described in item 2, characterized in that the judgment means judges the presence or absence of unevenness defects in the boundary portion based on the difference between a projection vector obtained by projecting a normal vector of the boundary portion in the first map onto a plane based on the inspection object, and a projection vector obtained by projecting a normal vector of the boundary portion in the second map onto a plane based on the inspection object. (Item 7) 2. The image processing device according to item 1, wherein the acquisition means acquires a map representing a distribution of luminance in a plane based on the inspection object from the plurality of captured images. (Item 8) 8. The image processing device according to item 7, wherein the determining means determines whether or not there is an outlier in the brightness of the boundary portion in the map, and determines whether or not there is a concavo-convex defect in the boundary portion based on the result of the determination. (Item 9) 2. The image processing device according to item 1, wherein the determining means determines the presence or absence of unevenness defects in an area of ​​the map that is not included in the boundary portion in the inspection area of ​​the map based on the uniformity of the map in the area. (Item 10) The test object is an industrial product produced by solidifying the contents of a container, The plurality of captured images are images of the industrial product captured from different positions. 10. The image processing device according to any one of items 1 to 9, (Item 11) 11. The image processing device according to item 10, wherein the boundary portion is included in an uneven slope formed along the edge of the container by interfacial tension between the content and the container. (Item 12) 10. The image processing device according to any one of items 1 to 9, wherein the boundary portion is an area where a slope occurs in a convex shape formed by embossing or a concave shape formed by debossing. (Item 13) 13. The image processing device according to any one of items 1 to 12, wherein the determining means outputs a result of determining whether or not there is a concavo-convex defect in the boundary portion. (Item 14) an acquisition means for acquiring a first map representing a distribution of normal vectors based on a plurality of captured images of the object to be inspected, and acquiring a second map representing a distribution of normal vectors based on a plurality of captured images of a reference object; a determination means for obtaining a filter coefficient according to a direction of a projection vector obtained by projecting a normal vector of a boundary portion between an inspection area and a non-inspection area in the second map onto a plane based on the object to be inspected, and determining the presence or absence of a concavo-convex defect in the boundary portion of the first map based on the filter coefficient; An image processing device comprising: (Item 15) Item 15. The image processing device according to item 14, wherein the determination means generates a map of scalar values ​​from the first map, and determines the presence or absence of unevenness defects in the boundary portion of the first map based on a result of a filter process using the filter coefficient on the boundary portion in the map. (Item 16) An image processing method performed by an image processing device, comprising: an acquisition step in which an acquisition means of the image processing device acquires a map relating to unevenness based on a plurality of captured images of the inspection object; a determining step in which a determining means of the image processing device determines the presence or absence of unevenness defects in a boundary portion between an inspection area and a non-inspection area based on the uniformity of the map in the boundary portion; An image processing method comprising: (Item 17) An image processing method performed by an image processing device, comprising: an acquisition step in which an acquisition means of the image processing device acquires a first map representing a distribution of normal vectors based on a plurality of captured images of an object to be inspected, and acquires a second map representing a distribution of normal vectors based on a plurality of captured images of a reference object; a determination step in which a determination means of the image processing device acquires a filter coefficient according to a direction of a projection vector obtained by projecting a normal vector of a boundary portion between an inspection area and a non-inspection area in the second map onto a plane based on the inspection object, and determines the presence or absence of a concavo-convex defect in the boundary portion of the first map based on the filter coefficient; An image processing method comprising: (Item 18) A computer program for causing a computer to function as each of the means of the image processing device according to any one of items 1 to 15.

[0102] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0103] 101: Imaging device 102: Image processing device 1011: Imaging control unit 1012: Imaging unit 1013: Illumination unit 1021: Image processing unit 1022: Detection unit

Claims

1. an acquisition means for acquiring a map relating to unevenness based on a plurality of captured images of the inspection object; a determination means for determining the presence or absence of unevenness defects in a boundary portion between an inspection area and a non-inspection area based on the uniformity of the map in the boundary portion; An image processing device comprising:

2. The image processing apparatus according to claim 1 , wherein the acquisition means generates a first map representing a distribution of normal vectors from the plurality of captured images.

3. 3. The image processing device according to claim 2, wherein the determination means determines whether or not there is an outlier in the angle of inclination based on the normal vector of the boundary portion in the first map, and determines whether or not there is a unevenness defect in the boundary portion based on the result of the determination.

4. 4. The image processing device according to claim 3, wherein the determination means determines whether or not there is an outlier in the length of a projection vector obtained by projecting a normal vector of the boundary portion in the first map onto a plane based on the inspection object, and determines whether or not there is a unevenness defect in the boundary portion based on a result of the determination.

5. 4. The image processing device according to claim 3, wherein the determination means determines whether or not there is an outlier in the length of a projection vector obtained by projecting a normal vector of the boundary portion in the first map onto an axis perpendicular to a plane based on the inspection object, and determines whether or not there is a unevenness defect in the boundary portion based on a result of the determination.

6. The acquisition means generates a second map representing a distribution of normal vectors based on a plurality of captured images of the reference object; 3. The image processing device according to claim 2, wherein the determination means determines the presence or absence of unevenness defects in the boundary portion based on a difference between a projection vector obtained by projecting a normal vector of the boundary portion in the first map onto a plane based on the inspection object, and a projection vector obtained by projecting a normal vector of the boundary portion in the second map onto a plane based on the inspection object.

7. The image processing apparatus according to claim 1 , wherein the acquisition means acquires a map representing a distribution of luminance on a plane based on the object from the plurality of captured images.

8. 8. The image processing apparatus according to claim 7, wherein the determining means determines whether or not there is an outlier in brightness of the boundary portion in the map, and determines whether or not there is a concave-convex defect in the boundary portion based on the result of the determination.

9. 2 . The image processing apparatus according to claim 1 , wherein the determining means determines the presence or absence of unevenness defects in an area not included in the boundary portion in the inspection area of ​​the map based on the uniformity of the map in that area.

10. The test object is an industrial product produced by solidifying the contents of a container, The plurality of captured images are images of the industrial product captured from different positions.

2. The image processing device according to claim 1,

11. The image processing device according to claim 10 , wherein the boundary portion is included in an uneven slope formed along the edge of the container by interfacial tension between the content and the container.

12. 2. The image processing device according to claim 1, wherein the boundary portion is a region where a slope occurs in a convex shape formed by embossing or a concave shape formed by debossing.

13. 2. The image processing apparatus according to claim 1, wherein the determining means outputs a result of the determination of the presence or absence of a concave-convex defect in the boundary portion.

14. an acquisition means for acquiring a first map representing a distribution of normal vectors based on a plurality of captured images of the object to be inspected, and acquiring a second map representing a distribution of normal vectors based on a plurality of captured images of a reference object; a determination means for obtaining a filter coefficient according to a direction of a projection vector obtained by projecting a normal vector of a boundary portion between an inspection area and a non-inspection area in the second map onto a plane based on the object to be inspected, and determining the presence or absence of a concavo-convex defect in the boundary portion of the first map based on the filter coefficient; An image processing device comprising:

15. 15. The image processing device according to claim 14, wherein the determination means generates a map of scalar values ​​from the first map, and determines the presence or absence of unevenness defects in the boundary portion of the first map based on a result of filtering the boundary portion in the map using the filter coefficients.

16. An image processing method performed by an image processing device, comprising: an acquisition step in which an acquisition means of the image processing device acquires a map relating to unevenness based on a plurality of captured images of the inspection object; a determining step in which a determining means of the image processing device determines the presence or absence of unevenness defects in a boundary portion between an inspection area and a non-inspection area based on the uniformity of the map in the boundary portion; An image processing method comprising:

17. An image processing method performed by an image processing device, comprising: an acquisition step in which an acquisition means of the image processing device acquires a first map representing a distribution of normal vectors based on a plurality of captured images of an object to be inspected, and acquires a second map representing a distribution of normal vectors based on a plurality of captured images of a reference object; a determination step in which a determination means of the image processing device acquires a filter coefficient corresponding to a direction of a projection vector obtained by projecting a normal vector of a boundary portion between an inspection area and a non-inspection area in the second map onto a plane based on the inspection object, and determines the presence or absence of a concavo-convex defect in the boundary portion of the first map based on the filter coefficient; An image processing method comprising:

18. A computer program for causing a computer to function as each of the means of the image processing apparatus according to any one of claims 1 to 15.

Images