Surface inspection device and program

Abstract

To make it possible to confirm whether a portion contributing to calculation of a numerical value representing a surface quality matches a portion focused by an operator, which is different from a case where a relation between the numerical value representing the surface quality and the portion contributing to the calculation is unclear.SOLUTION: A surface inspection apparatus comprises: an imaging device for imaging a surface of an object to be inspected; and a processor for processing an image imaged by the imaging device and calculating a numerical value representing a surface quality. The processor allows the image including an index for identifying a position of a portion contributing to calculation of the numerical value and the numerical value to be displayed on a display device.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] The present invention relates to a surface inspection device and a program. [Background technology]

[0002] Today, various products use parts molded from synthetic resins (hereinafter referred to as "molded products"). However, visually observable defects can appear on the surface of molded products. These defects include "sink marks," which are unintentionally formed depressions, and "welds," which form where molten resin meets. Even in the case of a textured finish, which intentionally creates unevenness on the surface, differences from the expected texture can appear. Texture changes depending on a combination of factors such as color, gloss, and unevenness. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-264882 Summary of the Invention [Problem to be solved by the invention]

[0004] Some devices that inspect the surface quality of objects display the quality of the inspected part as a single number. Unlike sensory inspections, quantified numbers are highly objective. However, simply displaying a number does not reveal the main parts that contributed to the calculation of the number. As a result, workers cannot confirm whether the part they are focusing on is being used to calculate the number, or whether parts they did not expect are being used to calculate the number.

[0005] Unlike the case where the relationship between the numerical value representing the surface quality and the part that contributed to the calculation is unknown, the present invention aims to make it possible to confirm whether the part that contributed to the calculation of the numerical value representing the surface quality matches the part that the worker is focusing on. [Means for solving the problem]

[0006] The invention described in claim 1 includes an imaging device for imaging the surface of an object to be inspected, and processing the image captured by the imaging device. , which corresponds to the entire inspection range of the captured image. and a processor that calculates a numerical value representing the quality of the surface, wherein the processor displays the image including an index that identifies the position of the part that contributed to the calculation of the numerical value and the numerical value on a display device, and when there are multiple candidates for the numerical value, the processor displays the index that identifies the position of the part that corresponds to the largest numerical value. Claim 2 The invention described in claim 1 is characterized in that the index for specifying the position indicates a boundary between the part and other parts. 1 2. A surface inspection apparatus according to claim 1. Claim 3 The invention described in claim 1 is characterized in that the indicator for specifying the position is a frame line surrounding the periphery of the part. 2 2. A surface inspection apparatus according to claim 1. Claim 4 In the invention described in claim 1, when there are a plurality of candidates for the numerical value, the processor displays the indicator that identifies the position of the portion corresponding to each candidate. 1 2. A surface inspection apparatus according to claim 1. Claim 5 In the invention described in claim 2, when a plurality of the indices are displayed, the processor associates the order of magnitude of the corresponding numerical values ​​with the display format of the indices. 4 2. A surface inspection apparatus according to claim 1. Claim 6 In the invention described in claim 1, when a plurality of the indicators are displayed, the processor switches the display of the numerical values ​​and the indicators in descending order of the magnitude of the corresponding numerical values. 4 2. A surface inspection apparatus according to claim 1. Claim 7 The invention described in claims 1 to 5 is characterized in that the processor displays the numerical values ​​alongside the corresponding indicators. 6 1. A surface inspection device according to any one of claims 1 to 9. Claim 8The invention described in claim 1 to claim 2 further comprises the processor notifying a user when a luminance difference in the image corresponding to the portion is greater than a predetermined threshold. 7 1. A surface inspection device according to any one of claims 1 to 9. Claim 9 The invention described in claims 1 to 5 is characterized in that, when the inspection starts, the processor displays the numerical value and the index on the image. 7 1. A surface inspection device according to any one of claims 1 to 9. Claim 10 In the invention described in claim 1 to claim 2, when the processor receives an operation to instruct the start of the examination, the processor displays the numerical value and the index on the image. 7 1. A surface inspection device according to any one of claims 1 to 9. Claim 11 The invention described in claims 1 to 5 is characterized in that the device body is portable. 10 1. A surface inspection device according to any one of claims 1 to 9. Claim 12 The invention described in the item (1) is a computer that processes an image of the surface of an object to be inspected taken by an imaging device, Corresponding to the entire inspection range of the captured image A program for realizing a function of calculating a numerical value representing the surface quality, a function of displaying the image including an index specifying the position of the portion that contributed to the calculation of the numerical value and the numerical value on a display device, and a function of displaying the index specifying the position of the portion corresponding to the largest numerical value when there are multiple candidates for the numerical value. 。 [Effects of the Invention]

[0007] According to the invention of claim 1, even when there are multiple candidates for the numerical value representing the surface quality, the position of the part corresponding to the numerical value displayed on the screen can be confirmed. Claim 2 According to the described invention, it is easy to check the parts that contributed to the calculation of the numerical values. Claim 3 According to the described invention, it is possible to simultaneously check the content and position of the image of the part that contributed to the calculation of the numerical value. Claim 4According to the described invention, when there are multiple candidates for the numerical value representing the surface quality, the position of the part related to the quality can be confirmed. Claim 5 According to the described invention, it is possible to confirm the relationship between the magnitude of the numerical values ​​that represent the surface quality and the position of the corresponding parts. Claim 6 According to the described invention, it is possible to confirm the position of the part corresponding to the numerical value representing the surface quality. Claim 7 According to the described invention, the relationship between the numerical value representing the surface quality and the corresponding part can be easily confirmed. Claim 8 According to the described invention, the user can be asked to confirm the calculated value and the location of the part that contributed to the calculation. Claim 9 According to the described invention, operability during testing can be improved. Claim 10 According to the described invention, operability during testing can be improved. Claim 11 According to the described invention, the accuracy of surface quality inspection can be improved even if the surface of the object to be inspected and the range imaged by the surface inspection device are indefinite. Claim 12 According to the described invention, even if there are multiple candidates for the numerical value representing the surface quality, it is possible to confirm the position of the part corresponding to the numerical value displayed on the screen. 。 [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of use of a surface inspection device assumed in the first embodiment. [Figure 2] 1A and 1B are diagrams illustrating examples of defects that appear on the surface of an inspection object. (A) shows an example of a sink mark, and (B) shows an example of a weld. [Figure 3] FIG. 2 is a diagram illustrating an example of a hardware configuration of the surface inspecting device used in the first embodiment. [Figure 4]1A and 1B are diagrams illustrating an example of the structure of the optical system of the surface inspection device according to embodiment 1. (A) shows a schematic diagram of the internal structure of the housing of the surface inspection device, and (B) shows the structure of the opening that is pressed against the surface of the inspection target during inspection. [Figure 5] 4 is a flowchart illustrating an example of an inspection operation by the surface inspection device used in the first embodiment. [Figure 6] FIG. 2 is a diagram illustrating an example of an operation screen displayed on a display. [Figure 7] This figure explains the principle of score calculation. (A) shows an example image acquired as an object of inspection, (B) shows an example cross section in the x direction of a dent formed in the y-axis direction, and (C) shows the brightness profile of the image. [Figure 8] FIG. 10 is a diagram illustrating an example of an operation screen on which the main contributing factors of the score are displayed. [Figure 9] FIG. 10 is a diagram illustrating another example of the operation screen on which the main contributing factors of the score are displayed. [Figure 10] 10A and 10B are diagrams illustrating an example of how a frame line is displayed when a plurality of stripe-like patterns are included in an image captured by a camera. [Figure 11] 10A and 10B are diagrams illustrating another example of how a frame line is displayed when a plurality of stripe-like patterns are included in an image captured by a camera. [Figure 12] 10A and 10B are diagrams illustrating another example of how a frame line is displayed when a plurality of stripe-like patterns are included in an image captured by a camera. [Figure 13] 10A and 10B are diagrams illustrating other examples of border display when multiple streak patterns are included, where (A) is a screen displaying a border showing the main factor location of the maximum score, and (B) is a screen displaying a border showing the main factor location of the second largest score. [Figure 14] 10A and 10B are diagrams illustrating examples of displaying scores that represent surface quality. [Figure 15] 10A and 10B are diagrams illustrating other display examples of scores representing surface quality. [Figure 16]10A and 10B are diagrams illustrating an example of additionally displaying the score of a partial area designated by an operator, in which (A) illustrates the designation of a partial area by an operator, and (B) shows an example of the display on the operation screen after the designation. [Figure 17] 10A and 10B are diagrams illustrating other examples of indicators representing key contributing areas used in calculating scores, in which (A) a symbol is displayed at a position that serves as a guide for the outer edge of the target partial area, and (B) an arrow indicates the target partial area. [Figure 18] 10A and 10B are diagrams illustrating other display examples of frame lines indicating partial regions that contributed to the calculation of the score. [Figure 19] 10 is a flowchart illustrating an example of an inspection operation by the surface inspection device used in the second embodiment. [Figure 20] FIG. 10 is a diagram illustrating an example of use of a surface inspection device assumed in a third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. <First Embodiment> <Example of using a surface inspection device> FIG. 1 is a diagram illustrating an example of use of a surface inspection device 1 assumed in the first embodiment. The imaging unit of the surface inspection device 1 used in the first embodiment is a so-called area camera, and the imaging range (hereinafter referred to as the "imaging range") is defined by a surface. Illumination (not shown) is configured to include a component that satisfies the specular reflection condition for the entire imaging range.

[0010] In the case of Fig. 1, the imaging range includes the entire object to be inspected (hereinafter also referred to as "inspection target") 10. However, the imaging range may include only a portion of the inspection target 10 that is of interest. In this embodiment, the inspection target 10 is assumed to be a molded product. In the case of inspection using an area camera, the surface inspection device 1 and the inspection object 10 are performed in a stationary state. In other words, the surface of the inspection object 10 is inspected in a state where the surface inspection device 1 and the inspection object 10 do not move relative to each other.

[0011] 1, inspection object 10 is plate-shaped, but the shape of inspection object 10 is arbitrary. For example, inspection object 10 may have a shape with a curved surface, such as a sphere or cylinder, in addition to a polyhedron. The actual inspection object 10 may have holes, notches, protrusions, steps, and the like. The surface finish of the inspection object 10 may be untreated, mirror-finished, semi-mirror-finished, or textured.

[0012] The surface inspection device 1 inspects the surface of an inspection object 10 for defects and texture. Defects include sink marks and welds. Sink marks are depressions on the surface that occur in thick parts or ribs, while welds are lines that occur where the leading edges of molten resin meet inside the mold. Defects also include scratches and dents caused by collisions. Texture is a visual and tactile impression, and is influenced by the color, gloss, and roughness of an object's surface. Surface roughness includes lines that appear when cutting a mold. These types of lines are different from defects.

[0013] FIG. 2 is a diagram illustrating examples of defects that appear on the surface of the inspection object 10. (A) shows an example of a sink mark, and (B) shows an example of a weld. In FIGS. 2(A) and 2(B), the defect locations are indicated by dashed lines. There are four sink marks in FIG. 2(A). The surface inspection device 1 in this embodiment is not limited to inspection of defects and texture, but is also used to inspect surface stains.

[0014] The surface inspection device 1 generates an image in which defects on the surface of the inspection object 10 are emphasized, and also quantifies and outputs the results of evaluating the texture. The defects here are irregularities or streaks that appear on what should be a flat surface, such as sink marks or welds. Texture is evaluated using a numerical value (hereinafter also referred to as a "score"). The score is an example of a numerical value that represents the surface quality of the inspection object 10.

[0015] The score is calculated using, for example, multivariate analysis, which analyzes features that appear in the brightness distribution, such as a streak pattern that extends along the direction of the sink mark. In addition, there is also a method using artificial intelligence to calculate the score. For example, by feeding an image captured by a camera into a learning model that uses deep machine learning to determine the relationship between the image of the defect and the score, a score for a partial area within the inspection range can be calculated.

[0016] 1 is placed parallel to a plane defined by the X-axis and Y-axis. In this case, the normal to the surface of inspection object 10 is parallel to the Z-axis. On the other hand, the surface inspection device 1 is placed vertically above the inspection object 10. In other words, the optical axis of the optical system used by the surface inspection device 1 to image the inspection object 10 is set approximately parallel to the normal to the surface of the inspection object 10. Hereinafter, the conditions required for this optical axis will also be referred to as "imaging conditions."

[0017] At this time, the surface inspecting device 1 is installed at a position that satisfies the imaging conditions. The surface inspecting device 1 may be installed by being fixed to a specific member, or may be installed so as to be detachable from the specific member. However, the surface inspection device 1 may be a portable device. If the device is portable, an operator can inspect any surface by holding the surface inspection device 1 in his / her hand, for example, and pointing the light receiving surface toward the inspection target 10. In FIG. 1, for the purpose of explaining the positional relationship between the surface inspection apparatus 1 and the inspection object 10, the appearance of the surface inspection apparatus 1 is simplified and shown as a substantially rectangular parallelepiped.

[0018] <Configuration of surface inspection equipment> FIG. 3 is a diagram illustrating an example of the hardware configuration of the surface inspecting device 1 used in the first embodiment. 3 includes a processor 101 that controls the operation of the entire apparatus, a ROM (Read Only Memory) 102 in which a BIOS (Basic Input Output System) and the like are stored, a RAM (Random Access Memory) 103 used as a work area for the processor 101, an auxiliary storage device 104 that stores programs and image data, a display 105 that displays an image of the surface of the inspection object 10 and information related to operations, an operation reception device 106 that receives operations from an operator, a camera 107 that images the surface of the inspection object 10, a light source 108 that illuminates the surface of the inspection object 10, and a communication IF (Interface) 109 used for communication with the outside. The processor 101 and each part are connected via a signal line 110 such as a bus.

[0019] The processor 101, the ROM 102, and the RAM 103 function as a so-called computer. The processor 101 realizes various functions through the execution of a program. For example, the processor 101 executes a program to calculate a score that evaluates the texture of the surface of the imaged inspection object 10. Image data obtained by capturing an image of the surface of the inspection object 10 is stored in the auxiliary storage device 104. The auxiliary storage device 104 may be, for example, a semiconductor memory or a hard disk drive. Firmware and application programs are also stored in the auxiliary storage device 104. Hereinafter, firmware and application programs are collectively referred to as "programs." In addition, the program that realizes the functions described in this embodiment and other embodiments described later can be provided by communication means, and can also be provided by being stored on a recording medium such as a CD-ROM.

[0020] The display 105 is, for example, a liquid crystal display or an organic EL display, and displays an image of the entire inspection object 10 or a specific portion of the inspection object 10. The display 105 is also used to position the imaging range relative to the inspection object 10. In this embodiment, the display 105 is provided integrally with the device main body, but it may be an external device connected via the communication IF 109, or may be part of another device connected via the communication IF 109. For example, the display 105 may be the display of another computer connected via the communication IF 109.

[0021] The operation reception device 106 is composed of a touch sensor arranged on the display 105, physical switches, buttons, etc. arranged on the housing. In this embodiment, a power button and an image capture button are provided as examples of physical buttons. When the power button is operated, for example, the light source 108 is turned on and image capture by the camera 107 is started. When the image capture button is operated, a specific image that the camera 107 was capturing at the time of operation is acquired as an image for inspection.

[0022] A device that integrates the display 105 and the operation reception device 106 is called a touch panel. The touch panel is used to receive user operations on keys displayed as software (hereinafter also referred to as "soft keys").

[0023] In this embodiment, a color camera is used as the camera 107. The imaging element of the camera 107 is, for example, a CCD (=Charge Coupled Device) imaging sensor element or a CMOS (=Complementary Metal Oxide Semiconductor) imaging element. Since a color camera is used as camera 107, it is possible in principle to observe not only the brightness but also the color tone of the surface of inspection object 10. Camera 107 is an example of an imaging device.

[0024] In this embodiment, a white light source is used as the light source 108. The white light source generates light in which light in the visible light range is evenly mixed. In this embodiment, a parallel light source is used as the light source 108. A telecentric lens is placed on the optical axis of the camera 107.

[0025] In this embodiment, light source 108 is placed at an angle such that the light component specularly reflected on the surface of inspection object 10 is mainly incident on camera 107 . The communication IF 109 is configured with modules that comply with wired or wireless communication standards, and may be, for example, an Ethernet (registered trademark) module, a USB (Universal Serial Bus), a wireless LAN, or the like.

[0026] <Optical system structure> 4A and 4B are diagrams illustrating an example of the structure of the optical system of the surface inspecting device 1 according to embodiment 1. (A) shows a schematic diagram of the internal structure of the housing 100 of the surface inspecting device 1, and (B) shows the structure of the opening 111 that is pressed against the surface of the inspection object 10 during inspection. The opening 111 is provided with an opening 111A through which illumination light that illuminates the surface of the inspection object 10 and reflected light reflected on the surface of the inspection object 10 are input and output, and a flange 111B that surrounds the outer edge of the opening 111A.

[0027] 4, the opening 111A and the flange 111B are both circular, but the opening 111A and the flange 111B may have other shapes, such as a rectangle. It is not necessary for opening 111A and flange 111B to have similar shapes, and opening 111A may be circular and flange 111B may be rectangular.

[0028] The flange 111B is used to position the imaging direction of the surface inspection device 1 relative to the surface of the inspection object 10. In other words, the flange 111B is used to position the camera 107 and the light source 108 relative to the surface being inspected. The flange 111B also serves to prevent or reduce the incidence of external or ambient light into the opening 111A.

[0029] The housing 100 shown in FIG. 4(A) has a structure in which two roughly cylindrical members are joined together, with a light source 108 attached to one member side and a camera 107 and a processor 101 attached to the other member side. A display 105 and an operation reception device 106 are attached to the side of the housing 100 on the side where the camera 107 is attached.

[0030] The MTF (=Modulation Transfer Function) is generally uniform within the field of view of the camera 107. Therefore, the contrast does not vary much depending on the position within the field of view, and the surface of the inspection object 10 can be faithfully imaged.

[0031] 4A, the normal to the surface of the flat plate-shaped inspection object 10 is indicated by N0. Also in FIG. 4A, the optical axis of the illumination light output from the light source 108 is indicated by L1, and the optical axis of the light specularly reflected on the surface of the inspection object 10 is indicated by L2. The optical axis L2 here coincides with the optical axis of the camera 107.

[0032] The surface of the actual inspection object 10 has irregularities due to its structure or design, curved surfaces, steps, joints, minute irregularities formed during the molding process, and the like. Therefore, as the normal N0 of the inspection object 10, the average value of the normal N0 of the area AR of interest within the inspection object 10 or the normal N0 of a specific position P of interest may be used.

[0033] Furthermore, as the normal N0 of the inspection object 10, the normal N0 of an average virtual surface or a representative portion of the inspection object 10 may be used. 4, the optical axis L1 of the illumination light output from the light source 108 and the optical axis L2 of the camera 107 are both attached at an angle θ with respect to the normal line N0. The angle θ is, for example, approximately 30° or approximately 45°.

[0034] <Inspection operation> 5 is a flowchart for explaining an example of the inspection operation by the surface inspection device 1 used in embodiment 1. The symbol S shown in the drawing means a step. The processing shown in FIG. 5 is realized through the execution of a program by the processor 101 (see FIG. 4). In the surface inspecting device 1 of this embodiment, the light source 108 (see FIG. 4) is turned on by operating the power button, and the camera 107 (see FIG. 4) starts capturing an image. The captured image is displayed on the display 105 (see FIG. 4).

[0035] Fig. 6 is a diagram illustrating an example of an operation screen 120 displayed on the display 105. The operation screen 120 shown in Fig. 6 includes a display field 121 for an image captured by the camera 107 (hereinafter referred to as a "captured image field"), a score field 122, a display field 123 for an image in which the features of a partial region that contributed to the calculation of the score are highlighted (hereinafter referred to as a "highlighted image field"), and a legend 124. A distribution of brightness values, that is, a grayscale image, is displayed in the captured image field 121. In the case of Fig. 6, a line 121A that indicates the outer edge of the inspection range used in calculating the score is displayed.

[0036] 6, the inspection range is the area surrounded by four lines 121A. A score representing the surface quality is calculated for the image within the inspection range. A legend 124 is shown to the right of the captured image field 121. In the case of Fig. 6, the shading of the captured image field 121 corresponds to gradation values ​​from "192" to "255". In the case of the operation screen 120 shown in FIG. 6, the score has not yet been calculated, so the score field 122 is blank and no image is displayed in the highlighted image field 123 either.

[0037] Returning to the explanation of Figure 5. In this embodiment, when an operator checking the image displayed on the display 105 operates the image capture button, the image to be used for evaluating the surface quality is determined. Therefore, the processor 101, which has started the inspection operation by operating the power button, determines whether or not it has accepted the operation of the image capturing button (Step 1). The operation of the operation button is an example of an operation to instruct the start of an inspection. While a negative result is obtained in step 1, processor 101 repeats the determination in step 1.

[0038] If a positive result is obtained in step 1, the processor 101 acquires an image to be used for the examination (step 2). Specifically, the image displayed on the display 105 at the time the imaging button was operated is acquired. In the case of this embodiment, when the image capture button is operated, even if the camera 107 continues to capture images, updating of the images displayed in the captured image field 121 (see FIG. 6) is stopped. Next, the processor 101 calculates a score using the luminance profile within the inspection range (step 3).

[0039] 7 is a diagram illustrating the principle of score calculation. (A) shows an example image acquired as an object of inspection, (B) shows an example cross section of a depression formed in the Y-axis direction in the x-direction, and (C) shows the brightness profile S of the image. The depression shown in Fig. 7(B) represents, for example, a sink mark. For convenience of explanation, Fig. 7(B) shows a cross-sectional shape of an isosceles triangle, but of course, this shape is only one example.

[0040] In this case, the luminance profile S shown in FIG. 7C is given as a change in the luminance value (hereinafter referred to as "representative luminance value") that represents each coordinate in the X-axis direction. The representative luminance value here is given by the integral value of the luminance values ​​of each pixel with the same x-coordinate. A convex waveform in the luminance profile S indicates an area that is brighter than the surroundings, and a concave waveform indicates an area that is darker than the surroundings.

[0041] The score is calculated as the difference between the maximum and minimum values ​​of the luminance profile S, for example. The score depends on the width, height, depth, number, etc. of the irregularities formed on the surface. For example, even if the height of the protrusions or the depth of the recesses are the same, the score of a partial region in which a protrusion or recess with a greater width is formed will be higher.

[0042] Furthermore, even if the width of the convex or concave portion is the same, a partial region in which a higher convex portion or a deeper concave portion is formed will have a higher score. In this embodiment, a high score means poor quality. In this embodiment, the partial region that contributes to the calculation of the score is defined as the region between the start point of the convex waveform and the end point of the concave waveform of the brightness profile S shown in FIG. 7(C).

[0043] Returning to the explanation of Figure 5. Once the score is calculated, processor 101 extracts the partial region corresponding to the maximum score (step 4). The partial region here is the image portion within the inspection region that contributed to the calculation of the score. If multiple maximum score values ​​are found, multiple partial regions are extracted. Basically, one partial region is extracted. Next, processor 101 displays a frame indicating the extracted partial region within the image (step 5). Processor 101 also generates an image in which the characteristics of the extracted partial region are emphasized (hereinafter referred to as an "enhanced image") and displays it separately (step 6).

[0044] In this embodiment, the processor 101 extracts a specific periodic component that appears in a specific direction from the extracted partial region, and generates an enhanced image by overlaying a feature image on the original image through inverse transformation of the extracted periodic component. To extract the periodic components, for example, two-dimensional DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), FFT (Fast Fourier Transform), etc. are used. When converting the feature image back to the original image, the intensity component (i.e., brightness value) of each pixel is normalized by the maximum value to expand the range of gradation in the feature image. Also, by mapping color components to the intensity component of the feature image, it is possible to distinguish it from the original image, which is expressed in grayscale.

[0045] By displaying the highlighted image, it becomes possible to check the surface condition even when it is difficult to visually check minute structures in a grayscale image captured of the surface of the partial region for which the score was calculated. In this embodiment, the generated highlighted image is displayed alongside the grayscale image captured by the camera 107 on the same operation screen. Additionally, processor 101 displays the corresponding score on operation screen 120 (see FIG. 6) and ends the process (step 7).

[0046] <Example of operation screen> An example of a screen displayed when the surface inspection device 1 inspects the inspection object 10 will be described below.

[0047] <Screen example 1> 8 is a diagram illustrating an example of the operation screen 120 on which the main contributing factors of the score are displayed. In FIG. 8, the parts corresponding to those in FIG. 6 are assigned the same reference numerals. In the case of FIG. 8, a striped pattern of adjacent high and low brightness areas appears near the center of the captured image field 121. In the score field 122 of the operation screen 120, the numerical value "3.1" is displayed as the score calculated for this partial region. Furthermore, a frame 125 indicating the area that contributed to the calculation of the score (hereinafter referred to as the "main contributing area") is displayed superimposed on the original image in the captured image field 121. In the case of Fig. 8, the frame 125 is a rough rectangle with rounded corners.

[0048] The display of this frame line 125 makes it possible for the worker to determine whether the partial area used to calculate the numerical value displayed in the score column 122 matches the partial area that the worker is focusing on. The frame 125 here is an example of an "indicator for identifying the position of the part that contributed to the calculation of the score." A frame 125 shown in FIG. 8 surrounds the partial region that contributed to the calculation of the score, thereby providing a boundary between the partial region that contributed to the calculation of the score and other partial regions.

[0049] On the operation screen 120 shown in FIG. 8, a legend 126 regarding the frame line 125 is added below the captured image field 121. 8, an enhanced image of the area surrounded by a frame line 125 is displayed in an enhanced image field 123 on the operation screen 120. Therefore, even if it is difficult to confirm the surface condition of the inspection object 10 from only the image captured by the camera 107, it becomes possible to confirm the surface condition.

[0050] <Screen example 2> 9 is a diagram illustrating another example of the operation screen 120 on which the main contributing factors of the score are displayed. In FIG. 9, the parts corresponding to those in FIG. 8 are assigned the same reference numerals. In the case of FIG. 9, a frame 125 indicating the main factor location that contributed to the calculation of the score is displayed near the top of the captured image field 121. The captured image field 121 shown in Fig. 9 includes a streak-like pattern in which high brightness and low brightness are adjacent to each other, but the brightness difference within the partial area surrounded by the frame line 125 is greater. For this reason, the frame line 125 is displayed near the top of the captured image field 121. The score is "5".

[0051] However, the portion surrounded by frame line 125 corresponds to the structural outer edge of inspection object 10 (see FIG. 1) and its background. Therefore, the worker can realize that the score displayed on the operation screen is not the score for the part he or she is interested in. In this case, the worker can take a new image of the object of inspection, etc.

[0052] If a pattern of brightness differences exceeding a threshold is detected in the image, the processor 101 may output an error signal to notify that the captured image is not suitable for inspecting surface quality. Processor 101 may also output an error signal when the calculated score exceeds a threshold. For example, when the threshold is "4" and a score of "5" is calculated, processor 101 outputs an error signal.

[0053] When an error signal is output, the processor 101 may discard the image used to calculate the score and autonomously reacquire another image. The processor 101 may also notify the operator of the need for reimaging. The notification may be made using a display or a sound. In this case, the processor 101 may display the reason why reimaging is considered necessary.

[0054] <Screen example 3> Fig. 10 is a diagram illustrating an example of how a frame line 125 is displayed when multiple stripe patterns are included in an image captured by the camera 107 (see Fig. 3). In Fig. 10, parts corresponding to those in Fig. 8 are assigned the same reference numerals. In the case of Fig. 10, two horizontal stripe patterns are included in the captured image field 121 of the image to be inspected. In other words, in Fig. 10, there are two candidates for the score to be displayed in the score field 122.

[0055] Even in this case, processor 101 (see FIG. 3) displays the largest of the two scores in score column 122. Processor 101 also displays frame line 125, which indicates the streak pattern that contributed to the calculation of the score displayed in score column 122, superimposed on the image in captured image column 121. Specifically, processor 101 displays frame line 125 that surrounds the lower of the two streak patterns. This display lets the worker know that the score in the score column 122 has been calculated from the pattern below.

[0056] <Screen example 4> Fig. 11 is a diagram illustrating another example of displaying frame line 125 when multiple stripe patterns are included in an image captured by camera 107 (see Fig. 3). In Fig. 11, parts corresponding to those in Fig. 8 are assigned the same reference numerals. In the case of FIG. 11 as well, the captured image field 121 of the image to be inspected includes two horizontal stripe patterns.

[0057] Here, if the two scores calculated for the two partial regions corresponding to the two stripe patterns are the same value, two frame lines 125 are displayed in the captured image field 121. However, even when partial regions corresponding to scores where the difference between the maximum score and the second or subsequent score is smaller than the threshold value are displayed with frame lines 125, the operation screen 120 shown in FIG. 11 appears.

[0058] Even if there are three or more scores for which the difference between the maximum score and the second or subsequent score is smaller than the threshold, the number of frame lines 125 displayed in the captured image field 121 may be limited to a predetermined number. For example, only two frame lines 125 indicating the partial area corresponding to the maximum score and the partial area corresponding to the second highest score may be displayed in the captured image field 121.

[0059] In the example of Fig. 11, the two frame lines 125 are displayed in the same format. Therefore, a worker looking at the operation screen 120 shown in Fig. 11 can confirm that there are two partial areas with the same score or two partial areas with approximately the same score. 11, it is not possible to know whether the two scores corresponding to the two partial regions are the same or different values. Furthermore, if there is a difference in the scores, it is not possible to know which partial region corresponds to the score displayed in score column 122. In the case of FIG. 11 as well, the highlighted image field 123 displays the highlighted image of the partial region corresponding to the maximum score.

[0060] <Screen example 5> Fig. 12 is a diagram illustrating another example of displaying frame line 125 when multiple stripe patterns are included in an image captured by camera 107 (see Fig. 4). In Fig. 12, parts corresponding to those in Fig. 11 are assigned the same reference numerals. In the example screen shown in Fig. 12, differences in score size are expressed by different colors of frame lines 125. For example, frame line 125A surrounding the partial area corresponding to the maximum score is displayed in red, and frame line 125B surrounding the partial area corresponding to the second largest score is displayed in green. In the example of Fig. 12, differences in display color are expressed by different line types.

[0061] If it is possible to distinguish the difference in the magnitude of the scores, the brightness of frame line 125A and the brightness of frame line 125B may be changed, or only one of frame line 125A and frame line 125B may be displayed by blinking. Also, the difference in the magnitude of the score may be expressed by the difference in the type of line, such as a double line or a dashed line. In other words, the difference in the magnitude of the score may be expressed by the difference in the display format. 12, an explanation of frame 125B is added as legend 126. This additional explanation enables the operator to know which of the partial regions indicated by frame 125A and the partial region indicated by frame 125B corresponds to the maximum score.

[0062] <Screen example 6> 13A and 13B are diagrams illustrating other examples of displaying the frame line 125 when multiple streak patterns are included. (A) is a screen displaying the frame line 125 indicating the main factor location for the maximum score, and (B) is a screen displaying the frame line 125 indicating the main factor location for the second largest score. In FIG. 13, parts corresponding to those in FIG. 11 are denoted by the same reference numerals. The example screen shown in FIG. 13 is used when multiple scores greater than a predetermined threshold are found, or when multiple scores whose difference from the maximum score is less than the threshold are found.

[0063] In the case of FIG. 13, the position of the frame line 125 superimposed on the image captured by the camera 107 and the numerical value displayed in the score field 122 are switched in sequence. For example, when the worker taps on the captured image field 121, the screen of Figure 13(A) switches to the screen of Figure 13(B), and when the worker taps on the captured image field 121 again, the screen of Figure 13(B) switches to the screen of Figure 13(A).

[0064] 13(A) displays "3.1" in the score field 122 of the operation screen 120, and the phrase "Main contributing area for maximum score" is displayed in the legend 126. Upon checking this display, the worker knows that the score of the partial area corresponding to the frame line 125 displayed in the captured image field 121 is "3.1", and that this value is the maximum score within the inspection range. 13(B), the score field 122 of the operation screen 120 displays "3," and the legend 126 displays the phrase "Main contributing area for second highest score." Upon checking this display, the worker knows that the score of the partial area corresponding to the frame line 125 displayed in the captured image field 121 is "3," and that this value is the second highest score within the inspection range.

[0065] Note that the operation screen 120 shown in Figures 13(A) and 13(B) is designed to display two scores, but if there are three or more scores to display, the operation screen 120 will switch between three or more screens. However, if the number of screens to be switched is greater than a predetermined threshold, a function can be provided that allows the user to return to the previous screen by performing a specific operation, thereby making the worker's confirmation work more efficient than when screens can only be switched sequentially. Furthermore, the processor 101 may switch the screen every few seconds.

[0066] <Screen example 7> Fig. 14 is a diagram illustrating an example of displaying scores that represent surface quality. In Fig. 14, parts corresponding to those in Fig. 8 are assigned the same reference numerals. In the case of the operation screen 120 shown in FIG. 14, the score field 122 is not arranged below the highlighted image field 123. 14, a score field 122A is displayed alongside the frame line 125. This display makes it easier for the operator to understand the relationship between the score and the corresponding partial area. In the example of FIG. 14, the score column 122A is placed below the frame line 125, but it may be placed in another position, for example, on the right side, left side, or upper side.

[0067] Fig. 15 is a diagram illustrating another example of displaying scores that represent surface quality. In Fig. 15, parts corresponding to those in Fig. 14 are assigned the same reference numerals. In the case of Fig. 15, two frame lines 125 are displayed in the captured image field 121. In the case of Fig. 15, the display format of the two frame lines 125 is the same as in screen example 4 (see Fig. 11). However, in the case of the operation screen 120 shown in FIG. 15, the corresponding score column 122A is displayed alongside the frame lines 125, so even if the display format of the two frame lines 125 is the same, it is possible to know the score calculated from the corresponding partial area. In the case of FIG. 15, the score of the upper pattern is "3" and the score of the lower pattern is "3.1".

[0068] <Screen example 8> 16A and 16B are diagrams illustrating an example of additionally displaying the score of a partial area designated by an operator. (A) is a diagram illustrating the designation of a partial area by an operator, and (B) shows an example of the display of the operation screen 120 after the designation. In Fig. 16, parts corresponding to those in Fig. 15 are assigned the same reference numerals. 16(A) assumes that the display 105 is a touch panel. The worker moves his / her fingertip 130 so as to surround the partial area of ​​the display field 121 for which the worker wants to check the score, thereby indicating to the processor 101 the partial area for which the worker wants to calculate the score.

[0069] Legend 126 in Fig. 16(B) indicates that frame line 125C is the location specified by the worker. In Fig. 16(B), the frame line 125 is displayed in a color different from that of frame line 125 displayed by processor 101. In Figs. 16(A) and 16(B), the difference in color is expressed by the difference in line type. In the case of Figure 16(B), the score of the part specified by the worker is "3." This function allows the worker to freely check the score of any part that concerns them. The point where the score is calculated can be specified not only by specifying the outer edge of the corresponding partial area but also by tapping on the point of interest. In this case, processor 101 may determine the partial area including the tapped point and calculate the score.

[0070] <Screen example 9> 17A and 17B are diagrams illustrating other examples of indicators representing the main contributing areas used in calculating the score. (A) shows a case where a symbol is displayed at a position that serves as a guide for the outer edge of the target partial area, and (B) shows a case where the target partial area is represented by an arrow. In FIG. 17, parts corresponding to those in FIG. 6 are assigned the same symbols. 17(A), triangular symbols 125D are displayed at the four corners of the main contributing area used to calculate the score. This display also allows the operator to know the partial area used to calculate the score. Note that the shape of the symbols 125D is an example, and other shapes such as a circle or a star may also be used.

[0071] On the other hand, on the operation screen 120 shown in FIG. 17(B), the approximate position of the main factor part used in calculating the score is indicated by an arrow 125E. The display using the arrow 125E does not reveal the extent of the main cause location. However, if the outer edge of the structure is shown, as in the case of screen example 2 (see FIG. 9), it becomes clear that the partial area used to calculate the score is different from the area the worker is focusing on.

[0072] <Screen example 10> 18 is a diagram showing another example of displaying the frame 125 indicating the partial region that contributed to the calculation of the score. In Fig. 18, the parts corresponding to those in Fig. 8 are assigned the same reference numerals. In the case of the operation screen 120 shown in FIG. 18, a sink mark having a curved shape is displayed in the image field 121 captured by the camera 107 (see FIG. 4). While the frame line 125 in the other screen examples is roughly rectangular, the frame line 125 shown in Fig. 18 is curved. That is, the shape of the frame line 125 is not limited to a roughly rectangular shape, but can take various shapes depending on the shape of the defect.

[0073] <Embodiment 2> In this embodiment, a surface inspection device 1 (see FIG. 1) that does not require operation of an image capture button when calculating a score will be described. The external configuration of the surface inspecting device 1 in this embodiment is the same as that of the surface inspecting device 1 described in the first embodiment.

[0074] Fig. 19 is a flowchart for explaining an example of the inspection operation by the surface inspecting device 1 used in the embodiment 2. In Fig. 19, parts corresponding to those in Fig. 5 are assigned the same reference numerals. In the case of FIG. 19, processor 101 (see FIG. 4) turns on light source 108 (see FIG. 4) by operating the power button, starts capturing images with camera 107 (see FIG. 4), and simultaneously performs score calculation and the like.

[0075] For this reason, the processor 101 acquires an image being captured by the camera 107 (step 11), and calculates a score using the brightness profile within the inspection range (step 3). Thereafter, processor 101 extracts the partial area corresponding to the maximum score (step 4), and performs the following operations: displaying a frame 125 (see FIG. 8) indicating the extracted partial area, displaying an image of the corresponding partial area, displaying the corresponding score, etc. (steps 5 to 7).

[0076] After step 7 is completed, processor 101 returns to step 11 and repeats the series of processes. By repeating this series of processes, the score displayed on operation screen 120 (see FIG. 8) continues to be updated in accordance with changes in the image captured by camera 107. That is, when the position at which the camera 107 captures an image changes, the positions of the score and the frame line 125 also change.

[0077] <Third Embodiment> 20 is a diagram for explaining an example of use of the surface inspection device 1A assumed in the embodiment 3. In FIG. 20, parts corresponding to those in FIG. 1 are denoted by the same reference numerals. The surface inspection device 1A used in this embodiment uses a so-called line camera as the imaging unit, so the imaging range is linear.

[0078] In the case of this embodiment, during inspection, the inspection object 10 is moved in the direction of the arrow while being placed on the uniaxial stage 20. By moving the uniaxial stage 20 in one direction, the entire inspection object 10 is imaged. Note that, except that a line camera is used as the camera 107 (see FIG. 4), the positional relationship between the camera 107 (see FIG. 4) and the light source 108 (see FIG. 4) and the like are the same as those in the first embodiment.

[0079] <Other embodiments> (1) Although the embodiments of the present invention have been described above, the technical scope of the present invention is not limited to the scope of the above-described embodiments. It is clear from the claims that various modifications and improvements to the above-described embodiments are also included in the technical scope of the present invention.

[0080] (2) In the above-described embodiment, a color camera is used as the camera 107 (see FIG. 4), but a monochrome camera may also be used. Also, only the green (G) component of the color camera may be used to inspect the surface of the inspection object 10 (see FIG. 1).

[0081] (3) In the above embodiment, a white light source is used as the light source 108 (see FIG. 4), but the color of the illumination light may be any color. Furthermore, the illumination light is not limited to visible light, but may be infrared light, ultraviolet light, or the like.

[0082] (4) In the above embodiment, the surface inspection device 1 (see FIG. 1) using one light source 108 (see FIG. 4) has been described, but the surface of the inspection object 10 may be illuminated using multiple light sources. For example, two light sources may be used. In this case, one light source may be arranged at an angle where specularly reflected light components are mainly incident on camera 107 (see FIG. 4), and the other light source may be arranged at an angle where diffusely reflected light components are mainly incident on camera 107. In this case, the two light sources may be arranged on either side of the optical axis of camera 107, or may be arranged side by side on one side of the optical axis of camera 107.

[0083] (5) In the above-described embodiment, a parallel light source is used as the light source 108 (see FIG. 4). However, a non-parallel light source such as a point light source or a surface light source may also be used. Also, a non-telecentric lens may be used on the optical axis of the camera 107 (see FIG. 4). When a telecentric lens or parallel light is not used, the device can be made smaller and less expensive than the surface inspection device 1 (see FIG. 1) described in the embodiment.

[0084] (6) In the above-described embodiment, the processor 101 (see FIG. 3) of the surface inspection device 1 (see FIG. 1) that captures the image of the inspection object 10 (see FIG. 1) realizes the function of calculating the score and displaying the frame line 125 (see FIG. 8) indicating the main partial area that contributed to the score calculation on the operation screen 120 (see FIG. 8). However, the same function may also be realized by the processor of an external computer or server that acquires image data from the surface inspection device 1.

[0085] (7) The processor in each of the above-mentioned embodiments refers to a processor in a broad sense, and includes general-purpose processors (e.g., CPUs, etc.) as well as dedicated processors (e.g., GPUs (Graphical Processing Units), ASICs (Application Specific Integrated Circuits), FPGAs (Field Programmable Gate Arrays), programmable logic devices, etc.). Furthermore, the operations of the processor in each of the above-described embodiments may be performed by a single processor alone, or may be performed by multiple processors located in physically separate locations in cooperation with each other. Furthermore, the order in which the operations of the processors are performed is not limited to the order described in each of the above-described embodiments, and may be changed individually. [Explanation of symbols]

[0086] 1, 1A... surface inspection device, 10... inspection object, 20... single-axis stage, 101... processor, 102... ROM, 103... RAM, 104... auxiliary storage device, 105... display, 106... operation reception device, 107... camera, 108... light source, 109... signal line, 111... opening, 111A... opening, 111B... flange, 120... operation screen, 122, 122A... score column, 125, 125A, 125B, 125C... frame line, 125D... symbol, 125E... arrow

Claims

1. an imaging device for imaging the surface of an object to be inspected; a processor that processes the image captured by the imaging device and calculates a numerical value representing the quality of the surface corresponding to the entire inspection range of the captured image; the processor displays the image including an indicator specifying the position of the portion that contributed to the calculation of the numerical value and the numerical value on a display device; When there are multiple candidates for the numerical value, the processor displays the indicator that identifies the position of the portion corresponding to the largest numerical value. Surface inspection equipment.

2. the indicator for identifying the position indicates a boundary between the portion and other portions; The surface inspection device according to claim 1 .

3. The indicator for identifying the position is a frame line surrounding the periphery of the portion. The surface inspection device according to claim 2.

4. When there are a plurality of candidates for the numerical value, the processor displays the indicator that identifies the position of the portion corresponding to each candidate. The surface inspection device according to claim 1 .

5. When a plurality of the indices are displayed, the processor associates the order of magnitude of the corresponding numerical values ​​with the display format of the indices. The surface inspection device according to claim 4.

6. When the processor displays a plurality of the indicators, the processor switches the display of the numerical values ​​and the indicators in descending order of magnitude of the corresponding numerical values. The surface inspection device according to claim 4.

7. The processor displays the numerical value alongside the corresponding indicator. The surface inspection device according to any one of claims 1 to 6.

8. the processor notifies a user when a luminance difference in the image corresponding to the portion is greater than a predetermined threshold. The surface inspection device according to any one of claims 1 to 7.

9. the processor displays the numerical value and the indicator on the image when the examination begins. The surface inspection device according to any one of claims 1 to 7.

10. When the processor receives an operation to instruct the start of the examination, the processor displays the numerical value and the indicator on the image. The surface inspection device according to any one of claims 1 to 7.

11. The surface inspection device according to any one of claims 1 to 10, characterized in that the device itself is portable.

12. A computer processes an image of the surface of an object to be inspected taken by an imaging device. a function of calculating a numerical value representing the quality of the surface corresponding to the entire inspection range of the captured image; a function of displaying the image including an index specifying the position of the portion that contributed to the calculation of the numerical value and the numerical value on a display device; a function of displaying the indicator that identifies the position of the portion corresponding to the largest numerical value when there are multiple candidates for the numerical value; A program to achieve this.

Images