Surface Inspection Equipment

Abstract

To provide a surface inspection device capable of stably performing a quantitative evaluation of surface conditions of an inspected object without the variation in results between inspectors.SOLUTION: A surface inspection device to quantitatively evaluate the distribution state of surface unevenness of an inspected object comprises an image acquisition part to acquire the surface image of an inspected object, an area division part to divide the acquired image into multiple small areas, and a digitization processing part to convert a pixel value in each pixel of the small area to numerical data representing the convex-likeness and the concave-likeness of the pixel based on a learning model learning a convex-likeness and a concave-likeness of the unevenness state in the small area.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to a surface inspection apparatus for quantitatively evaluating the surface condition of an object that has been subjected to processing such as scraping. [Background technology]

[0002] The sliding surfaces of machine tools are scraped to ensure flatness and oil reservoirs. Scraping is a metal processing method that uses tools such as scrapers to finish the surface of castings and other objects flat. The surface formed by scraping is called a scraped surface. The main evaluation criteria for a scraped surface are flatness, the percentage of contact surfaces, and the number of contacts per unit area. Of these, the percentage of contact surfaces and the number of contacts per unit area are judged by visual inspection by an inspector, and are highly subjective.

[0003] To solve such problems, a method has been proposed for quantitatively evaluating the quality of a scraped surface by using image analysis to generate a binary image that distinguishes the scraped surface into contact and non-contact areas (e.g., Non-Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Non-Patent Document 1] Hirotaka Tsutsumi, Takato Yamagata, Hiromi Isobe, Katsumi Fukuda: "Development of a Simple Image Measurement System for Measuring the Per-Bascal of Scraped Surfaces", Proceedings of the 2013 Japan Society for Precision Engineering Spring Meeting, pp.49-50, 2013 Summary of the Invention [Problem to be solved by the invention]

[0005] In the system of Non-Patent Document 1, it is difficult to set the threshold value when generating a binary image that distinguishes between hit and non-hit areas, and there has been a demand for a system that allows for more stable quantitative evaluation.

[0006] The present invention has been made in view of the above, and has an object to provide a surface inspection device that can stably perform quantitative evaluation of the surface condition of an object to be inspected without variation in results depending on the inspector. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems and achieve the object, a surface inspection apparatus according to the present invention is a surface inspection apparatus for quantitatively evaluating the distribution state of unevenness on the surface of an object to be inspected, comprising: an image acquisition unit that acquires an image of the surface of the object to be inspected; and an area division unit that divides the acquired image into a plurality of small areas. A rule for classifying the brightness distribution of the surface of the object into regions of convexities and regions of concaveities by associating the image of the surface of the object with the result of classifying the convexities and concaveities in the image. and a digitization processing unit that converts the pixel values ​​of each pixel in the small region into numerical data that indicates the likelihood of the pixel being a convex or concave part, based on a learning model that has learned the above. [Effects of the Invention]

[0008] The surface inspection apparatus according to the present invention has the effect of enabling stable surface inspection of an object to be performed without variations in results depending on the inspector. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a schematic diagram showing a schematic configuration of a surface inspection device according to an embodiment. [Figure 2] FIG. 2 is a hardware block diagram showing an example of the hardware configuration of a mobile terminal included in the surface inspection device of the embodiment. [Figure 3] FIG. 3 is a diagram for explaining the image processing performed by the surface inspection device. [Figure 4] FIG. 4 is a diagram showing an example of a learning model included in the surface inspection device. [Figure 5] FIG. 5 is a functional block diagram showing an example of the functional configuration of the surface inspection device according to the embodiment. [Figure 6] FIG. 6 is a flowchart showing an example of the flow of processing performed by the surface inspecting apparatus according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of a surface inspection device according to the present disclosure will be described in detail with reference to the drawings. Note that the present invention is not limited to these embodiments. Furthermore, the components in the following embodiments include those that are replaceable and easily conceivable by a person skilled in the art, or those that are substantially the same.

[0011] (Embodiment) The embodiment of the present disclosure is an example of a surface inspection device 10 that quantitatively evaluates the surface condition of an object 12 that has been manually processed by an operator, such as a scraped surface.

[0012] (Schematic configuration of surface inspection device) The general configuration of a surface inspection device 10 will be described with reference to Fig. 1. Fig. 1 is a schematic diagram showing the general configuration of a surface inspection device according to an embodiment.

[0013] The surface inspection device 10 according to this embodiment is built into a mobile terminal 18 and quantitatively evaluates the surface condition of the test object 12 by analyzing an image of the surface of the test object 12 taken by a camera 20.

[0014] The mobile terminal 18 incorporating the surface inspection device 10 is, for example, a tablet terminal. The mobile terminal 18 includes a camera 20 and a display device 34. The camera 20 is installed on the back side of the display device 34 of the mobile terminal 18, and captures images in response to operational instructions from an inspector 22.

[0015] The display device 34 displays the images captured by the camera 20 and the evaluation results obtained by the surface inspection device 10. The display device 34 is, for example, a liquid crystal display or an organic EL display.

[0016] The subject 12 is illuminated by a light 16 and photographed by a camera 20. The light 16 is, for example, an incandescent bulb or an LED, but it is preferable that the light 16 can be powered by DC and can be continuously lit without flicker. If flicker occurs when the light is turned on, it is preferable that the camera 20 has a function for inputting an image with the flicker reduced.

[0017] A tag 14 is attached near the subject 12, and the camera 20 takes an image so that the subject 12 and the tag 14 appear in the same image.

[0018] The tag 14 is a two-dimensional code, a barcode, or the like, but is not limited to these. The tag 14 has a predetermined shape and size. Identification information that uniquely identifies the subject 12 and identification information that uniquely identifies the inspector 22 are registered in the tag 14. The tag 14 is an example of a marker in the present disclosure.

[0019] (Hardware configuration of surface inspection device) The hardware configuration of the surface inspecting apparatus 10 will be described with reference to Fig. 2. Fig. 2 is a hardware block diagram showing an example of the hardware configuration of a mobile terminal provided in the surface inspecting apparatus of the embodiment.

[0020] The surface inspecting device 10 (portable terminal 18) includes a control unit 31 for controlling each unit. The control unit 31 includes a CPU (Central Processing Unit) 31a, a ROM (Read Only Memory) 31b, and a RAM (Random Access Memory) 31c. The CPU 31a is connected to the ROM 31b and the RAM 31c via an internal bus 33 such as an address bus and a data bus. The CPU 31a loads various programs and data stored in the ROM 31b and the storage unit 32 into the RAM 31c. The CPU 31a controls each unit of the surface inspecting device 10 by operating in accordance with the various programs loaded into the RAM 31c. In other words, the control unit 31 has the configuration of a general computer.

[0021] The control unit 31 is connected to a storage unit 32 , a display device 34 , an operation device 35 , a camera controller 36 , and a communication controller 37 via an internal bus 33 .

[0022] The storage unit 32 is a storage device such as a hard disk drive (HDD) or a solid state drive (SSD). The storage unit 32 may also be a non-volatile memory such as a flash memory that retains stored information even when the power is turned off. The storage unit 32 stores a control program 32a, learning data 32b, and evaluation data 32c. The storage unit 32 also stores image data currently being processed, such as an input image I(x, y) (see FIG. 3) captured by the camera 20 and a divided image J(x, y) (see FIG. 3).

[0023] The control program 32 a is a program that controls the overall operation of the surface inspection apparatus 10 .

[0024] The learning data 32b includes a learning model that learns the relationship between images of the surfaces of a plurality of test objects 12 and the likelihood of a convex portion or a concave portion of the test object 12. The learning model is described by, for example, a neural network. Details will be described later (see FIG. 4).

[0025] The evaluation data 32c is data indicating the results of quantifying the ratio of convex portions and the number of convex portions per unit area of ​​the test object 12 by applying the learning data 32b to an image of the surface of the test object 12. The evaluation data 32c may also be a quantified ratio of concave portions and the number of concave portions per unit area of ​​the test object 12. The evaluation data 32c may also include information registered in the tag 14, an input image I(x,y), which will be described later, and an evaluation image K(x,y) and a binary image L(x,y) indicating the evaluation result.

[0026] The display device 34 displays various screens generated in response to instructions from the control unit 31. The display device 34 is configured by a device such as an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display.

[0027] The operation device 35 acquires an input operation by the operator and transmits it to the control unit 31. The operation device 35 is configured by a device such as a touch panel, for example.

[0028] The camera controller 36 controls the operation of the camera 20 in response to commands from the control unit 31 .

[0029] The communication controller 37 is a controller that controls communication between the surface inspection device 10 (portable terminal 18) and a server device (not shown) or the like in response to commands from the control unit 31. The surface inspection device 10 (portable terminal 18) transmits information such as the evaluation results of the test specimens 12 that it has acquired to the server device at a predetermined timing. The server device collects and manages the evaluation results of the test specimens 12 transmitted from multiple surface inspection devices 10.

[0030] (Image processing performed by surface inspection equipment) The image processing performed by the surface inspecting device 10 will be described with reference to Fig. 3. Fig. 3 is a diagram for explaining the image processing performed by the surface inspecting device.

[0031] 3 is an example of an image captured by the camera 20. The input image I(x, y) shows an object 12 and a tag 14. It is assumed that the object 12 and the tag 14 are attached in advance so that they face the same direction.

[0032] When the surface inspection device 10 detects that the inspector 22 has given an instruction to capture an image, it causes the camera 20 to capture an input image I(x, y), provided that the tag 14 is captured horizontally. At this time, the distance between the camera 20 and the test object 12 may be measured, and the input image I(x, y) may be acquired only if the tag 14 is captured horizontally and the distance between the camera 20 and the test object 12 is within a predetermined range. Whether the distance between the camera 20 and the test object 12 is within the predetermined range can be determined, for example, by the size of the tag 14 captured in the input image I(x, y).

[0033] The orientation of the subject 12 in the input image I(x, y) is specified in this way to ensure that the orientation of the subject 12 in the input image I(x, y) is always the same, thereby preventing variations in the evaluation results due to differences in the orientation of the subject 12.

[0034] In order to evaluate the surface condition of the test object 12 captured in the captured input image I(x,y), the surface inspection device 10 first divides the input image I(x,y) into regions of a predetermined size. Specifically, the surface inspection device 10 extracts only the region of the test object 12 from the input image I(x,y) and divides the extracted region of the test object 12 vertically and horizontally into regions the same size as the tags 14. For example, if the evaluation of the scraped surface is performed based on a region of 1 inch in length and width, a tag 14 of 1 inch in length and width is affixed near the test object. Note that the process of extracting only the region of the test object 12 from the input image I(x,y) can be performed by, for example, setting the color and brightness of the table on which the test object 12 is placed to a different color and brightness from those of the test object 12, and then analyzing the color and brightness of the input image I(x,y). In addition, the process of extracting the area of ​​tag 14 from the input image I(x, y) is performed, for example, by a commonly used two-dimensional code reading application as a process of identifying the area of ​​tag 14 before reading the two-dimensional code.

[0035] By dividing the area of ​​the subject 12 into portions the size of the tag 14, a divided image J(x, y) as shown in Fig. 3 is obtained. The divided image J(x, y) is an image that is sampled into, for example, 128 vertical pixels and 128 horizontal pixels, and each pixel is quantized to 8 bits (i.e., the pixel value takes any value from 0 to 255). Note that the size of the divided image J(x, y) is an example and is not limited to 128 vertical pixels and 128 horizontal pixels.

[0036] The surface inspection device 10 generates a feature map M (see FIG. 4), which will be described later, for each pixel of interest in a segmented image J(x, y), based on the distribution of pixel values ​​of multiple pixels surrounding the pixel and on a predetermined rule. The surface inspection device 10 inputs multiple feature values ​​contained in the feature map M to a network N (see FIG. 4), which will be described later. The network N is a network that has been trained to output a numerical value corresponding to the likelihood of a pixel of interest being a convex part (likely a hit part) or a concave part (likely a non-hit part) based on the multiple segmented images J(x, y) provided. The structure of the network N will be described in detail later.

[0037] By applying the network N to each pixel of the divided image J(x,y), for example, an evaluation image K(x,y) as shown in FIG. 3 is obtained. The evaluation image K(x,y) is an image of the same size as the divided image J(x,y), and has pixel values ​​corresponding to the convexity output by the network N. Here, as an example, the convexity takes a value between 0.0 (least likely to be a convex) and 1.0 (most likely to be a convex). Note that the evaluation image K(x,y) may also be an image that stores pixel values ​​corresponding to the concavity. In that case, values ​​between 0.0 (least likely to be a concavity) and 1.0 (most likely to be a concavity) are stored in the evaluation image K(x,y).

[0038] The surface inspecting apparatus 10 binarizes the evaluation image K(x, y) using a predetermined threshold value to generate a binary image L(x, y) in which, for example, convex portions, i.e., hit portions, are black (pixel value 0) and concave portions, i.e., non-hit portions, are white (pixel value 255). The binarization threshold may be set arbitrarily, but is set to, for example, 0.5 (convex portion-likelihood, i.e., hit portion-likelihood is 50%). Note that the evaluation image K(x, y) in which pixel values ​​between 0.0 and 1.0 are stored may be binarized as is, or the pixel values ​​of the evaluation image K(x, y) may be converted to values ​​between 0 and 255 before being binarized. The surface inspecting apparatus 10 may also generate a binary image L(x, y) in which convex portions, i.e., hit portions, are white (pixel value 255) and concave portions, i.e., non-hit portions, are black (pixel value 0).

[0039] The surface inspecting device 10 calculates the area ratio of convex portions (hit portions) in the divided image J(x,y) by calculating the total number of pixels constituting the convex portions in the binary image L(x,y) generated in this way. The surface inspecting device 10 also calculates the number of convex portions. The number of convex portions represents the number of hit portions per unit area of ​​the test object 12. The surface inspecting device 10 then associates information specifying the position of the divided image J(x,y) in the test object 12 (for example, information specifying the position A from the left, B from the top, etc.), the calculated binary image L(x,y), the area ratio of convex portions, the number of convex portions, and information on the read tag 14, and stores them in the evaluation data 32c (see FIG. 2). The evaluation data 32c is transmitted to a server device that manages evaluation results of multiple test objects 12 at the instruction of the inspector 22.

[0040] The surface inspecting device 10 performs the same process on all the divided images J(x, y).

[0041] (Learning model structure) The structure of a learning model that the surface inspecting device 10 uses to calculate the likelihood of a pixel of interest in a divided image J(x, y) being a convex part or a concave part will be described with reference to Fig. 4. Fig. 4 is a diagram showing an example of a learning model that the surface inspecting device has.

[0042] The learning model is described by a network N. The network N forms a so-called convolutional neural network (CNN). The network N forms a three-layer neural network consisting of an input layer L1, an intermediate layer L2, and an output layer L3. Information related to pixel values ​​of the divided image J(x, y) is input to the input layer L1. The intermediate layer L2 performs a product-sum operation (convolution) on multiple outputs of the input layer L1 to clarify the characteristics of the values ​​input to the input layer L1. The output layer L3 performs a pooling process, an information compression operation, on the multiple outputs of the intermediate layer L2 to output a numerical value representing the likelihood of a convex portion (likely a hit portion) of the object 12 (for example, a numerical value between 0 and 1, with 1 being most likely to be a convex portion) and a numerical value representing the likelihood of a concave portion (likely a non-hit portion) (for example, a numerical value between 0 and 1, with 1 being most likely to be a concave portion).

[0043] Different weighting coefficients are assigned to the multiple links connecting the input layer L1 and the intermediate layer L2 of the network N, and to the multiple links connecting the intermediate layer L2 and the output layer L3. The values ​​of the weighting coefficients are determined by repeatedly training the network N. Training is performed by an operator who performs training indicating the correct answer for the likelihood of convexities and concavities obtained when segmented images J(x,y) acquired from multiple objects 12 are input to the network N. By repeatedly performing such training, the values ​​of the weighting coefficients assigned to each link are adjusted so as to output values ​​close to the given correct answer. By performing such training, the network N learns rules for distinguishing the brightness distribution on the surface of the object 12 into convex and concave regions. The network N (training model) can then associate the segmented image J(x,y) of the surface of the object 12 with the results of distinguishing convexities and concavities in the segmented image J(x,y).

[0044] The input layer L1 has a plurality of nodes (node ​​60a, node 60b, ... node 60n). Each node receives input of values ​​related to the pixel values ​​of a pixel of interest and its neighboring pixels in the divided image J(x, y).

[0045] In this embodiment, a plurality of feature quantities calculated from the pixel value of a pixel P(a,b) of interest in the divided image J(x,y) and the pixel values ​​of pixels neighboring the pixel P(a,b) are input to the input layer L1. More specifically, as shown in FIG. 4 , a plurality of feature quantities calculated from the pixel value of the pixel P(a,b) in the divided image J(x,y) and the pixel values ​​of pixels neighboring the pixel P(a,b) are input to each node (node ​​60a, node 60b, ... node 60n) of the input layer L1. The plurality of feature quantities are, for example, local feature quantities of the image, such as a value obtained by smoothing the divided image J(x,y), a result of calculating the maximum or minimum value of neighboring pixels by applying a kernel of a predetermined size to the divided image J(x,y), and an edge intensity calculated by applying a kernel of a predetermined size to the divided image J(x,y). The type of local feature quantity to be used may be determined appropriately based on the type of the object 12, the characteristics of the scraping, etc. The calculated local feature amounts form a feature map M shown in Fig. 4. Then, the local feature amounts of the image at pixel P(a, b) stored in the feature map M are input to node 60a, node 60b, ... node 60n, respectively.

[0046] The middle layer L2 has a plurality of nodes (node ​​70a, node 70b, ... node 70m). All of the nodes in the middle layer L2 are connected to all of the nodes in the input layer L1.

[0047] The output layer L3 has a plurality of nodes (node ​​80a, node 80b). All of the nodes in the output layer L3 are connected to all of the nodes in the intermediate layer L2. The node 80a outputs a numerical value representing the likelihood of a convex portion of the object 12. The node 80b outputs a numerical value representing the likelihood of a concave portion of the object 12.

[0048] 4, the pixel value of pixel P(a,b) of divided image J(x,y) and the pixel values ​​of pixels in the vicinity of pixel P(a,b) may be directly input to node 60a, node 60b, ... node 60n. In other words, the feature map M does not need to be generated.

[0049] In addition, in FIG. 4, the intermediate layer L2 may be further divided into a plurality of layers, and the product-sum operation may be repeated a plurality of times.

[0050] (Functional configuration of surface inspection device) The functional configuration of the surface inspecting device 10 will be described with reference to Fig. 5. Fig. 5 is a functional block diagram showing an example of the functional configuration of the surface inspecting device according to the embodiment.

[0051] The control unit 31 of the surface inspection device 10 loads the control program 32a into the RAM 31c and runs it, thereby realizing, as functional units, an image acquisition unit 51, a marker extraction unit 52, an area division unit 53, a digitization processing unit 54, a binarization processing unit 55, an evaluation result output unit 56, a display control unit 57, an operation control unit 58, and a communication control unit 59, all of which are shown in Fig. 5. Note that some or all of these functions may be realized by dedicated hardware.

[0052] The image acquiring unit 51 acquires an image (input image I(x, y) in FIG. 3) of the surface of the subject 12. The image acquiring unit 51 also acquires the image of the surface of the subject 12 on the condition that the tag 14 attached near the subject 12 is captured in a predetermined orientation.

[0053] The marker extraction unit 52 extracts the tag 14 attached near the subject 12 from the input image I(x, y). The marker extraction unit 52 also reads various information registered in the extracted tag 14. Furthermore, the marker extraction unit 52 determines whether the orientation of the extracted tag 14 is correct, i.e., whether the tag 14 is photographed horizontally.

[0054] The region dividing unit 53 divides the acquired input image I(x,y) into a plurality of small regions. The region dividing unit 53 divides the input image I(x,y) into a plurality of small regions each having the same size as the tag 14 extracted from the input image I(x,y). The region dividing unit 53 also generates a divided image J(x,y) by scaling (resizing) each of the divided small regions to a predetermined size, such as 128×128 pixels.

[0055] The digitization processing unit 54 converts the pixel value of each pixel in the small region into numerical data indicating the likelihood of the pixel being a convex or concave, based on the network N (learning model) that has learned the likelihood of the pixel being a convex or concave in the uneven state in the small region obtained by dividing the input image I(x,y). This results in an evaluation image K(x,y).

[0056] The binarization processing unit 55 identifies the positions of the convex and concave portions by binarizing the numerical data converted by the digitization processing unit 54 into a numerical value that represents a predetermined likelihood of a convex portion or a concave portion. Furthermore, the binarization processing unit 55 calculates the ratio of convex portions, the number of convex portions, etc. based on the results of the binarization processing.

[0057] The evaluation result output unit 56 outputs the evaluation image K(x, y) generated by the digitization processing unit 54, the binary image L(x, y) generated by the binarization processing unit 55, the ratio of convex portions, the number of convex portions, etc. to the display device 34. The evaluation result output unit 56 also stores these images and numerical values ​​in the storage unit 32 as evaluation data 32c.

[0058] The display control unit 57 causes the control unit 31 to generate various display information to be output to the display device 34 provided in the surface inspection apparatus 10 (mobile terminal 18).

[0059] The operation control unit 58 acquires operation information of the operation device 35 provided in the surface inspecting apparatus 10 (portable terminal 18). In addition, the operation control unit 58 causes the control unit 31 to perform control corresponding to the operation information of the operation device 35.

[0060] The communication control unit 59 controls the communication between the surface inspection device 10 (portable terminal 18) and the server device.

[0061] (Processing flow performed by surface inspection equipment) The flow of processing performed by the surface inspecting apparatus 10 will be described with reference to Fig. 6. Fig. 6 is a flowchart showing an example of the flow of processing performed by the surface inspecting apparatus of the embodiment.

[0062] The image acquisition unit 51 captures an image of the subject 12 (step S11).

[0063] The marker extraction unit 52 extracts the tag 14 from the captured input image I(x, y) (step S12). At this time, the marker extraction unit 52 reads information registered in the tag 14.

[0064] The marker extraction unit 52 determines whether the orientation of the extracted tag 14 is correct, i.e., whether the tag 14 is photographed horizontally (step S13). If it is determined that the orientation of the extracted tag 14 is correct (step S13: Yes), the process proceeds to step S14. On the other hand, if it is not determined that the orientation of the extracted tag 14 is correct (step S13: No), the process returns to step S11 and the subject 12 is photographed again.

[0065] If it is determined in step S13 that the orientation of the extracted tag 14 is correct, the image acquisition unit 51 stores the captured input image I(x, y) in, for example, the storage unit 32 (step S14).

[0066] The region dividing unit 53 divides the input image I(x, y) into a plurality of small regions each having the same size as the tag 14 extracted from the input image I(x, y) (step S15).

[0067] Furthermore, the region dividing unit 53 performs predetermined scaling on the small regions divided in step S15 (step S16). By scaling, a divided image J(x, y) of a preset size, such as 128×128 pixels, is obtained.

[0068] The digitization processing unit 54 calculates the likelihood of each pixel of the divided image J(x, y) being a convex portion or a concave portion (step S17). By the processing of step S17, an evaluation image K(x, y) is generated from the divided image J(x, y).

[0069] The binarization processing unit 55 generates a binary image L(x, y) from the evaluation image K(x, y) generated by the digitization processing unit 54, in which convex portions are black and concave portions are white (step S18).

[0070] The evaluation result output unit 56 outputs, for example, the binary image L(x, y) generated in step S18, the ratio of convex portions in the binary image L(x, y), the number of convex portions, etc. to the display device 34 (step S19). Although not shown in the flowchart, the output information is stored in the evaluation data 32c as appropriate.

[0071] The evaluation result output unit 56 determines whether all the divided images J(x, y) have been processed (step S20). If it is determined that all the divided images J(x, y) have been processed (step S20: Yes), the surface inspection device 10 ends the processing of Fig. 6. On the other hand, if it is not determined that all the divided images J(x, y) have been processed (step S20: No), the process returns to step S16 to process the next divided image J(x, y).

[0072] (Modification of the embodiment) In the above-described embodiment, the processing from step S14 onwards shown in Figure 6 is performed only when the orientation of the tag 14 in the image of the subject 12 acquired by the image acquisition unit 51 is correct (appears horizontally).

[0073] In this step, if the orientation of the tag 14 is not horizontal, the acquired input image I(x, y) may be rotated and corrected so that the tag 14 is displayed horizontally, and then the processing from step S14 onward shown in Fig. 6 may be performed on the image whose orientation has been corrected. This can save the effort of changing the orientation of the surface inspection device 10 (portable terminal 18) and capturing images of the specimen 12 multiple times. (Effects of this embodiment)

[0074] As described above, the surface inspection device 10 of this embodiment quantitatively evaluates the distribution of unevenness on the surface of the specimen 12, and includes an image acquisition unit 51 that acquires an image of the surface of the specimen 12, a region division unit 53 that divides the acquired image into a plurality of small regions, and a digitization processing unit 54 that converts the pixel value of each pixel in a small region into numerical data indicating the likelihood of the pixel being a convex or concave, based on a network N (learning model) that has learned the likelihood of the unevenness in the small region being a convex or concave. Therefore, quantitative evaluation of the surface condition of the specimen 12 can be performed without variation in results depending on the inspector.

[0075] The surface inspection device 10 of this embodiment also includes a binarization processing unit 55 that identifies the positions of convex and concave portions by binarizing the numerical data converted by the digitization processing unit 54 into a numerical value that represents a predetermined likelihood of a convex portion or a concave portion. This makes it possible to obtain information that makes it easy for an inspector to visually confirm the validity of the evaluation results.

[0076] Furthermore, in the surface inspection device 10 of this embodiment, the image acquisition unit 51 acquires an image of the surface of the specimen 12 on the condition that the tag 14 (marker) attached near the specimen 12 is captured in a predetermined orientation. Therefore, the specimen 12 can always be imaged in the same orientation, and the surface condition of the specimen 12 can be evaluated under the same conditions.

[0077] Furthermore, in the surface inspection device 10 of this embodiment, the region dividing unit 53 divides the image into small regions each having the size of the tag 14. Therefore, the size of the divided images can always be made constant for different specimens 12.

[0078] Furthermore, in the surface inspection apparatus 10 of this embodiment, the network N (learning model) is a model that has learned the rules for classifying the brightness distribution on the surface of the test object 12 into regions of convexities and regions of concaveities by associating the image of the surface of the test object 12 with the results of classifying convexities and concaveities in the image. Therefore, it is possible to prevent variations in the evaluation results depending on the inspector.

[0079] Furthermore, in the surface inspection device 10 of this embodiment, the network N (learning model) is described by a neural network. Therefore, by performing learning in advance, it is possible to evaluate the surface state of the inspection object 12 with high reliability.

[0080] Furthermore, in the surface inspecting apparatus 10 of this embodiment, the surface of the inspection object 12 is subjected to a scraping process. Therefore, the surface inspecting apparatus 10 can quantitatively evaluate the quality of the scraping process on the surface of the inspection object 12.

[0081] In this embodiment, an example has been described in which the surface inspection device 10 evaluates the surface condition of an inspection object 12 that has been subjected to scraping. However, the scope of application of the surface inspection device 10 is not limited to this. In other words, the surface inspection device 10 of this embodiment can also be applied to evaluating the surface condition of an inspection object 12 that has been subjected to processing other than scraping.

[0082] Furthermore, in this embodiment, the network N evaluates the likelihood of a convex portion and a concave portion of the object 12, but the network N may also evaluate the likelihood of a convex portion, a concave portion, and a region other than the object 12. This makes it possible to omit the process of extracting only the region of the object 12.

[0083] Specifically, the network N described in Fig. 4 is modified so that the output layer L3 has three nodes. The output layer L3 then outputs three types of information: a convexity likelihood (1,0,0), a concavity likelihood (0,1,0), and a region other than the subject 12 (0,0,1). This allows the modified network N to recognize regions other than the subject 12, and therefore, to exclude regions other than the subject 12 from the evaluation results.

[0084] Although the embodiments of the present invention have been described above, the above-described embodiments are presented as examples and are not intended to limit the scope of the present invention. This novel embodiment can be embodied in various other forms. Furthermore, various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. Furthermore, this embodiment is included within the scope and spirit of the invention, and is also included in the inventions and their equivalents as defined in the claims. [Explanation of symbols]

[0085] 10 Surface inspection equipment 12 Subject 14 Tags (markers) 16. Lighting 18 Mobile devices 20 Camera 22 Inspector 32b Training data 32c evaluation data 51 Image acquisition unit 52 Marker Extraction Unit 53 Area division part 54 Digitization processing section 55 Binarization processing section 56 Evaluation result output section I(x,y) Input image (image) J(x,y) divided image K(x,y) evaluation image L(x,y) binary image M feature map N network (learning model)

Claims

1. A surface inspection apparatus for quantitatively evaluating a distribution state of projections and recesses on a surface of a test object, comprising: an image acquisition unit for acquiring an image of a surface of the subject; A region dividing unit that divides the acquired image into a plurality of small regions; a digitization processing unit that converts the pixel values ​​of each pixel in the small region into numerical data indicating the likelihood of the pixel being a convex portion or a concave portion, based on a learning model that has learned a rule for discriminating the brightness distribution of the surface of the object into convex portions and concave portions by correlating an image of the surface of the object with a result of discriminating convex portions and concave portions in the image; A surface inspection device comprising:

2. a binarization processing unit that identifies positions of convex portions and concave portions by binarizing the numerical data converted by the digitization processing unit into a numerical value that represents a predetermined convex portion-likeness or concave portion-likeness, The surface inspection apparatus according to claim 1 .

3. the image acquisition unit acquires an image of a surface of the subject on the condition that a marker attached near the subject is captured in a predetermined orientation; 3. A surface inspection device according to claim 1 or 2.

4. The region dividing unit divides the image into small regions each having a size of the marker. The surface inspection apparatus according to claim 3.

5. The learning model is described by a neural network. The surface inspection apparatus according to claim 1 .

6. The subject has a surface that has been subjected to a scraping process. The surface inspection device according to claim 1 or 2.

Images