METHOD AND DEVICE FOR DETECTING SURFACE DEFECTs

Description

BACKGROUND1. Field of the Invention

[0001] The present invention relates to a method and apparatus for detecting a surface defect.2. Discussion of Related Art

[0002] With the recent development of artificial intelligence, the number of intelligent factories incorporating artificial intelligence into a manufacturing process is increasing. Image classification using deep learning as one artificial intelligence technology is used to classify defects that occur in the manufacturing process, with which conventional defect classification tasks performed by human is replaced and which contributes to process efficiency and quality advancement.

[0003] Meanwhile, various shapes and types of defects may occur in cold-rolled steel sheets as they are subjected to rolling and recoiling processes. Since the manufacturing process varies depending on the use and quality of products, the distribution of defect characteristics becomes more diverse. Since specific defects of products may cause fatal defects of products during product manufacturing, a cold-rolled steel surface defect detector (SDD) is introduced to detect defects for quality inspection, and recently, a method of detecting defects using the SDD and classifying the defects using deep learning-based models has been researched.

[0004] However, since the defect images detected by the SDD have various sizes and aspect ratios, it is necessary to adjust the defect images to a standardized model. In general, in order to process data into an image of a standardized size, a crop preprocessing method of cutting out some areas of the image, a resize preprocessing method of adjusting the size of the image into the standardized model, and a padding preprocessing method of setting a padding value set to an edge of the image are used.

[0005] In the case of such general preprocessing methods, image features may be lost, which causes a problem that the reliability of the deep learning-based classification models is degraded. In particular, since defect images generated from various manufacturing processes according to different uses and required quality have a more diverse distribution of defect characteristics, such as shapes and types of the defects, there is a need for the development for a technology capable of improving the defect classification performance for the defect images.

[0006] Document US 2020 / 0 160 083 A1 discloses a computer implemented method for surface defect inspection. The method includes recording an optical image of a surface including a defect, converting the optical image including the defect into a heat map, extracting a region of interest including the defect from the heat map, and comparing the region of interest including the defect from the heat map to a binary classification model using a sliding window based voting mechanism to determine if the defect is greater than or less than a threshold failure value.

[0007] Document US 2020 / 0 364 906 A1 discloses an image inspection apparatus. A defect candidate portion is extracted based on a pixel value of an input inspection target image. An inspection window is set in a region including the extracted defect candidate portion. An image within the inspection window is input to the classifier, and thus, it is determined whether the inspection target image is classified into a first class or a second class.

[0008] Document CN 114 170 208 A discloses a paper product defect detection method based on artificial intelligence. The method comprises the steps: obtaining a preprocessed paper product RGB image; converting the preprocessed RGB image of the paper product into a binary defect image; obtaining an area index of each defect region; obtaining a defect RGB image; obtaining a defect difference index of each defect region; obtaining a defect distribution index of each defect area; the defect degree of the paper product is calculated through the area index, the defect difference index and the defect distribution index, and whether the paper product is qualified or not is judged according to the defect degree of the paper product.

[0009] Document CN 111 242 899 A discloses an image-based defect detection method and computer readable storage medium. The image defect detection method comprises the following steps: acquiring at least one image of a to-be-detected object; extracting a plurality of target defect sub-regions from the image; judging defect types of the plurality of target defect sub-regions by utilizing a first processing method; generating at least one target defect area from the plurality of target defect sub-areas by using a second processing method; judging a first defect level of the target defect area according to a first criterion; and storing the first defect level. A first processing method is used to determine defect types of a plurality of target defect subareas extracted from an image of a to-be-detected object; and at least one target defect area is generated from the plurality of target defect subareas by using a second processing method, and the first defect level of the target defect area is judged according to a first criterion, so as to judge the defects of the to-be-detected object.SUMMARY OF THE INVENTION

[0010] The present invention is directed to providing a method and apparatus for detecting a surface defect.

[0011] To this end, the present invention provides a method of detecting a surface defect in accordance with claim 1, a computer program stored in a recording medium in accordance with claim 9, and an apparatus for detecting a surface defect in accordance with claim 10. According to the present invention, the method includes acquiring a target image, detecting at least one defect area from the target image, generating a first defect matrix having a size corresponding to the target image and an element that is a first defect score calculated based on the number of defect areas, calculating a plurality of second defect scores by summing first defect scores in a range corresponding to a predetermined reference window in the first defect matrix, and extracting an image corresponding to the reference window from the target image based on the second defect score and generating a defect image.

[0012] In addition, the method may further include inputting the defect image as input data into a network function and generating surface defect information, wherein the network function may be trained to output the surface defect information using an image with a size corresponding to the reference window as input data.

[0013] In addition, the element of the first defect matrix is determined according to the number of defect areas detected from the target image at a position corresponding to the element.

[0014] In addition, the calculating of the second defect score is performed by calculating the second defect score, which is the sum of the first defect scores of the first defect matrix corresponding to the predetermined reference window, while moving the reference window with respect to the first defect matrix, and the generating of the defect image may be performed by extracting an image corresponding to a position of the reference window with the largest second defect score from the target image.

[0015] In addition, the method may further include generating a second defect matrix by applying a standard deviation for each local area of the target image to the first defect matrix, wherein the calculating of the second defect score may be performed on the second defect matrix with which the first defect matrix is replaced.

[0016] In addition, the generating of the second defect matrix may include generating a standard deviation matrix by calculating the standard deviation for each local area from the target image and summing weights of the first defect matrix and the standard deviation matrix.

[0017] In addition, the method may further include generating a third defect matrix by removing an element corresponding to a non-defect area from the second defect matrix, wherein the calculating of the second defect score may be performed on the third defect matrix with which the second defect matrix is replaced.

[0018] In addition, the generating of the third defect matrix may include generating a non-defect area having a size corresponding to the target image and representing at least one of background and padding from the target image, and performing an AND operation on the non-defect matrix and the first defect matrix.

[0019] In addition, the detecting of the defect area may include generating boundary value information from the target image, generating boundary direction information from the target image, selecting information on a boundary value greater than or equal to a predetermined critical value based on the boundary value information and the boundary direction information, and detecting a defect area based on the selected boundary value information and the corresponding boundary direction information.

[0020] In addition, operations from the detecting of the defect area to the generating of the defect image may be performed when at least one of horizontal and vertical sizes of the target image is larger than that of the reference window.

[0021] In addition, the method may further include adding padding to a periphery of the target image so that the target image corresponds to the reference window when at least one of horizontal and vertical sizes of the target image is smaller than that of the reference window.

[0022] According to another aspect of the present invention, there is provided a computer program. The program may be stored in a recording medium to execute the method of detecting the surface defect according to the embodiment of the present invention.

[0023] According to yet another aspect of the present invention, there is provided an apparatus for detecting a surface defect. The apparatus includes a memory in which a program for detecting a surface defect is stored, and a processor configured to acquire a target image, detect at least one defect area from the target image, generate a first defect matrix having a size corresponding to the target image and an element that is a first defect score calculated based on the number of defect areas, calculate a plurality of second defect scores by summing first defect scores in a range corresponding to a predetermined reference window in the first defect matrix, and extract an image corresponding to the reference window from the target image based on the second defect score to generate a defect image by.

[0024] In addition, the processor may generate a second defect matrix by applying a standard deviation for each local area of the target image to the first defect matrix, and calculate the second defect score using the second defect matrix with which the first defect matrix is replaced.

[0025] In addition, the processor may generate a third defect matrix by removing an element corresponding to a non-defect area from the second defect matrix, and calculate the second defect score using the third defect matrix with which the second defect matrix is replaced.BRIEF DESCRIPTION OF THE DRAWINGS

[0026] A brief description of each drawing will be made for better understanding for the accompanying drawings cited in the present invention. FIG. 1 is a flowchart of a method of detecting a defect according to an embodiment of the present invention. FIG. 2 is a block diagram of an apparatus for detecting a defect according to the embodiment of the present invention. FIG. 3 shows exemplary views of target images on which defects are formed. FIGS. 4 and 5 are exemplary views of image preprocessing for detecting a defect according to the embodiment of the present invention. FIG. 6 is a view for describing a process of detecting a defect according to the embodiment of the present invention. DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

[0027] Since the present invention may have various changes and various embodiments, specific embodiments are illustrated in the accompanying drawings and will be described in detail. However, it should be understood that it is not intended to limit the scope of the present invention to specific embodiments.

[0028] In describing the present invention, when it is determined that the detailed description of a related known technology may unnecessarily obscure the present invention, detailed description thereof will be omitted.

[0029] The terms used in the present specification are only used to describe embodiments and are not intended to restrict and / or limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. In addition, numbers (e.g., first and second) used in the present specification are merely identifiers for distinguishing one component from another component.

[0030] Throughout the present specification, when a first component is described as being "connected to" a second component, this includes not only a case in which the first component is "directly connected" to the second component, but also a case in which the first component is "indirectly connected" to the second component with a third component interposed therebetween. In addition, when a certain portion is described as "including," a certain component, this means further including other components rather than precluding other components unless especially stated otherwise.

[0031] In addition, in the present disclosure, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or." In other words, unless otherwise specified or clear from the context, "X uses A or B" is intended to mean one of the natural inclusive substitutions. In other words, when X uses A, X uses B, or X uses both A and B, "X uses A or B" can be applied to any of the above cases. In addition, it should be understood that the term "and / or" as used in the present specification should be understood to indicate and include all possible combinations of one or more of the listed related items.

[0032] In addition, terms such as "unit," "instrument," "part," and "module" described in the present disclosure refer to a unit that processes at least one function or operation, which can be implemented by hardware such as a processor, a micro processor, a micro controller, a central processing unit (CPU), a graphics processing unit (GPU), an accelerate processor unit (APU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA), software, or a combination of hardware and software.

[0033] In addition, it should be noted that components in the present disclosure are merely distinguished according to a main function of which each component is in charge. In other words, two or more components to be described below may be combined into one component, or one component may be divided into two or more components for each more detailed function. In addition, it goes without saying that each component to be described below may additionally perform some or all of functions of which other components are in charge in addition to the main function of which each component is in charge, and some of the main functions of which each component is in charge may be performed by being dedicated by other components.

[0034] A method according to an embodiment of the present invention may be performed on a personal computer, work station, or server computer device provided with computational ability or performed on a separate device for this purpose.

[0035] In addition, the method may be performed on one or more computational devices. For example, at least one or more operations of the method according to the embodiment of the present invention may be performed in a client device, and other operations may be performed in a server device. In this case, the client device and the server device may be connected via a network and may transmit and receive a result of computation. Alternatively, the method may be performed by a distributed computing technology.

[0036] Throughout the present specification, a network function may be used interchangeably with a computational model, a neural network, or a neural network. The neural network may be generally composed of a set of interconnected computational units, which can be referred to as nodes. These nodes may be referred to as neurons. The neural network may be configured to include at least one node. Nodes (or neurons) constituting the neural network may be interconnected by one or more links.

[0037] In the neural network, one or more nodes connected through the links may form the relative relationship between an input node and an output node. The concept of the input node and the output node is relative, and any node in an output node relationship with respect to one node may be in an input node relationship with respect to another node, and vice versa. As described above, the relationship between the input node and the output node may be generated with respect to the links. One or more output nodes may be connected to one input node through the links, and vice versa.

[0038] In the relationship between the input node and the output node connected through one link, a value of data of the output node may be determined based on data input to the input node. Here, the link interconnecting the input node and the output node may have a weight. The weight may be variable and may be changed by the user or algorithm in order for a neural network to perform a desired function. For example, when one or more input nodes are interconnected to one output node by respective links, the output node may determine an output node value based on values input to input nodes connected to the output node and a weight set to a link corresponding to each input node.

[0039] A subset of the nodes constituting the neural network may form a layer. Some of the nodes constituting the neural network may form one layer based on distances from an initial input node. For example, a set of nodes with a distance n from the initial input node may constitute n layers. The distance from the initial input node can be defined by the minimum number of links that should be passed to reach the corresponding node from the initial input node. However, the definition of a layer is arbitrary for description purposes, and an order of a layer in a neural network can be defined in a different manner from the above description. For example, layers of nodes can be defined by a distance from a final output node.

[0040] The neural network may include a deep neural network (DNN) that includes a plurality of hidden layers in addition to an input layer and an output layer. The deep neural network may be used to identify latent structures of data. The deep neural network may include a convolutional neural network (CNN), a recurrent neural network (RNN), an auto encoder, a generative adversarial network (GAN), a restricted Boltzmann machines (RBM), a deep trust network (DBN), a Q network, a U network, a Siamese network, a generative adversarial network (GAN), and the like. The description of the above-described deep neural network is only an example, and the present invention is not limited thereto.

[0041] The neural network may be trained by at least one way of supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. The training of the neural network may be a process of applying knowledge for the neural network to perform a specific operation to the neural network.

[0042] Hereinafter, embodiments of the present invention will be sequentially described in detail.

[0043] FIG. 1 is a flowchart of a method of detecting a defect according to an embodiment of the present invention.

[0044] In operation S110, a target image is acquired. Here, the target image may be a surface image of an object from which defects will be detected.

[0045] For example, the target image may be an image about a surface of a cold-rolled steel sheet. In the case of the cold-rolled steel sheet, a pickling process, which is a surface treatment process for removing fine metal contaminants or cleaning a surface by immersing the cold-rolled steel sheet in a strong acid solution, is applied. Therefore, in the case of some alloys or high carbon steels, a hydrogen embrittlement problem may occur, and hydrogen generated from acids may react on a surface to cause defects in a metal. In addition, since the pickling process is a first process of the manufacturing process of the cold-rolled steel sheet and defects occurring during a conveying or manufacturing process of a coil may be detected complexly, the types of defects found in the pickling process are diverse as compared to the types of defects found in other processes. Since sizes of defects vary by up to 5,000 times and a size of a coil used in a cold-rolling process is always constant, absolute sizes of the defects that occur are very important in classifying the defects.

[0046] According to the embodiment, the target image may be received from an external database server or acquired through photographing from a photographing device (e.g., a surface defect detector (SDD)) connected to an apparatus for detecting a defect via wired or wireless communication. However, the present invention is not limited thereto.

[0047] In operation S120, at least one defect area is detected from the target image. Here, the defect area may be an area including defects formed on a surface and / or an area expected to include the defects formed on the surface. According to the embodiment, the defect area may include only an area corresponding to a defect or further include an area adjacent to the defect. For example, one defect may belong to a plurality of defect areas.

[0048] In the embodiment, operation S120 may include generating boundary value information from the target image, generating boundary direction information from the target image, selecting information on a boundary value greater than or equal to a predetermined critical value based on the boundary value information and the boundary direction information, and detecting a defect area based on the selected boundary value information and the corresponding boundary direction information.

[0049] In the embodiment, operation S120 may be performed by inputting the target image into a network function. Here, the network function may be trained to output the defect area from the target image.

[0050] In the embodiment, the defect area may have a quadrangular shape, and at least one defect may be positioned or expected to be positioned inside the quadrangular boundary. However, the present invention is not limited thereto.

[0051] In operation S130, a first defect score is calculated from the target image to generate a first defect matrix. Specifically, the first defect matrix is generated by calculating the first defect score based on the number of defect areas detected in operation S120 and using the first defect score as an element of the matrix. In particular, the element of the first defect matrix is determined according to the number of defect areas detected from the target image at a position corresponding to the element.

[0052] For example, when a defect is present at a predetermined pixel position in the target image and a plurality of defect areas with respect to the defect are detected, the first defect score may be calculated based on the number of plurality of defect areas, and the first defect score may be assigned to the corresponding pixel position and the corresponding position in the matrix.

[0053] In the embodiment, the first defect score may be calculated by summing the number of defect areas. However, the present invention is not limited thereto, and the first defect score may be calculated in various ways based on the square of the number of defect areas, deviation, and the like.

[0054] In the embodiment, the first defect matrix may have a size corresponding to the target image. Here, when the first defect matrix has the size corresponding to the target image, it may mean that the number of rows and columns of the first defect matrix corresponds to the number of horizontal pixels and the number of vertical pixels of the target image so that the first defect matrix includes sufficient pieces of information about the target image. In particular, when the numbers correspond to each other, it may mean that the numbers are the same or N or 1 / N times (here, N is a positive integer).

[0055] For example, the number of rows and the number of columns in the first defect matrix may be the same as the number of vertical pixels and the number of horizontal pixels of the target image. For example, when the target image is downsampled and / or upsampled, the first defect matrix may have a size corresponding to the downsampled and / or upsampled target image. However, the present invention is not limited thereto.

[0056] In operation S140, a second defect matrix may be generated. The second defect matrix may be obtained by applying a standard deviation for each local area of the target image to the first defect matrix. When the defect is relatively large, the influence of the defect on the standard deviation increases, and thus it is possible to increase the accuracy of defect detection in consideration of the standard deviation.

[0057] Specifically, operation S140 may include generating a standard deviation matrix by calculating the standard deviation for each local area in the target image, and combining the first defect matrix with the standard deviation matrix.

[0058] Here, the standard deviation matrix may be generated by calculating a standard deviation for an image in a deviation window while moving the deviation window having a predetermined size on the target image and using the calculated standard deviation as an element. In this case, a size of the deviation window may correspond to a reference window. However, the present invention is not limited thereto, and deviation windows with various sizes may be applied. Likewise, a stride of the deviation window may mean M or 1 / M times a size of the reference window (here, M is a positive integer). However, the present invention is not limited thereto.

[0059] In addition, the standard deviation matrix may have the size that is the same as that of the first defect matrix. When the size of the standard deviation matrix is smaller than that of the first defect matrix, a predetermined element may be added to the standard deviation matrix (i.e., rows and / or columns may be increased) so that the size of the standard deviation matrix is the same as that of the first defect matrix. In this case, a value of the element may be the same or similar to a value of an adjacent element. However, the present invention is not limited thereto.

[0060] In the embodiment, the sum of the first defect matrix may be a weighted sum. Specifically, the first defect matrix may be generated by multiplying the first defect matrix by a first weight, multiplying the standard deviation matrix by a second weight, and then summing a result of the multiplication. By adjusting the first weight and the second weight, a proportion of the number and standard deviation of defect areas on defect detection may be adjusted.

[0061] In operation S150, a third defect matrix may be generated. Specifically, the third defect matrix may be generated by removing elements corresponding to non-defect areas from the second defect matrix.

[0062] In the embodiment, operation S150 may include generating a non-defect matrix representing at least one of background and padding from the target image, and performing an AND operation on the non-defect matrix and the second defect matrix.

[0063] Here, the non-defect matrix may have a size corresponding to the target image and have the size that is the same as that of the second defect matrix. In addition, the background is an area excluding defects from the target image and may include, for example, a surface image without defects. The padding may refer to a margin added to an outside of the target image to adjust the size of the target image.

[0064] For example, in the non-defect matrix, elements corresponding to the background and padding may be set to a value of "0," and the other components may be set to a value of "1." Therefore, when the AND operation is performed on the non-defect matrix and the second defect matrix, elements of positions corresponding to the background and padding in the third defect matrix become 0, and the other elements may be the same as the elements of the second defect matrix.

[0065] In the above example, although the element of the non-defect matrix are set to "1" or "0," the present invention is not limited thereto, and various values that may distinguish the elements corresponding to the background and padding from the other elements can be applied.

[0066] In the embodiment, a filter may be applied to the non-defect matrix to remove noise in the non-defect matrix. For example, noise may be removed by applying a Gaussian filter to a non-defect matrix of which an element is set to "1" or "0." Then, all elements other than "1" may be set to "0." However, the present invention is not limited thereto.

[0067] In operation S160, a second defect score is calculated. Specifically, the second defect score is calculated by summing the first defect scores in a range corresponding to the reference window in the third defect matrix.

[0068] Here, the reference window is a virtual area with a predetermined size, and the size of the reference window may correspond to a size of a defect image generated in operation S170. For example, the reference window may be expressed as a matrix of which elements are all "1." However, the present invention is not limited thereto.

[0069] Specifically, operation S160 is performed by summing the first defect scores of the third defect matrix corresponding to the reference window while moving the reference window with respect to the third defect matrix. In operation S160, a plurality of second defect scores are calculated, and a position of the reference window with respect to the third defect matrix may be assigned to each second defect score.

[0070] For example, the second defect score may be calculated by moving the reference window with respect to the third defect matrix, perform the AND operation on the corresponding elements of a matrix of the reference window and the third defect matrix, and then summing a result of the calculation.

[0071] In operation S170, a defect image is generated. The defect image may be an image selected to effectively represent defect characteristics from the target image.

[0072] Specifically, in operation S170, the defect image is generated by extracting an image corresponding to the reference window from the target image based on the second defect score. According to the present invention, operation S170 is performed by extracting an image corresponding to a position of the reference window with the largest second defect score from the target image. In other words, the second defect score with the highest score may be selected, and the defect image may be generated from the target image based on the position of the reference window assigned to the corresponding second defect score.

[0073] In the embodiment, the method 100 may further include inputting a defect image as input data into a network function and generating surface defect information. Here, the network function may be trained to output surface defect information using an image with a size corresponding to the reference window as input data. In addition, the surface defect information may include at least one of information about whether defects are present and defect classification information. The classification information may include, for example, carbon, oil mark, roll mark, rust, scab, scratch, slip mark, and the like. However, the present invention is not limited thereto.

[0074] In the embodiment, operations S120 to S170 may be performed when at least one of the horizontal and vertical sizes of the target image is larger than that of the reference window. In other words, when at least a portion of the size of the target image is larger than that of the reference window, operations S120 to S170 may be performed to generate a defect image by extracting only a portion that best represents the defect characteristics from the target image.

[0075] In the embodiment, the method 100 may further include adding padding to the periphery of the target image so that the target image corresponds to the reference window when at least one of the horizontal and vertical sizes of the target image is smaller than that of the reference window. The adding of the padding may be performed by adding an image corresponding to white, black, or a background color to the periphery of the target image. In this case, the background color may be a color of a background that is a surface image without defects in the target image or a color of a background added during an image processing process involved in the process of generating the target image. However, the present invention is not limited thereto.

[0076] In the embodiment, when a first size that is one of the horizontal and vertical sizes of the target image is smaller than that of the reference window and a second size that is the other is larger than that of the reference window, both the removal of at least a portion of the target image and the addition of padding may be performed.

[0077] Specifically, operations S120 to S170 may be performed to remove at least a portion of the target image. Additionally, adding padding to the target image may be performed. In this case, although addition may be performed first and then removal may be performed, the present invention is not limited thereto, and removal may be performed first and then addition may be performed, or addition and removal may be performed simultaneously. Therefore, the target image in operations S120 to S170 may be a target image without padding added or target image with padding added.

[0078] In the embodiment, at least one operation of method 100 may not be performed. Specifically, at least one of operations S140 and S150 may not be performed.

[0079] For example, operations S140 and S150 may not be performed. In this case, operation S160 may be performed after operation S130, and in operation S160, the second defect score may be calculated by summing the first defect scores in a range corresponding to the reference window in the first defect matrix rather than the third defect matrix.

[0080] In addition, for example, operation S140 may not be performed. Therefore, operation S150 may be performed after operation S130, and in operation S150, the third defect matrix may be generated by removing the element corresponding to the non-defect area from the first defect matrix rather than the second defect matrix.

[0081] In addition, for example, operation S150 may not be performed. In this case, operation S160 may be performed after operation S140, and in operation S160, the second defect score may be calculated by summing the first defect scores in a range corresponding to the reference window in the second defect matrix rather than the third defect matrix.

[0082] The method 100 of FIG. 1 is illustrative, and various configurations can be applied according to embodiments of the present invention.

[0083] FIG. 2 is a block diagram of an apparatus for detecting a defect according to the embodiment of the present invention.

[0084] An apparatus 200 for detecting a defect performs the method 100 of FIG. 1, but is not limited thereto.

[0085] Referring to FIG. 2, the apparatus 200 for detecting the defect includes a memory 230 and a processor 240. The apparatus 200 may further include a communication unit 210 and an input unit 220.

[0086] The communication unit 210 may receive data from the outside. The communication unit 210 may include a wired or wireless communication unit. When the communication unit 210 includes the wired communication unit, the communication unit 210 may include a local area network (LAN), a wide area network (WAN), a value added network (VAN), a mobile radio communication network, a satellite communication network, and one or more components that enable communication through a combination thereof. In addition, when the communication unit 210 includes the wireless communication unit, the communication unit 210 may transmit and receive data or signals wirelessly using cellular communication, wireless LAN (e.g., Wi-Fi), or the like. In the embodiment, the communication unit 210 may transmit and receive data or signals with an external device or external server under the control of the processor 240.

[0087] The input unit 220 may receive various user commands through external manipulation. To this end, the input unit 220 may include or connect one or more input devices. For example, the input unit 220 may be connected to various input interfaces such as a keypad and mouse to receive the user commands. To this end, the input unit 220 may include not only a USB port but also an interface such as Thunderbolt. In addition, the input unit 220 may include and combine various input devices such as a touch screen and button to receive external user commands.

[0088] The memory 230 stores a program and may store program commands for operating the processor 240 and temporarily or permanently store input / output data. The memory 230 may include a flash memory type, hard disk type, multimedia card micro type, or card type memory (e.g., a secure card (SD) or extreme digital (XD) memory), and a storage medium of at least one type of a random access memory (RAM), a static RAM (SRAM), a read-only memory (ROM), an electrically erasable programmable ROM (EEPROM), a programmable ROM (PROM), a magnetic memory, a magnetic disk, and an optical disk.

[0089] In addition, the memory 230 may store various network functions and algorithms and store various data, programs (one or more instructions), applications, software, commands, codes, and the like for driving and controlling the apparatus 200.

[0090] The processor 240 may control the overall operation of the apparatus 200. The processor 240 may execute one or more programs or software stored in the memory 230. The processor 240 may be a central processing unit (CPU), graphics processing unit (GPU), or dedicated processor 240 in which the methods according to the embodiments of the present invention are performed.

[0091] In an aspect of the present invention, the processor 240 acquires the target image, detects at least one defect area from the target image, generates the first defect matrix having the size corresponding to the target image and using the first defect score calculated based on the number of defect areas as the element, calculates the plurality of second defect scores by summing the first defect scores in a range corresponding to the predetermined reference window in the first defect matrix, and generates the defect image by extracting the image corresponding to the reference window from the target image based on the second defect score.

[0092] In the embodiment, the processor 240 may input the defect image as input data into the network function and generate the surface defect information. Here, the network function may be trained to output the surface defect information using the image with the size corresponding to the reference window as input data.

[0093] In the present invention, the processor 240 determines the element of the first defect matrix according to the number of defect areas detected from the target image at the position corresponding to the element.

[0094] In the present invention, the processor 240 calculates the second defect score, which is the sum of the first defect scores of the first defect matrix corresponding to the reference window, while moving the predetermined reference window with respect to the first defect matrix. The processor 240 generates the defect image by extracting the image corresponding to the position of the reference window with the highest second defect score from the target image.

[0095] In the embodiment, the processor 240 may generate the second defect matrix by applying the standard deviation for each local area of the target image to the first defect matrix. The processor 240 may calculate the second defect score using the second defect matrix with which the first defect matrix is replaced.

[0096] In the embodiment, the processor 240 may generate the standard deviation matrix by calculating the standard deviation for each local area in the target image and generate the second defect matrix by summing weights of the first defect matrix and the standard deviation matrix.

[0097] In the embodiment, the processor 240 may generate the third defect matrix by removing the elements corresponding to the non-defect areas from the second defect matrix. The processor 240 may calculate the second defect score using the third defect matrix with which the second defect matrix is replaced.

[0098] In the embodiment, the processor 240 may generate the non-defect matrix having the size corresponding to the target image and representing at least one of the background and padding in the target image and generate the third defect matrix by performing the AND operation on the non-defect matrix and the second defect matrix.

[0099] In the embodiment, the processor 240 may detect the defect area by generating the boundary value information from the target image, generating the boundary direction information from the target image, selecting information on the boundary value greater than or equal to the predetermined critical value based on the boundary value information and the boundary direction information, and detecting the defect area based on the selected boundary value information and the corresponding boundary direction information.

[0100] In the embodiment, the processor 240 may detect the defect area or generate the defect image when at least one of the horizontal and vertical sizes of the target image is larger than that of the reference window.

[0101] In the embodiment, the processor 240 may add padding to the periphery of the target image so that the target image corresponds to the reference window when at least one of the horizontal and vertical sizes of the target image is smaller than that of the reference window.

[0102] The apparatus illustrated in FIG. 2 is illustrative, and various configurations can be applied according to embodiments of the present invention.

[0103] FIG. 3 shows exemplary views of target images on which defects are formed.

[0104] The defects illustrated in FIG. 3 occur on the surface of the cold-rolled steel sheet and may mean carbon, an oil mark, a roll mark, rust, scab, scratch, or slip mark in the order from FIGS. 3A to 3G.

[0105] FIG. 3 is illustrative, and various configurations can be applied according to embodiments of the present invention.

[0106] FIGS. 4 and 5 are exemplary views of image preprocessing for detecting a defect according to the embodiment of the present invention.

[0107] Referring to FIG. 4, when both the horizontal and vertical sizes of the target image are smaller than that of the reference window, a defect image may be generated by adding padding to the periphery of the target image.

[0108] Referring to FIG. 5, a case in which the horizontal size of the target image is smaller than that of the reference window and the vertical size of the target image is larger than that of the reference window is illustrated. In this case, padding may be added to the target image in the horizontal direction, which is smaller than that of the reference window, and the defect image may be generated by extracting the area that best represents the defect characteristics in the vertical direction, which is larger than that of the reference window.

[0109] FIGS. 4 and 5 are illustrative, and various configurations can be applied according to embodiments of the present invention.

[0110] FIG. 6 is a view for describing a process of detecting a defect according to the embodiment of the present invention.

[0111] As illustrated, when a target image with defects formed on a surface is acquired, at least one defect area may be detected by performing boundary detection on the image or the like. A first defect matrix may be generated by calculating a first defect score based on the defect area.

[0112] In addition, a deviation matrix may be generated by calculating a standard deviation for each local area of the target image, and a second defect matrix may be generated by weight-multiplying the deviation matrix and the first defect matrix.

[0113] Although not illustrated, a third defect matrix may be generated by removing the non-defect areas (background and blank space) from the second defect matrix. In the third defect matrix, the second defect score may be calculated while moving the reference window, and based on this, a defect image may be generated from the reference window at a position with the greatest second defect score.

[0114] FIG. 6 is illustrative, and various configurations can be applied according to embodiments of the present invention.

[0115] The method according to the embodiment of the present invention can be implemented in the form of program commands that may be performed through various computer devices and recorded on a computer-readable medium. The computer-readable medium may include program commands, data files, data structures, and the like alone or in combination. The program commands recorded on the medium is designed and constructed for the present invention or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as a hard disk, a floppy disk, and a magnetic tape, optical media such as a CD-ROM and a DVD, and magneto-optical media such as a floptical disk, and hardware devices specifically configured to store and execute program commands, such as a ROM, a RAM, and a flash memory. Examples of the program commands include not only machine language code such as that produced by a compiler, but also high-level language code that may be executed by a computer using an interpreter or the like.

[0116] In addition, the method according to the disclosed embodiments may be provided by being included in a computer program product. The computer program product may be traded between sellers and buyers as commodities.

[0117] The computer program product may include a software program and a computer-readable storage medium in which the software program is stored. For example, the computer program product may include a product in the form of a software program (e.g., a downloadable app) distributed electronically by a manufacturer of an electronic device or through an electronic marketplace (e.g., a Google Play Store or an App Store). For electronic distribution, at least some of the software programs may be stored in a storage medium or temporarily generated. In this case, the storage medium may be a manufacturer's server, an electronic market server, or a relay server's storage medium for temporarily storing SW programs.

[0118] The computer program product may include a storage medium of a server or a storage medium of a client device in a system composed of the server and the client device. Alternatively, when a third device (e.g., a smartphone) communicatively connected to the server or client device is present, the computer program product may include a storage medium of the third device. Alternatively, the computer program product may include a software program, which is transmitted from the server to the client device or third device or from the third device to the client device.

[0119] In this case, one of the server, the client device, and the third device may execute the computer program product to perform the method according to the disclosed embodiments. Alternatively, two or more of the server, the client device, and the third device may execute the computer program product to perform the method according to the disclosed embodiments in a distributed manner.

[0120] For example, the server (e.g., a cloud server or artificial intelligence server) may execute the computer program product stored on the server and control the client device communicatively connected to the server to perform the method according to the disclosed embodiments.

[0121] According to the present invention, it is possible to classify defects of target images having various sizes and aspect ratios.

[0122] In addition, it is possible to increase classification accuracy by minimizing feature losses of various types of surface defect images.

[0123] In addition, it is possible to prevent increase or decrease in feature information by preventing the resizing of the target images.

Claims

1. A computer-implemented method of detecting a surface defect, comprising: acquiring (S110) a target image; detecting (S120) at least one defect area from the target image; generating (S130) a first defect matrix having a size corresponding to the target image, wherein the first defect matrix has a plurality of elements, wherein each element is a first defect score calculated based on the number of defect areas detected from the target image at a position corresponding to the element; calculating (S160) a plurality of second defect scores, wherein each second defect score is the sum of the first defect scores of the first defect matrix corresponding to a reference window, while moving the reference window with respect to the first defect matrix, wherein the reference window is a virtual area with a predetermined size; and generating (S170) a defect image by extracting an image corresponding to a position of the reference window with the largest second defect score from the target image.

2. The method of claim 1, further comprising inputting the defect image as input data into a network function and generating surface defect information, wherein the network function is trained to output the surface defect information using an image with a size corresponding to the reference window as input data.

3. The method of claim 1, further comprising generating (S140) a second defect matrix by applying a standard deviation for each local area of the target image to the first defect matrix, wherein the calculating of the second defect score is performed on the second defect matrix with which the first defect matrix is replaced, wherein the standard deviation is applied by generating a standard deviation matrix by calculating the standard deviation for each local area in the target image, and combining the first defect matrix with the standard deviation matrix.

4. The method of claim 3, wherein the generating (S140) of the second defect matrix includes summing weights of the first defect matrix and the standard deviation matrix, wherein the weights are summed by multiplying the first defect matrix by a first weight, multiplying the standard deviation matrix by a second weight, and then summing a result of the multiplication.

5. The method of claim 3, further comprising generating (S150) a third defect matrix by removing an element corresponding to a non-defect area from the second defect matrix, wherein the calculating of the second defect score is performed on the third defect matrix with which the second defect matrix is replaced, wherein the removing the element corresponding to the non-defect area includes generating a non-defect matrix representing at least one of background and padding from the target image, and performing an AND operation on the non-defect matrix and the second defect matrix.

6. The method of claim 3, wherein the detecting (S120) of the defect area includes generating boundary value information from the target image, generating boundary direction information from the target image, selecting information on a boundary value greater than or equal to a predetermined critical value based on the boundary value information and the boundary direction information, and detecting a defect area based on the selected boundary value information and the corresponding boundary direction information.

7. The method of claim 1, wherein operations from the detecting (S120) of the defect area to the generating (S170) of the defect image are performed when at least one of horizontal and vertical sizes of the target image is larger than that of the reference window.

8. The method of claim 1, further comprising adding padding to a periphery of the target image so that the target image corresponds to the reference window when at least one of horizontal and vertical sizes of the target image is smaller than that of the reference window.

9. A computer program stored in a recording medium to execute the method of any one of claims 1 to 8.

10. An apparatus for detecting a surface defect, comprising: a memory (230) in which a program for detecting a surface defect is stored; and a processor (240) configured to acquire a target image, detect at least one defect area from the target image, generate a first defect matrix having a size corresponding to the target image, wherein the first defect matrix has a plurality of elements, wherein each element is a first defect score calculated based on the number of defect areas detected from the target image at a position corresponding to the element, calculate a plurality of second defect scores, wherein each second defect score is the sum of the first defect scores of the first defect matrix corresponding to a reference window, while moving the reference window with respect to the first defect matrix, wherein the reference window is a virtual area with a predetermined size, and generate a defect image by extracting an image corresponding to a position of the reference window with the largest second defect score from the target image.

11. The apparatus of claim 10, wherein the processor (240) generates a second defect matrix by applying a standard deviation for each local area of the target image to the first defect matrix, and calculates the second defect score using the second defect matrix with which the first defect matrix is replaced, wherein the standard deviation is applied by generating a standard deviation matrix by calculating the standard deviation for each local area in the target image, and combining the first defect matrix with the standard deviation matrix.

12. The apparatus of claim 11, wherein the processor (240) generates a third defect matrix by removing an element corresponding to a non-defect area from the second defect matrix, and calculates the second defect score using the third defect matrix with which the second defect matrix is replaced, wherein the generating of the third defect matrix includes generating a non-defect matrix having a size corresponding to the target image and representing at least one of background and padding from the target image, and performing an AND operation on the non-defect matrix and the second defect matrix to remove the element corresponding to the non-defect area.