Method for detecting surface defect and apparatus thereof

Abstract

To provide a method and apparatus for detecting a surface defect, and a program configured to classify defects of target images having various sizes and aspect ratios.SOLUTION: A method includes the steps of: acquiring a target image; detecting at least one defect area from the target image; generating a first defect matrix having a size corresponding to the target image and an element that is a first defect score calculated based on the number of defect areas; calculating a plurality of second defect scores by summing first defect scores in a range corresponding to a predetermined reference window in the first defect matrix; and extracting an image corresponding to the reference window from the target image based on the second defect score and generating a defect image.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] This application relates to a surface defect detection method and apparatus.

Background Art

[0002] In recent years, with the development of artificial intelligence, the number of intelligent factories that integrate artificial intelligence into the manufacturing process has been increasing. Among these, video classification using deep learning, one of them, is utilized to classify defects occurring in the manufacturing process, replacing the conventional manual defect classification work and contributing to process efficiency improvement and quality enhancement.

[0003] On the other hand, various forms and types of defects can occur in cold-rolled steel sheets through the rolling and rolling processes. Since the production process varies depending on the application and quality, the characteristic distribution of defects appears more diversely. Specific defects may cause fatal defects during product manufacturing. Therefore, a cold-rolled steel sheet surface defect detector ( SDD ) has been introduced to detect defects, and recently, a method of detecting defects using a surface defect detector and classifying them into a deep learning-based model has been studied.

[0004] However, since the defect images detected by the surface defect detector have various sizes and ratios, it is necessary to adjust them to a standardized model. Generally, a Crop preprocessing method of cutting out a part of the image area to process data into an image of a standardized size, a Resize preprocessing method of adjusting the size of the image to a standardized model, and a Padding preprocessing method of setting padding values in the image frame are used.

[0005] In such a general preprocessing method, the characteristics of the video may be lost, which causes a problem of reducing the reliability of the deep learning-based classification model. In particular, defective images generated from processes with different production processes according to different uses and required qualities show a more diverse distribution of defect characteristics such as the form and type of defects. Therefore, there is a need to develop a technology for improving the defect classification performance for such defective images. Summary of the Invention Problems to be Solved by the Invention

[0006] An object of the present application is to provide a surface defect detection method and apparatus. Means for Solving the Problems

[0007] According to an embodiment of the present application, a surface defect detection method is provided. The method may include: acquiring a target image; detecting at least one defect region from the target image; generating a first defect matrix having a size corresponding to the target image and having a first defect score calculated based on the number of the defect regions as a component (Element); calculating a plurality of second defect scores by summing the first defect scores within a range corresponding to a predetermined reference window in the first defect matrix; and extracting an image corresponding to the reference window from the target image based on the second defect scores to generate a defect image.

[0008] Further, as input data, the method may further include inputting the defect image into a network function to generate surface defect information, and the network function may be learned to output the surface defect information using an image having a size corresponding to the reference window as the input data.

[0009] Further, the component of the first defect matrix may be determined by the number of defect regions detected from the target image at a position corresponding to the component.

[0010] Further, the step of calculating the second defect score is performed by calculating a second defect score, which is the sum of the first defect scores of the first defect matrix corresponding to the reference window, while moving a predetermined reference window with respect to the first defect matrix. The step of generating the defect image may be performed by extracting an image corresponding to the position of the reference window with the largest second defect score from the target image.

[0011] It further includes a step of generating a second defect matrix by applying the standard deviation of each local region of the target image to the first defect matrix, and the step of calculating the second defect score may be performed for the second defect matrix by replacing the first defect matrix.

[0012] Further, the step of generating the second defect matrix may include a step of calculating the standard deviation for each local region from the target image to generate a standard deviation matrix, and a step of performing a weighted sum of the first defect matrix and the standard deviation matrix.

[0013] It further includes a step of generating a third defect matrix by removing components corresponding to non-defective regions in the second defect matrix, and the step of calculating the second defect score may be performed for the third defect matrix by replacing the second defect matrix.

[0014] Further, the step of generating the third defect matrix may include a step of generating a non-defect matrix having a size corresponding to the target image and representing at least one of the background and padding in the target image, and a step of performing a logical AND operation on the non-defect matrix and the first defect matrix.

[0015] Further, the step of detecting the defective region may include: generating boundary line value information in the target image; generating boundary line direction information in the target image; selecting boundary line value information greater than or equal to a predetermined threshold based on the boundary line value information and the boundary line direction information; and detecting a defective region based on the selected boundary line value information and the corresponding boundary line direction information.

[0016] Further, the steps of detecting the defective region to generating the defective image may be performed when at least one of the horizontal size and the vertical size of the target image is larger than the reference window.

[0017] Further, when at least one of the horizontal size and the vertical size of the target image is smaller than the reference window, the method may further include adding padding around the target image so that the target image corresponds to the reference window.

[0018] According to an embodiment of the present application, a computer program is provided. The program can be stored in a recording medium to execute the surface defect detection method according to the embodiment of the present application.

[0019] According to an embodiment of the present application, an apparatus for detecting surface defects is provided. The apparatus may include: a memory storing a program for detecting surface defects; a processor configured to obtain a target image, detect at least one defective region from the target image, generate a first defect matrix having a size corresponding to the target image and including a first defect score calculated based on the number of defective regions as a component, calculate a plurality of second defect scores by summing the first defect scores within a range corresponding to a predetermined reference window in the first defect matrix, and extract an image corresponding to the reference window from the target image based on the second defect scores to generate a defective image.

[0020] Further, the processor can apply the standard deviation for each local region of the target image to the first defect matrix to generate a second defect matrix, replace the first defect matrix, and calculate a second defect score for the second defect matrix.

[0021] Further, the processor can generate a third defect matrix by removing components corresponding to non-defect regions from the second defect matrix, replace the second defect matrix, and calculate a second defect score for the third defect matrix.

Advantages of the Invention

[0022] According to the embodiments of the present application, defect classification for target images of various sizes and ratios is enabled.

[0023] Also, characteristic loss can be minimized and classification accuracy can be improved with various types of surface defect images.

[0024] Also, expansion or contraction of characteristic information can be prevented without resizing the target image.

[0025] The effects obtained from the embodiments of the application are not limited to the effects described above, and other effects not mentioned can be clearly understood by those with ordinary knowledge in the technical field to which the present application belongs from the following description. To more fully understand the drawings cited in the present application, a brief description of each drawing is provided.

Brief Description of the Drawings

[0026]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Embodiments for Carrying Out the Invention

[0027] Since the technical idea of the present application can be modified in various ways and can have various embodiments, specific embodiments are illustrated in the drawings and will be described in detail. However, this is not intended to limit the technical idea of the present application to specific embodiments, but includes all modifications, equivalents, or alternatives included in the scope of the technical idea of the present application.

[0028] In explaining the technical idea of the present application, when it is determined that a specific description of related known technologies may unnecessarily obscure the gist of the present application, the detailed description thereof will be omitted.

[0029] The terms used in this specification are used for explaining embodiments and are not intended to limit and / or restrict the present application. Singular expressions include plural expressions unless the context clearly has a different meaning. Also, the numbers used in this specification (for example, the first, the second, etc.) are merely identification symbols for distinguishing one component from other components.

[0030] In this specification, when it is said that a certain part is connected to another part, this includes not only the case where they are directly connected but also the case where they are indirectly connected with other configurations interposed therebetween. Also, when it is said that a certain part includes a certain component, this means that, unless otherwise stated to the contrary, it does not exclude other components and may further include other components.

[0031] Also, the term "or" in this application is intended to mean an inclusive "or" rather than an exclusive "or". That is, unless otherwise specified or unclear from the context, it is intended to mean either of the natural inclusive substitutions of "X uses A or B". That is, the above "X uses A or B" applies to any of the following cases: when X uses A, when X uses B, or when X uses both A and B. Also, the term "and / or" used in this specification refers to and includes all possible combinations of one or more of the listed related components.

[0032] Also, terms such as "~ part", "~ machine", "~ character", "~ module" described in this application mean a unit that processes at least one function or operation, which can be implemented by hardware and software such as a processor, microprocessor, microcontroller, CPU (Central Processing Unit), GPU (Graphics Processing Unit), APU (Accelerate Processor Unit), DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array), or a combination of hardware and software.

[0033] And it should be made clear that the classification of the components in this application is only a classification according to the main functions each component is responsible for. That is, two or more components described below may be combined into one component, or one component may be divided into two or more components according to more refined functions. And it goes without saying that each of the components described below may additionally perform some or all of the functions of other components in addition to the main function it is responsible for, and some of the functions of the main function each component is responsible for may be solely performed by other components.

[0034] The method according to the embodiments of the present application can be performed on a personal computer, a work station, a server computer device, etc. having computing capabilities, or on another device therefor.

[0035] Also, the method can be performed on one or more computing devices. For example, at least one or more steps of the method according to the embodiments of the present application can be performed on a client device, and other steps can be performed on a server device. In this case, the client device and the server device can be connected to a network to transmit and receive calculation results. Alternatively, the method can be performed by distributed computing technology.

[0036] In this specification, the network function can be used in the same meaning as a calculation model, a neural network. A neural network can generally be composed of a set of interconnected computing units called nodes. These nodes can be called neurons. A neural network is composed of including at least one or more nodes. The nodes (or neurons) constituting the neural network can be interconnected by one or more links.

[0037] Within a neural network, one or more nodes connected via a link can form a relationship with respect to input nodes and output nodes relatively. The concepts of input nodes and output nodes are relative. Any node that is in the relationship of an output node with respect to one node can be in the relationship of an input node with respect to another node, and vice versa can also hold. As described above, the relationship between input nodes and output nodes can be generated centering on the link. One or more output nodes can be connected to one input node via a link, and vice versa can also hold.

[0038] In the relationship between an input node and an output node connected via a link, the data of the output node can have its value determined based on the data input to the input node. Here, the link interconnecting the input node and the output node can have a weight value. The weight value can be variable and can be variable by a user or an algorithm to perform the function desired by the neural network. For example, when one or more input nodes are interconnected by respective links to one output node, the output node can determine the value of the output node based on the values input to the input nodes connected to the output node and the weight values set for the respective links corresponding to the input nodes.

[0039] A subset of the nodes constituting the neural network can form a layer. A part of the nodes constituting the neural network can form one layer based on the distance from the first input node. For example, a set of nodes at a distance of n from the first input node can form the n-th layer. The distance from the first input node can be defined by the minimum number of links that must be traversed to reach the corresponding node from the first input node. However, such a definition of a layer is arbitrary for illustrative purposes, and the order of the layers within the neural network can be defined in a manner different from that described above. For example, the layer of nodes can be defined by the distance from the final output node.

[0040] A neural network may include a deep neural network (DNN) with multiple hidden layers in addition to an input layer and an output layer. Using a deep neural network, the latent structures of data can be grasped. Deep neural networks may include convolutional neural networks (CNNs), recurrent neural networks (RNNs), autoencoders, GANs (Generative Adversarial Networks), restricted Boltzmann machines (RBMs), deep belief networks (DBNs), Q networks, U networks, Sham networks, generative adversarial networks (GANs), etc. The description of the deep neural networks mentioned above is only illustrative, and this application is not limited thereto.

[0041] A neural network can be trained in at least one of supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. The training of a neural network can be a process of applying knowledge for the neural network to perform a specific operation to the neural network.

[0042] Hereinafter, the embodiments of this application will be described in detail in sequence.

[0043] FIG. 1 is a flowchart of a defect detection method according to an embodiment of this application.

[0044] In step S110, a target image can be acquired. Here, the target image may be a surface image of an object for which a defect is to be detected.

[0045] For example, the target image may relate to the surface of a cold-rolled steel sheet. In the case of a cold-rolled steel sheet, a pickling process, which is a surface treatment process for removing fine metal contaminants by immersing the metal in a strong acid solution or cleaning the surface, is applied. As a result, in the case of some alloys and high-carbon steels, the problem of hydrogen embrittlement may occur, and hydrogen generated from the acid may react with the surface to cause defects in the metal. Also, the pickling process is the first process to proceed among the manufacturing processes of cold-rolled steel sheets, and since it can comprehensively detect defects generated during the coil transportation process or the production process, the types of defects discovered in the pickling process are more diverse than the types of defects discovered in other processes. The size of the defects can vary by up to 5,000 times, and since the size of the coil used in the cold-rolling process is always constant, the absolute size of the generated defects is very important in defect classification.

[0046] According to an embodiment, the target image can be received from an external database server or acquired by photographing from a photographing device (e.g., a surface defect detector (SSD), etc.) connected to the defect detection device via wired and wireless communication. However, it is not limited thereto.

[0047] In step S120, at least one defect region can be detected from the target image. Here, the defect region may mean a region including a defect formed on the surface and / or a region expected to include a defect formed on the surface. According to an embodiment, the defect region may include only the region corresponding to the defect or may further include a region adjacent to the defect. For example, one defect may belong to a plurality of defect regions.

[0048] In an embodiment, step S120 may include: generating boundary line value information from a target image; generating boundary line direction information from the target image; selecting boundary line value information greater than or equal to a predetermined threshold based on the boundary line value information and the boundary line direction information; and detecting a defective region based on the selected boundary line value information and the corresponding boundary line direction information.

[0049] In an embodiment, step S120 may be performed by inputting a target image into a network function. Here, the network function may be trained to output a defective region from the target image.

[0050] In an embodiment, the defective region may have a rectangular shape, and at least one defect may be located or expected to be located inside the boundary of the rectangular shape. However, it is not limited thereto.

[0051] In step S130, a first defect score may be calculated from the target image to generate a first defect matrix. Specifically, the first defect matrix may calculate a first defect score based on the number of defective regions detected in step S120, and generate the first defect score as a component of the matrix. In particular, the component of the first defect matrix may be determined by the number of defective regions detected from the target image at the position corresponding to the component.

[0052] For example, if a defect exists at a predetermined pixel position in the target image and a plurality of defective regions are detected for the defect, a first defect score is calculated based on the number of the plurality of defective regions, and the first defect score is assigned to the corresponding pixel position and the position in the corresponding matrix.

[0053] In an embodiment, the first defect score can be calculated by summing up the number of defective regions. However, it is not limited thereto, and the first defect score can be calculated in various ways, such as based on the square of the number of defective regions, deviation, etc.

[0054] In an embodiment, the first defect matrix may have a size corresponding to the target image. Here, the first defect matrix having a size corresponding to the target image may mean that the number of rows and columns of the first defect matrix corresponds to the number of horizontal pixels and the number of vertical pixels of the target image, and the first defect matrix sufficiently includes the information of the target image. In particular, corresponding to each other in terms of the number may mean that the numbers are the same as each other, or N times or 1 / N times (where N is a positive integer).

[0055] For example, the number of rows and columns in the first defect matrix may be the same as the number of vertical pixels and the number of horizontal pixels of the target image. For example, when the target image is downsampled and / or upsampled, the first defect matrix may have a size corresponding to the downsampled and / or upsampled target image. However, it is not limited thereto.

[0056] In step S140, a second defect matrix may be generated. The second defect matrix may be obtained by applying the standard deviation for each local region of the target image to the first defect matrix. When the defect is relatively large, the accuracy of defect detection can be improved by considering up to the standard deviation in terms of the increasing influence of the defect on the standard deviation.

[0057] Specifically, step S140 may include a step of calculating the standard deviation for each local region from the target image to generate a matrix of the standard deviation, and a step of combining the first defect matrix and the matrix of the standard deviation.

[0058] Here, the standard deviation matrix can be generated with the calculated standard deviations of the images within the deviation window as components while moving a deviation window having a predetermined size over the target image. At this time, the size of the deviation window may correspond to the reference window. However, it is not limited thereto, and deviation windows of various sizes can be applied. Similarly, the moving interval (Stride) of the deviation window may mean M times or 1 / M times the size of the reference window (where M is a positive integer). However, it is not limited to this.

[0059] Also, the standard deviation matrix may be the same size as the first defect matrix. When the size of the standard deviation matrix is smaller than that of the first defect matrix, predetermined components can be added to the standard deviation matrix (i.e., increasing rows and / or columns) to make it the same size as the first defect matrix. In this case, the values of the components may be the same as or similar to the values of the adjacent Component However, it is not limited to this.

[0060] In an embodiment, the summation of the first defect matrix and the standard deviation matrix may be a weighted sum. Specifically, after multiplying the first defect matrix by a first weighting value and multiplying the standard deviation matrix by a second weighting value, they can be summed with each other to generate a second defect matrix. By adjusting the first weighting value and the second weighting value, the number of defect regions and the proportion that the standard deviation contributes to defect detection can be adjusted.

[0061] A third defect matrix can be generated in step S150. Specifically, the third defect matrix can be generated by removing the components corresponding to the non-defect regions in the second defect matrix.

[0062] In an embodiment, step S150 may include a step of generating a non-defect matrix representing at least one of the background and padding in the target image, and a step of performing a logical product operation on the non-defect matrix and the second defect matrix.

[0063] Here, the non-defective matrix may have a size corresponding to the target image and may have the same size as the second defective matrix. Also, the background is the area excluding the defects in the target image and may include, for example, a surface image where no defects are formed. Padding may mean a margin area added outside the target image for adjusting the size of the target image.

[0064] For example, the components corresponding to the background and padding in the non-defective matrix may have the value of " 0 ", and the other components may be set to the value of "1". Thus, when performing a logical product operation on the non-defective matrix and the second defective matrix, the components at the positions corresponding to the background and padding in the third defective matrix become zero, and the other components may be the same as those of the second defective matrix.

[0065] In the above example, the components of the non-defective matrix are set to "1" or "0", but it is not limited thereto, and various numerical values that can distinguish the components corresponding to the background and padding from the other components can be applied.

[0066] In an embodiment, a predetermined filter can be applied to the non-defective matrix to remove noise therein. For example, a Gaussian Filter can be applied to the non-defective matrix whose components are set to "1" or "0" to remove noise. Thereafter, all components other than "1" can be set to "0". However, it is not limited thereto.

[0067] In step S160, a second defect score can be calculated. Specifically, the second defect score can be calculated by summing the first defect scores within the range corresponding to the reference window in the third defective matrix.

[0068] Here, the reference window is a virtual area having a predetermined size, and the size of the reference window may correspond to the size of the defect image generated in step S170. For example, the reference window may sometimes be represented by a matrix where all components are "1". However, it is not limited thereto.

[0069] Specifically, step S160 can be performed by moving a reference window with respect to the third defect matrix and summing up the first defect scores of the third defect matrix corresponding to the reference window. In step S160, a plurality of second defect scores can be calculated, and each second defect score is assigned a position of the reference window for the third defect matrix.

[0070] For example, while moving the reference window with respect to the third defect matrix, after performing a logical AND operation on the corresponding components of the reference window matrix and the third defect matrix, the results of the operations can be summed up to calculate the second defect score.

[0071] In step S170, a defect image can be generated. The defect image can mean an image selected as effectively representing the defect characteristics in the target image.

[0072] Specifically, in step S170, based on the second defect score, an image corresponding to the reference window can be extracted from the target image to generate a defect image. For example, step S170 can be performed by extracting an image corresponding to the position of the reference window with the largest second defect score from the target image. That is, the second defect score with the highest score is selected, and a defect image can be generated in the target image based on the position of the reference window assigned to the corresponding second defect score.

[0073] In an embodiment, method 100 may further include inputting a defective image as input data into a network function to generate surface defect information. Here, the network function may be learned to output surface defect information using an image sized corresponding to a reference window as input data. Further, the surface defect information may include at least one of presence / absence information of a defect and classification information of the defect. The classification information may include, for example, Carbon, Oil Mark, Roll Mark, Rust, Scab, Scratch, Slip Mark, etc., but is not limited thereto.

[0074] In an embodiment, S 120 ~S170 steps may be performed when at least one of the horizontal size and the vertical size of the target image is larger than the reference window. That is, when the size of the target image is at least partially larger than the reference window, S120~S170 steps may be performed to extract only the portion that best represents the defect characteristics from the target image and generate a defective image.

[0075] In an embodiment, method 100 may further include adding padding around the target image so that the target image corresponds to the reference window when at least one of the horizontal size and the vertical size of the target image is smaller than the reference window. The step of adding padding may be performed by adding an image corresponding to white, black, or the background color around the target image. At this time, the background color may be the color of the background which is the surface image where no defect is formed in the target image or the color of the background added in the image processing process involved in the generation process of the target image, but is not limited thereto.

[0076] In an embodiment, when the first size is smaller than the reference window among the horizontal size and the vertical size of the target image, and the other second size is larger than the reference window, both removal and padding addition can be performed on at least a part of the target image.

[0077] Specifically, in order to remove at least a part of the target image, steps S120 to S170 can be performed. Further, a step of adding padding to the target image can be performed. In this case, the addition can be performed first and then the removal can be performed, but it is not limited thereto. The removal can be performed first, Addition and then the addition can be performed, or the addition and the removal can be performed simultaneously. Therefore, the target image referred to in S120 to S170 can mean a target image without padding added or a target image with padding added.

[0078] In an embodiment, at least one step in method 100 may not be performed. Specifically, at least one of step S140 and step S150 may not be performed.

[0079] For example, steps S140 and S150 may not be performed. In this case, step S160 will be performed after step S130. In step S160, instead of the third defect matrix, the first defect score can be summed within the range corresponding to the reference window in the first defect matrix to calculate the second defect score.

[0080] Also, for example, step S140 may not be performed. Therefore, step S150 can be performed after step S130. In step S150, instead of the second defect matrix, components corresponding to the non-defective region in the first defect matrix can be removed to generate the third defect matrix.

[0081] Also, for example, the S150 step may not be performed. Therefore, the S160 step is performed after the S140 step. In the S160 step, instead of the third defect matrix, the first defect scores can be summed within the range corresponding to the reference window in the second defect matrix to calculate the second defect score.

[0082] The method 100 in FIG. 1 is exemplary, and various configurations can be applied according to the embodiments of the present application.

[0083] FIG. 2 is a block diagram of a defect detection device according to an embodiment of the present application.

[0084] The defect detection device 200 can perform the method 100 in FIG. 1 and the like, but is not limited thereto.

[0085] Referring to FIG. 2, it may include a communication unit 210, an input unit 220, a memory 230, and a processor 240.

[0086] The communication unit 210 can receive data from the outside. The communication unit 210 may include a wired / wireless communication unit. When the communication unit 210 includes a wired communication unit, the communication unit 210 may include one or more components that enable communication through a local area network (LAN), a wide area network (WAN), a value-added network (VAN), a mobile radio communication network, a satellite communication network, and combinations thereof. Also, when the communication unit 210 includes a wireless communication unit, the communication unit 210 can wirelessly transmit and receive data or signals using cellular communication, wireless LAN (e.g., Wi-Fi), etc. In an embodiment, the communication unit 210 can transmit and receive data or signals to / from an external device or an external server under the control of the processor 240.

[0087] The input unit 220 can receive various user commands by an external operation. For this purpose, the input unit 220 can include or be connected to one or more input devices. For example, the input unit 220 can be connected to interfaces for various inputs such as a keypad, a mouse, etc. to receive user commands. For this purpose, the input unit 220 can include not only a USB port but also interfaces such as Thunderbolt. Also, the input unit 220 can include various input devices such as a touch screen, buttons, or be combined with these to receive external user commands.

[0088] The memory 230 can store programs and / or program instructions for the operation of the processor 240, and can store input / output data temporarily or permanently. The memory 230 can include at least one type of storage medium among Flash Memory type, Hard Disk type, Multimedia Card Micro type, card type memory (e.g., SD or XD memory, etc.), RAM, SRAM, ROM, EEPROM, PROM, magnetic memory, magnetic disk, optical disk.

[0089] Also, the memory 230 can store various network functions and algorithms, and can store various data, programs (multiple instructions), applications, software, instructions, codes, etc. for driving and controlling the device 200.

[0090] The processor 240 can control the overall operation of the device 200. The processor 240 can execute one or more programs or software stored in the memory 230. The processor 240 can mean a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), or a dedicated processor 240 on which the method according to the embodiments of the present application is executed.

[0091] In an embodiment, the processor 240 acquires a target image, detects at least one defective region from the target image, generates a first defect matrix having a size corresponding to the target image and having, as an element, a first defect score calculated based on the number of defective regions, sums the first defect scores within a range corresponding to a predetermined reference window in the first defect matrix to calculate a plurality of second defect scores, and may extract, based on the second defect scores, an image corresponding to the reference window from the target image to generate a defective image.

[0092] In an embodiment, the processor 240 may input a defective image as input data to a network function to generate surface defect information. At this time, the network function may be learned to output surface defect information with an image having a size corresponding to the reference window as input data.

[0093] In an embodiment, the component of the first defect matrix of the processor 240 may be determined by the number of defective regions detected from the target image at the position corresponding to the component.

[0094] In an embodiment, the processor 240 can calculate a second defect score, which is the sum of the first defect scores of the first defect matrix corresponding to the reference window, while moving a predetermined reference window with respect to the first defect matrix. The processor 240 may extract an image corresponding to the position of the reference window having the largest second defect score from the target image to generate a defective image.

[0095] In an embodiment, the processor 240 may apply the standard deviation for each local region of the target image to the first defect matrix to generate a second defect matrix. The processor 240 can replace the first defect matrix and calculate a second defect score for the second defect matrix.

[0096] In an embodiment, the processor 240 may calculate the standard deviation for each local area from the target image to generate a matrix of standard deviations, and generate a second defect matrix by calculating the weighted sum of the first defect matrix and the matrix of standard deviations.

[0097] In an embodiment, the processor 240 may remove the components corresponding to the non-defective areas in the second defect matrix to generate a third defect matrix. The processor 240 can calculate the second defect score for the third defect matrix by replacing the second defect matrix.

[0098] In an embodiment, the processor 240 may generate a non-defect matrix having a size corresponding to the target image and representing at least one of the background and padding in the target image, and generate a third defect matrix by performing a logical product operation on the non-defect matrix and the first defect matrix.

[0099] In an embodiment, the processor 240 may generate boundary line value information for the target image, generate boundary line direction information for the target image, select boundary line value information greater than or equal to a predetermined threshold based on the boundary line value information and the boundary line direction information, and detect a defect area based on the selected boundary line value information and the corresponding boundary line direction information.

[0100] In an embodiment, the processor 240 may detect a defect area or generate a defect image when at least one of the horizontal size and the vertical size of the target image is larger than the reference window.

[0101] In an embodiment, when at least one of the horizontal size and the vertical size of the target image is smaller than the reference window, the processor 240 can add padding around the target image so that the target image corresponds to the reference window.

[0102] The apparatus shown in FIG. 2 is exemplary, and various configurations can be applied according to the embodiments of the present application.

[0103] FIG. 3 is an exemplary diagram of a target image with a defect formed.

[0104] The defects shown in FIG. 3 occur on the surface of the cold-rolled steel sheet, and may mean Carbon, Oil Mark, Roll Mark, Rust, Scab, Scratch, and Slip Mark in the order of (a) to (g).

[0105] FIG. 3 is exemplary, and various configurations according to the embodiments of the present application can be applied.

[0106] FIGS. 4 and 5 are exemplary diagrams of image preprocessing for detecting defects according to the embodiments of the present application.

[0107] Referring to FIG. 4, when both the horizontal size and the vertical size of the target image are smaller than the reference window, padding can be added around the target image to generate a defect image.

[0108] Referring to FIG. 5, a case is shown where the horizontal size of the target image is smaller than the reference window and the vertical size of the target image is larger than the reference window. In such a case, padding can be added to the target image in the horizontal direction that is smaller than the reference window, and in the vertical direction that is larger than the reference window, an area that best represents the defect characteristics can be extracted to generate a defect image.

[0109] FIGS. 4 and 5 are exemplary, and various configurations can be applied according to the embodiments of the present application.

[0110] FIG. 6 is a diagram for explaining the defect detection process according to the embodiments of the present application.

[0111] As shown in the figure, when a target image with a defect formed on its surface is acquired, boundary line detection or the like can be performed on this, and at least one defect region can be detected. A first defect score can be calculated based on the defect region to generate a first defect matrix.

[0112] Further, a standard deviation may be calculated for each local region of the target image to generate a deviation matrix, and the deviation matrix and the first defect matrix may be weighted and multiplied to generate a second defect matrix.

[0113] Although not shown, a non-defective region (background and margin) may be removed from the second defect matrix to generate a third defect matrix. In the third defect matrix, a second defect score may be calculated while moving a reference window, and a defective image may be generated from the reference window at the position having the largest second defect score based on this.

[0114] FIG. 6 is exemplary, and various configurations according to the embodiments of the present application can be applied.

[0115] The method according to the embodiments of the present application can be implemented in the form of program instructions to be performed by various computer means and can be recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be those specially designed and configured for the present application or those known and usable by those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, magnetic tapes, optical media such as CD-ROMs, DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program instructions such as ROMs, RAMs, flash memories. Examples of program instructions include not only machine language codes created by compilers but also high-level language codes that can be executed by a computer using an interpreter or the like.

[0116] Also, the method according to the disclosed embodiments can be provided included in a computer program product. The computer program product can be traded as a commodity between a seller and a purchaser.

[0117] The computer program product can include an S / W program and a computer-readable storage medium storing the S / W program. For example, the computer program product can include a commodity in the form of an S / W program (e.g., a downloadable app) that is electronically distributed through an electronic device manufacturer or an electronic market (e.g., the Google Play Store, the App Store). For electronic distribution, at least a part of the S / W program can be stored in a storage medium or can be temporarily generated. In this case, the storage medium can be the storage medium of the manufacturer's server, the electronic market's server, or a relay server that temporarily stores the SW program.

[0118] The computer program product can include the storage medium of the server or the storage medium of the client device in a system composed of a server and a client device. Also, if there is a third device (e.g., a smartphone) communicatively connected to the server or the client device, the computer program product can include the storage medium of the third device. Also, the computer program product can include the S / W program itself transmitted from the server to the client device or the third device, or transmitted from the third device to the client device.

[0119] In this case, one of the server, the client device, and the third device can execute the computer program product and perform the method according to the disclosed embodiments. Also, two or more of the server, the client device, and the third device can execute the computer program product and perform the method according to the disclosed embodiments in a distributed manner.

[0120] For example, a server (such as a cloud server or an artificial intelligence server, etc.) can execute a computer program product stored in the server, and control a client device communicatively connected to the server to perform the method according to the disclosed embodiment.

[0121] As described above in detail for the embodiments, the scope of the rights of this application is not limited thereto, and various variations and improvements by those skilled in the art using the basic concept of this application defined in the following claims also belong to the scope of the rights of this application.

Claims

1. A method for detecting surface defects, comprising: a step of acquiring a target image; a step of detecting at least one defect region from the target image; a step of generating a first defect matrix having a size corresponding to the target image and having a first defect score calculated based on the number of defect regions as an element; a step of calculating a plurality of second defect scores by summing the first defect scores within a range corresponding to a predetermined reference window in the first defect matrix; a step of generating a defect image by extracting an image corresponding to the reference window from the target image based on the second defect score. The method is characterized by the above.

2. The method further includes a step of inputting the defect image as input data into a network function to generate surface defect information, and the network function is learned to output the surface defect information using an image having a size corresponding to the reference window as input data. The method according to claim 1.

3. The component of the first defect matrix is determined by the number of defect regions detected from the target image at the position corresponding to the component. The method according to claim 1.

4. The step of calculating the second defect score is performed by calculating, while moving a predetermined reference window with respect to the first defect matrix, a second defect score that is the sum of the first defect scores of the first defect matrix corresponding to the reference window. The step of generating the defect image is performed by extracting an image corresponding to the position of the reference window having the largest second defect score from the target image. The method according to claim 1.

5. The method further includes a step of applying a standard deviation for each local region of the target image to the first defect matrix to generate a second defect matrix. The step of calculating the second defect score is performed for the second defect matrix by replacing the first defect matrix. The method according to claim 1.

6. The step of generating the second defect matrix includes a step of calculating a standard deviation for each local region from the target image to generate a standard deviation matrix, and a step of performing a weighted sum of the first defect matrix and the standard deviation matrix. The method according to claim 5.

7. The method further includes a step of removing a configuration corresponding to a non-defect region in the second defect matrix to generate a third defect matrix. The step of calculating the second defect score is performed on the third defect matrix by replacing the second defect matrix. The method according to claim 5. **Claim 8** The step of generating the third defect matrix includes generating a non-defect matrix having a size corresponding to the target image and representing at least one of the background and padding in the target image, and performing a logical product operation on the non-defect matrix and the first defect matrix. The method according to claim 7. **Claim 9** The step of detecting the defect region includes generating boundary line value information in the target image, generating boundary line direction information in the target image, selecting boundary line value information greater than or equal to a predetermined threshold based on the boundary line value information and the boundary line direction information, and detecting a defect region based on the selected boundary line value information and the corresponding boundary line direction information. The method according to claim 5. **Claim 10** The steps of detecting the defect region to generating the defect image are performed when at least one of the horizontal size and the vertical size of the target image is larger than the reference window. The method according to claim 1. **Claim 11** When at least one of the horizontal size and the vertical size of the target image is smaller than the reference window, the method further includes adding padding around the target image so that the target image corresponds to the reference window. The method according to claim 1. **Claim 12** Stored in a recording medium for executing the method according to any one of claims 1 to 11 A computer program characterized by the above. **Claim 13** An apparatus for detecting surface defects, A memory storing a program for detecting surface defects, A processor that acquires a target image, detects at least one defect region from the target image, generates a first defect matrix having a size corresponding to the target image and having a first defect score calculated based on the number of defect regions as a component, sums the first defect scores within a range corresponding to a predetermined reference window in the first defect matrix to calculate a plurality of second defect scores, and extracts an image corresponding to the reference window from the target image based on the second defect scores to generate a defect image. An apparatus characterized by the above. **Claim 14** The processor applies the standard deviation for each local region of the target image to the first defect matrix to generate a second defect matrix, replaces the first defect matrix, and calculates a second defect score for the second defect matrix. The apparatus according to claim 13.

15. The processor removes the configuration corresponding to the non-defect region from the second defect matrix to generate a third defect matrix. Replace the second defect matrix and calculate a second defect score for the third defect matrix. The apparatus according to claim 14.

Images