Surface defect detection method and apparatus

Abstract

To provide a method and apparatus for detecting a surface defect, and a program configured to classify defects of target images having various sizes and aspect ratios.SOLUTION: A method includes the steps of: acquiring a target image; detecting at least one defect area from the target image; generating a first defect matrix having a size corresponding to the target image and an element that is a first defect score calculated based on the number of defect areas; calculating a plurality of second defect scores by summing first defect scores in a range corresponding to a predetermined reference window in the first defect matrix; and extracting an image corresponding to the reference window from the target image based on the second defect score and generating a defect image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present application relates to a surface defect detection method and apparatus. [Background technology]

[0002] In recent years, advances in artificial intelligence have led to an increase in intelligent factories that incorporate artificial intelligence into their manufacturing processes. One such example is image classification using deep learning, which is used to classify defects that occur during the manufacturing process, replacing the traditional manual defect classification work and contributing to improved process efficiency and quality.

[0003] On the other hand, cold-rolled steel sheets can develop various types of defects through the rolling and rolling processes. As the production process changes depending on the application and quality, the distribution of defect characteristics becomes more diverse. As certain defects can cause fatal defects during product manufacturing, cold-rolled steel sheet surface defect detectors ( SDD ) have been introduced to detect defects, and recently, research has been conducted on methods to detect defects using surface defect detectors and classify them using deep learning-based models.

[0004] However, since defect images detected by a surface defect detector have various sizes and ratios, they need to be adjusted to a standardized model. Generally, to process data into an image of a standardized size, three preprocessing methods are used: a crop preprocessing method that cuts out a part of the image, a resize preprocessing method that adjusts the image size to a standardized model, and a padding preprocessing method that sets a padding value in the image frame.

[0005] These common preprocessing methods can result in the loss of image characteristics, which reduces the reliability of deep learning-based classification models. In particular, defect images generated through different production processes for different applications and required quality show a more diverse distribution of defect characteristics, such as defect morphology and type, so there is a need to develop technology to improve defect classification performance for such defect images. Summary of the Invention [Problem to be solved by the invention]

[0006] The present application aims to provide a method and apparatus for detecting surface defects. [Means for solving the problem]

[0007] According to an embodiment of the present application, there is provided a surface defect detection method, which may include: acquiring a target image, detecting at least one defect area from the target image, generating a first defect matrix having a size corresponding to the target image and having first defect scores calculated based on the number of the defect areas as elements, calculating a plurality of second defect scores by adding up first defect scores within a range corresponding to a predetermined reference window in the first defect matrix, and generating a defect image by extracting an image corresponding to the reference window from the target image based on the second defect scores.

[0008] The method may further include a step of inputting the defect image as input data into a network function to generate surface defect information, and the network function may be trained to output the surface defect information using an image of a size corresponding to the reference window as input data.

[0009] Furthermore, the elements of the first defect matrix may be determined by the number of defect areas detected from the target image at positions corresponding to the elements.

[0010] Furthermore, the step of calculating the second defect score may be performed by calculating a second defect score, which is the sum of the first defect scores of the first defect matrix corresponding to a predetermined reference window while moving the predetermined reference window relative to the first defect matrix, and the step of generating the defect image may be performed by extracting an image corresponding to the position of the reference window where the second defect score is the largest from the target image.

[0011] The method may further include applying a standard deviation for each local region of the target image to the first defect matrix to generate a second defect matrix, and the step of calculating the second defect score may be performed on the second defect matrix by replacing the first defect matrix.

[0012] In addition, the step of generating the second defect matrix may include the steps of: calculating standard deviations for each local region from the target image to generate a matrix of standard deviations; and weighting the first defect matrix and the matrix of standard deviations.

[0013] The method may further include a step of generating a third defect matrix by removing components corresponding to non-defective areas in the second defect matrix, and the step of calculating the second defect score may be performed on the third defect matrix by replacing the second defect matrix.

[0014] Furthermore, the step of generating the third defect matrix may include the steps of generating a non-defect matrix having a size corresponding to the target image and representing at least one of a background and padding in the target image, and performing a logical AND operation on the non-defect matrix and the first defect matrix.

[0015] In addition, the step of detecting the defective area may include the steps of generating boundary line value information in the target image, generating boundary line direction information in the target image, selecting boundary line value information that is equal to or greater than a predetermined threshold based on the boundary line value information and the boundary line direction information, and detecting the defective area based on the selected boundary line value information and the corresponding boundary line direction information.

[0016] The steps of detecting the defect area and generating the defect image may be performed when at least one of the horizontal size and the vertical size of the target image is larger than the reference window.

[0017] The method may further include adding padding around the target image so that the target image corresponds to the reference window if at least one of the horizontal and vertical sizes of the target image is smaller than the reference window.

[0018] According to an embodiment of the present application, a computer program is provided, which can be stored in a recording medium for executing the surface defect detection method according to the embodiment of the present application.

[0019] According to an embodiment of the present application, there is provided an apparatus for detecting surface defects, which may include: a memory storing a program for detecting surface defects; a processor for acquiring a target image, detecting at least one defect area from the target image, generating a first defect matrix having a size corresponding to the target image and including first defect scores calculated based on the number of the defect areas as elements, calculating a plurality of second defect scores by adding up the first defect scores within a range corresponding to a predetermined reference window in the first defect matrix, and generating a defect image by extracting an image corresponding to the reference window from the target image based on the second defect scores.

[0020] The processor may also apply the standard deviation for each local region of the target image to the first defect matrix to generate a second defect matrix, and replace the first defect matrix to calculate a second defect score for the second defect matrix.

[0021] The processor may also remove elements of the second defect matrix corresponding to non-defective areas to generate a third defect matrix, and replace the second defect matrix to calculate a second defect score for the third defect matrix. [Effects of the Invention]

[0022] Embodiments of the present application enable defect classification for target images of various sizes and ratios.

[0023] In addition, it is possible to improve classification accuracy by minimizing characteristic loss in various types of surface defect images.

[0024] Also, by not resizing the target image, it is possible to prevent the characteristic information from being enlarged or reduced.

[0025] The effects obtained from the embodiments of the application are not limited to the effects described above, and other effects not mentioned will be clearly understood by those skilled in the art to which this application pertains from the following description. In order to more fully understand the drawings referred to in this application, a brief description of each drawing is provided. [Brief explanation of the drawings]

[0026] [Figure 1] 1 is a flowchart of a defect detection method according to an embodiment of the present application. [Figure 2] FIG. 1 is a block diagram of a defect detection apparatus according to an embodiment of the present application. [Figure 3] FIG. 10 is an exemplary view of a target image in which a defect is formed; [Figure 4] FIG. 1 is an exemplary diagram of image pre-processing for defect detection according to an embodiment of the present application. [Figure 5]FIG. 1 is an exemplary diagram of image pre-processing for defect detection according to an embodiment of the present application. [Figure 6] 1A to 1C are diagrams for explaining a defect detection process according to an embodiment of the present application. DETAILED DESCRIPTION OF THE INVENTION

[0027] Since the technical idea of ​​the present application can be variously modified and can have various embodiments, specific embodiments are illustrated in the drawings and will be described in detail, but this is not intended to limit the technical idea of ​​the present application to the specific embodiments, and all modifications, equivalents, and alternatives included within the scope of the technical idea of ​​the present application are included.

[0028] In explaining the technical idea of ​​the present application, if it is determined that a detailed description of related publicly known art may unnecessarily obscure the gist of the present application, the detailed description will be omitted.

[0029] The terms used in this specification are used to describe the embodiments and are not intended to limit and / or restrict the present application. A singular expression includes a plural expression unless the context clearly indicates otherwise. Furthermore, numbers used in this specification (e.g., first, second, etc.) are merely identifiers for distinguishing one component from another.

[0030] In this specification, when a part is said to be connected to another part, this includes not only a direct connection but also an indirect connection via another structure in between. Furthermore, when a part is said to include a certain component, this does not exclude other components, and means that it may further include other components, unless otherwise specified.

[0031] Additionally, the term "or" in this application is intended to mean an inclusive "or" rather than an exclusive "or." That is, unless otherwise specified or clear from the context, it is intended to mean any of the natural inclusive permutations of "X uses A or B." That is, when X uses A, when X uses B, or when X uses both A and B, it is also applicable to any of the above cases of "X uses A or B." Additionally, as used herein, the term "and / or" refers to and includes all possible combinations of one or more of the associated listed configurations.

[0032] Furthermore, the terms "unit", "machine", "character", "module", etc. used in this application refer to a unit that processes at least one function or operation, which may be embodied in hardware and software such as a processor, microprocessor, microcontroller, CPU (Central Processing Unit), GPU (Graphics Processing Unit), APU (Accelerate Processor Unit), DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array), or a combination of hardware and software.

[0033] It is intended to clarify that the classification of components in this application is merely a classification based on the main function of each component. In other words, two or more components described below may be combined into one component, or one component may be divided into two or more components based on more specific functions. It goes without saying that each component described below may perform some or all of the functions of other components in addition to its own main function, and that some of the main functions of each component may be exclusively performed by other components.

[0034] The methods according to the embodiments of the present application may be performed on a personal computer, workstation, server computer device, etc. equipped with computing power, or on a separate device for this purpose.

[0035] The method may be performed by one or more computing devices. For example, at least one step of the method according to the embodiment of the present application may be performed by a client device, and other steps may be performed by a server device. In this case, the client device and the server device may be connected to a network to transmit and receive computation results. Alternatively, the method may be performed by a distributed computing technique.

[0036] In this specification, the term "network function" can be used interchangeably with the term "computational model" or "neural network." A neural network may be composed of a collection of interconnected computational units, generally called nodes. These nodes may be called neurons. A neural network is composed of at least one or more nodes. The nodes (or neurons) that make up a neural network may be interconnected by one or more links.

[0037] Within a neural network, one or more nodes connected via links may form a relative relationship of input node and output node. The concepts of input node and output node are relative, and any node that has an output node relationship with one node may also have an input node relationship with another node, and vice versa. As mentioned above, the input node to output node relationship may be generated around links. One or more output nodes may be connected to one input node via links, and vice versa.

[0038] In a relationship between an input node and an output node connected via a link, the value of the data of the output node can be determined based on the data input to the input node. Here, the link interconnecting the input node and the output node can have a weight. The weight can be variable and can be changed by a user or an algorithm to perform a desired function of the neural network. For example, when one or more input nodes are interconnected to one output node by respective links, the output node can determine the value of the output node based on the value input to the input node connected to the output node and the weight set for the link corresponding to each input node.

[0039] A subset of nodes constituting a neural network may constitute a layer. Some of the nodes constituting a neural network may constitute a layer based on their distance from the first input node. For example, a set of nodes that are n distances from the first input node may constitute n layers. The distance from the first input node may be defined by the minimum number of links that must be traversed to reach the node from the first input node. However, this definition of a layer is arbitrary for illustrative purposes, and the order of layers in a neural network may be defined in a manner different from that described above. For example, a layer of nodes may be defined by its distance from the final output node.

[0040] The neural network may include a deep neural network (DNN) that includes multiple hidden layers in addition to an input layer and an output layer. The deep neural network can be used to understand the latent structures of data. The deep neural network may include a convolutional neural network (CNN), a recurrent neural network (RNN), an autoencoder, a generative adversarial network (GAN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a Q-network, a U-network, a Siamese network, a generative adversarial network (GAN), and the like. The above description of the deep neural network is merely illustrative, and the present application is not limited thereto.

[0041] Neural networks can be trained using at least one of the following methods: supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. Training a neural network can be a process of applying knowledge to the neural network to enable the neural network to perform a specific operation.

[0042] Hereinafter, the embodiments of the present application will be described in detail one by one.

[0043] FIG. 1 is a flowchart of a defect detection method according to an embodiment of the present application.

[0044] In step S110, a target image may be acquired, where the target image may be a surface image of an object in which defects are to be detected.

[0045] For example, the target image may be the surface of a cold-rolled steel sheet. Cold-rolled steel sheets undergo a pickling process, a surface treatment in which the metal is immersed in a strong acid solution to remove fine metal contaminants or clean the surface. This can lead to hydrogen embrittlement issues in some alloys and high-carbon steels, and hydrogen generated from the acid can react with the surface and cause defects in the metal. Furthermore, the pickling process is the first step in the manufacturing process of cold-rolled steel sheets. Because it can detect defects that occur during the coil transportation or production process, the types of defects found during the pickling process are more diverse than those found in other processes. The size of defects can vary by up to 5,000 times. Because the size of the coils used in the cold-rolling process is always constant, the absolute size of the defects that occur is crucial for defect classification.

[0046] According to an embodiment, the target image may be received from an external database server or may be acquired by photography from a photography device (e.g., a surface defect detector (SSD)) connected to the defect detection device via wired and wireless communication, but is not limited thereto.

[0047] In step S120, at least one defect area may be detected from the target image. Here, the defect area may refer to an area including a defect formed on the surface and / or an area expected to include a defect formed on the surface. According to an embodiment, the defect area may include only an area corresponding to the defect, or may further include an area adjacent to the defect. For example, one defect may belong to multiple defect areas.

[0048] In an embodiment, step S120 may include the steps of generating boundary line value information in the target image, generating boundary line direction information in the target image, selecting boundary line value information that is greater than or equal to a predetermined threshold based on the boundary line value information and the boundary line direction information, and detecting defective areas based on the selected boundary line value information and corresponding boundary line direction information.

[0049] In an embodiment, step S120 may be performed by inputting the target image to a network function, where the network function may be trained to output defect regions from the target image.

[0050] In embodiments, the defect region may have a rectangular shape, with at least one defect located or expected to be located within the boundaries of the rectangular shape, but is not limited to such.

[0051] In step S130, a first defect matrix may be generated by calculating first defect scores from the target image. Specifically, the first defect matrix may be generated by calculating the first defect scores based on the number of defect areas detected in step S120 and using the first defect scores as matrix elements. In particular, the elements of the first defect matrix may be determined by the number of defect areas detected in the target image at positions corresponding to the elements.

[0052] For example, if a defect exists at a specific pixel location in a target image and multiple defect areas are detected for the defect, a first defect score is calculated based on the number of multiple defect areas, and the first defect score is assigned to the pixel location and its corresponding location in the matrix.

[0053] In an embodiment, the first defect score may be calculated by adding up the number of defect areas, but is not limited thereto, and the first defect score may be calculated in various ways, such as based on the square of the number of defect areas, deviation, etc.

[0054] In an embodiment, the first defect matrix may have a size corresponding to the target image. Here, the first defect matrix having a size corresponding to the target image may mean that the number of rows and columns of the first defect matrix correspond to the number of horizontal pixels and the number of vertical pixels of the target image, and the first defect matrix fully contains information of the target image. In particular, corresponding to each other in number may mean that the numbers are the same, N times, or 1 / N times each other (where N is a positive integer).

[0055] For example, the number of rows and columns in the first defect matrix may be the same as the number of vertical pixels and the number of horizontal pixels in the target image. For example, if the target image is downsampled and / or upsampled, the first defect matrix may have a size corresponding to the downsampled and / or upsampled target image. However, the present invention is not limited to this.

[0056] In step S140, a second defect matrix may be generated. The second defect matrix may be obtained by applying the standard deviation for each local region of the target image to the first defect matrix. Since the influence of a relatively large defect on the standard deviation increases when the defect is relatively large, the accuracy of defect detection can be improved by taking the standard deviation into consideration.

[0057] Specifically, step S140 may include calculating standard deviations for each local region from the target image to generate a standard deviation matrix, and combining the first defect matrix with the standard deviation matrix.

[0058] Here, the standard deviation matrix may be generated by calculating the standard deviation of the image within the deviation window while moving a deviation window having a predetermined size over the target image, and using the calculated standard deviation as its components. In this case, the size of the deviation window may correspond to the reference window. However, this is not limited thereto, and deviation windows of various sizes can be applied. Similarly, the stride of the deviation window may be M times or 1 / M times the size of the reference window (where M is a positive integer), but is not limited thereto.

[0059] Also, the standard deviation matrix may be the same size as the first defect matrix. If the size of the standard deviation matrix is ​​smaller than the first defect matrix, certain elements may be added to the standard deviation matrix (i.e., rows and / or columns may be increased) to make it the same size as the first defect matrix. In this case, the values ​​of the elements may be adjusted to match the adjacent component The value may be the same as or similar to the value of, but is not limited to,

[0060] In an embodiment, the first defect matrix and the standard deviation matrix may be combined using a weighted sum. Specifically, the first defect matrix may be multiplied by a first weight, the standard deviation matrix may be multiplied by a second weight, and then the resulting matrix may be combined to generate a second defect matrix. By adjusting the first and second weights, the weights given to the number of defect areas and the standard deviation in defect detection may be adjusted.

[0061] In step S150, a third defect matrix may be generated by removing elements corresponding to non-defective regions from the second defect matrix.

[0062] In an embodiment, the step S150 may include generating a non-defective matrix representing at least one of the background and padding in the target image, and performing a logical AND operation on the non-defective matrix and a second defective matrix.

[0063] Here, the non-defect matrix may have a size corresponding to the target image and may have the same size as the second defect matrix. The background may be a region of the target image excluding defects, for example, a surface image where no defects are formed. The padding may refer to a margin region added outside the target image to adjust the size of the target image.

[0064] For example, the components corresponding to the background and padding in a non-defective matrix are 0 " value may be set, and the other components may be set to a value of "1." Thus, when performing a logical product operation on the non-defective matrix and the second defective matrix, the components in the positions corresponding to the background and padding in the third defective matrix may be zero, and the other components may be the same as the components of the second defective matrix.

[0065] In the above example, the elements of the non-defective matrix are set to "1" or "0", but this is not limited thereto, and various values ​​that can distinguish the elements corresponding to the background and padding from other elements can be applied.

[0066] In an embodiment, a predetermined filter can be applied to the non-defective matrix to remove noise in the non-defective matrix. For example, a Gaussian filter can be applied to the non-defective matrix, whose components are set to "1" or "0", to remove noise. Then, all components other than "1" can be set to "0". However, this is not limiting.

[0067] In step S160, a second defect score can be calculated by summing up the first defect scores within a range corresponding to the reference window in the third defect matrix.

[0068] Here, the reference window is a virtual region having a predetermined size, and the size of the reference window may correspond to the size of the defect image generated in step S170. For example, the reference window may be expressed as a matrix whose components are all "1", but is not limited thereto.

[0069] Specifically, step S160 may be performed by summing the first defect scores of the third defect matrix corresponding to the reference window while moving the reference window relative to the third defect matrix, and in step S160, a plurality of second defect scores may be calculated, each of which is assigned a position of the reference window relative to the third defect matrix.

[0070] For example, the reference window is moved relative to the third defect matrix, and the corresponding elements of the matrix of the reference window and the third defect matrix are ANDed together, and then the results of the operation are all added together to calculate the second defect score.

[0071] In step S170, a defect image may be generated, which may refer to an image selected to effectively represent the defect characteristics in the target image.

[0072] Specifically, in step S170, a defect image may be generated by extracting an image corresponding to a reference window from the target image based on the second defect score. For example, step S170 may be performed by extracting an image corresponding to the position of the reference window with the largest second defect score from the target image. That is, the second defect score with the highest score may be selected, and a defect image may be generated in the target image based on the position of the reference window assigned to the second defect score.

[0073] In an embodiment, the method 100 may further include generating surface defect information by inputting the defect image as input data to a network function. Here, the network function may be trained to output surface defect information using an image of a size corresponding to a reference window as input data. The surface defect information may also include at least one of information on the presence or absence of a defect and defect classification information. The classification information may include, but is not limited to, carbon, oil mark, roll mark, rust, scab, scratch, slip mark, etc.

[0074] In an embodiment, S 120 Steps S120 to S170 may be performed when at least one of the horizontal size and vertical size of the target image is larger than the reference window. That is, when the size of the target image is at least partially larger than the reference window, steps S120 to S170 may be performed to extract only a portion of the target image that best represents the defect characteristics and generate a defect image.

[0075] In an embodiment, the method 100 may further include, if at least one of the horizontal and vertical sizes of the target image is smaller than the reference window, adding padding around the target image so that the target image corresponds to the reference window. The padding may be performed by adding an image corresponding to white, black, or a background color around the target image. In this case, the background color may be, but is not limited to, a background color that is a surface image without defects in the target image or a background color added during an image processing process involved in generating the target image.

[0076] In an embodiment, if a first size of the horizontal and vertical sizes of the target image is smaller than the reference window and a second size of the target image is larger than the reference window, both removal and addition of padding to at least a portion of the target image may be performed.

[0077] Specifically, steps S120 to S170 may be performed to remove at least a portion of the target image. Furthermore, a step of adding padding to the target image may be performed. In this case, the addition may be performed first, followed by the removal, but this is not limiting. The removal may be performed first, followed by the addition. addition Alternatively, addition and removal may be performed simultaneously. Therefore, the target image referred to in steps S120 to S170 may refer to a target image to which no padding has been added or a target image to which padding has been added.

[0078] In an embodiment, at least one step may not be performed in the method 100. Specifically, at least one of steps S140 and S150 may not be performed.

[0079] For example, steps S140 and S150 may not be performed. In this case, step S160 is performed after step S130. In step S160, the second defect score can be calculated by adding up the first defect scores within the range corresponding to the reference window in the first defect matrix, not in the third defect matrix.

[0080] Also, for example, step S140 may not be performed. Therefore, step S150 may be performed after step S130, and in step S150, components corresponding to non-defective regions in the first defect matrix may be removed to generate a third defect matrix, rather than the second defect matrix.

[0081] Also, for example, step S150 may not be performed. Therefore, step S160 is performed after step S140, and in step S160, the second defect score can be calculated by adding up the first defect scores within the range corresponding to the reference window in the second defect matrix, not the third defect matrix.

[0082] The method 100 of FIG. 1 is exemplary and may be adapted in various configurations according to embodiments of the present application.

[0083] FIG. 2 is a block diagram of a defect detection apparatus according to an embodiment of the present application.

[0084] The defect detection system 200 may perform a method such as, but not limited to, the method 100 of FIG.

[0085] Referring to FIG. 2, the device may include a communication unit 210, an input unit 220, a memory 230, and a processor 240.

[0086] The communication unit 210 may receive data from an external device. The communication unit 210 may include a wired or wireless communication unit. If the communication unit 210 includes a wired communication unit, the communication unit 210 may include one or more components that enable communication through a local area network (LAN), a wide area network (WAN), a value-added network (VAN), a mobile radio communication network, a satellite communication network, or a combination thereof. If the communication unit 210 includes a wireless communication unit, the communication unit 210 may transmit and receive data or signals wirelessly using cellular communication, a wireless LAN (e.g., Wi-Fi), or the like. In an embodiment, the communication unit 210 may transmit and receive data or signals to and from an external device or an external server under the control of the processor 240.

[0087] The input unit 220 may receive various user commands through external operations. To this end, the input unit 220 may include or be connected to one or more input devices. For example, the input unit 220 may be connected to various input interfaces, such as a keypad or a mouse, to receive user commands. To this end, the input unit 220 may include not only a USB port but also an interface such as Thunderbolt. The input unit 220 may also include various input devices, such as a touch screen or a button, or may be coupled to these to receive external user commands.

[0088] The memory 230 can store programs and / or program instructions for the operation of the processor 240 and can temporarily or permanently store input and output data. The memory 230 can include at least one type of storage medium from the following: a flash memory type, a hard disk type, a multimedia card micro type, a card-type memory (e.g., SD or XD memory), RAM, SRAM, ROM, EEPROM, PROM, magnetic memory, a magnetic disk, and an optical disk.

[0089] The memory 230 may also store various network functions and algorithms, and may store various data, programs (instructions), applications, software, instructions, code, etc. for driving and controlling the device 200.

[0090] The processor 240 can control the overall operation of the device 200. The processor 240 can execute one or more programs or software stored in the memory 230. The processor 240 can refer to a central processing unit (CPU), a graphics processing unit (GPU), or a dedicated processor 240 on which the methods according to the embodiments of the present application are executed.

[0091] In an embodiment, processor 240 may acquire a target image, detect at least one defect area from the target image, generate a first defect matrix having a size corresponding to the target image and having first defect scores calculated based on the number of defect areas as elements, calculate multiple second defect scores by adding up first defect scores within a range corresponding to a predetermined reference window in the first defect matrix, and extract an image corresponding to the reference window from the target image based on the second defect scores to generate a defect image.

[0092] In an embodiment, the processor 240 may input the defect image as input data to a network function to generate surface defect information, where the network function may be trained to output surface defect information using an image of a size corresponding to the reference window as input data.

[0093] In an embodiment, the processor 240 may determine an element of the first defect matrix according to the number of defect regions detected from the target image at a position corresponding to the element.

[0094] In an embodiment, the processor 240 may calculate a second defect score, which is the sum of the first defect scores of the first defect matrix corresponding to the reference window, while moving a predetermined reference window relative to the first defect matrix. The processor 240 may generate a defect image by extracting an image corresponding to the position of the reference window where the second defect score is the largest from the target image.

[0095] In an embodiment, the processor 240 may apply the standard deviation for each local region of the target image to the first defect matrix to generate a second defect matrix. The processor 240 may calculate second defect scores for the second defect matrix by replacing the first defect matrix.

[0096] In an embodiment, the processor 240 may calculate standard deviations for each local region from the target image to generate a matrix of standard deviations, and may generate a second defect matrix by weighting the first defect matrix and the matrix of standard deviations.

[0097] In an embodiment, processor 240 may remove entries in the second defect matrix that correspond to non-defective regions to generate a third defect matrix. Processor 240 may replace the second defect matrix and calculate second defect scores for the third defect matrix.

[0098] In an embodiment, the processor 240 may generate a non-defective matrix having a size corresponding to the target image and representing at least one of the background and padding in the target image, and perform a logical AND operation on the non-defective matrix and the first defective matrix to generate a third defective matrix.

[0099] In an embodiment, the processor 240 generates boundary value information in the target image, generates boundary direction information in the target image, selects boundary value information that is above a predetermined threshold based on the boundary value information and the boundary direction information, and can detect defective areas based on the selected boundary value information and corresponding boundary direction information.

[0100] In an embodiment, the processor 240 may detect a defect area or generate a defect image when at least one of the horizontal size and vertical size of the target image is larger than that of the reference window.

[0101] In an embodiment, if at least one of the horizontal and vertical dimensions of the target image is smaller than the reference window, the processor 240 may add padding around the target image so that the target image corresponds to the reference window.

[0102] The device shown in FIG. 2 is exemplary and may be configured in various ways according to embodiments of the present application.

[0103] FIG. 3 is an example of a target image in which a defect is formed.

[0104] The defects shown in FIG. 3 occur on the surface of the cold-rolled steel sheet, and can refer to carbon, oil marks, roll marks, rust, scabs, scratches, and slip marks, in that order (a) to (g).

[0105] FIG. 3 is an exemplary diagram, and various configurations according to embodiments of the present application are applicable.

[0106] 4 and 5 are exemplary diagrams of image pre-processing for defect detection according to an embodiment of the present application.

[0107] Referring to FIG. 4, if the target image is smaller in both horizontal and vertical size than the reference window, padding may be added around the target image to create a defective image.

[0108] 5, a case where the horizontal size of the target image is smaller than the reference window and the vertical size of the target image is larger than the reference window is illustrated. In such a case, padding may be added to the target image in the horizontal direction smaller than the reference window, and a region that best represents the defect characteristics may be extracted in the vertical direction larger than the reference window to generate a defect image.

[0109] 4 and 5 are exemplary and various configurations may be applied according to embodiments of the present application.

[0110] FIG. 6 is a diagram for explaining a defect detection process according to an embodiment of the present application.

[0111] As shown in the figure, when a target image having defects formed on its surface is acquired, boundary detection or the like can be performed on the target image to detect at least one defect area. A first defect score can be calculated based on the defect area to generate a first defect matrix.

[0112] Also, a deviation matrix may be generated by calculating the standard deviation for each local region of the target image, and a second defect matrix may be generated by weightedly multiplying the deviation matrix and the first defect matrix.

[0113] Although not shown, a third defect matrix may be generated by removing non-defective areas (background and white space) from the second defect matrix. In the third defect matrix, second defect scores may be calculated while moving the reference window, and based on this, a defect image may be generated from the reference window at the position having the largest second defect score.

[0114] FIG. 6 is an exemplary diagram, and various configurations according to embodiments of the present application are applicable.

[0115] Methods according to embodiments of the present application may be embodied in the form of program instructions that can be executed by various computer means and stored on a computer-readable medium. The computer-readable medium may include, alone or in combination, program instructions, data files, data structures, and the like. The program instructions stored on the medium may be those specially designed and constructed for the present application, or those known and available to those skilled in the art of computer software. Examples of computer-readable storage media include magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include not only machine language code, such as produced by a compiler, but also high-level language code that can be executed by a computer using an interpreter, for example.

[0116] In addition, the method according to the disclosed embodiments may be provided in a computer program product, which can be traded as a commodity between a seller and a buyer.

[0117] The computer program product may include a software program and a computer-readable storage medium on which the software program is stored. For example, the computer program product may include a product in the form of a software program (e.g., a downloadable app) that is electronically distributed by an electronic device manufacturer or through an online marketplace (e.g., Google Play Store, App Store). For electronic distribution, at least a portion of the software program may be stored in or temporarily generated on a storage medium. In this case, the storage medium may be a storage medium on a manufacturer's server, an online marketplace server, or an intermediary server that temporarily stores the software program.

[0118] In a system consisting of a server and a client device, the computer program product may include a storage medium of the server or a storage medium of the client device. Furthermore, if a third device (e.g., a smartphone) communicatively connected to the server or the client device is present, the computer program product may also include a storage medium of the third device. Furthermore, the computer program product may include the S / W program itself, which is transmitted from the server to the client device or the third device, or transmitted from the third device to the client device.

[0119] In this case, one of the server, the client device, and the third device may execute the computer program product to perform the method according to the disclosed embodiments, or two or more of the server, the client device, and the third device may execute the computer program product to perform the method according to the disclosed embodiments in a distributed manner.

[0120] For example, a server (e.g., a cloud server or an artificial intelligence server) can execute a computer program product stored on the server to control client devices communicatively connected to the server to perform methods according to the disclosed embodiments.

[0121] Although the embodiments have been described in detail above, the scope of the present application is not limited thereto, and various modifications and improvements made by those skilled in the art using the basic concepts of the present application as defined in the claims below also fall within the scope of the present application.

Claims

1. 1. A surface defect detection method, comprising: acquiring a target image; detecting at least one defect region from the target image; generating a first defect matrix having a size corresponding to the target image and having first defect scores calculated based on the number of defect areas as elements; calculating a plurality of second defect scores by summing up first defect scores within a range of the first defect matrix corresponding to a predetermined reference window while moving the reference window within the first defect matrix; and generating a defect image by extracting an image corresponding to the reference window from the target image based on the second defect score. A method characterized by:

2. The method further includes inputting the defect image as input data to a network function to generate surface defect information, wherein the network function is trained to output the surface defect information using an image of a size corresponding to the reference window as input data. The method of claim 1.

3. The elements of the first defect matrix are determined by the number of defect areas detected in the target image at positions corresponding to the elements. The method of claim 1.

4. The step of generating the defect image is performed by extracting an image corresponding to the position of the reference window where the second defect score is the largest from the target image. The method of claim 1.

5. The method further includes applying a standard deviation for each local region of the target image to the first defect matrix to generate a second defect matrix; The step of calculating the second defect scores is performed on the second defect matrix by replacing the first defect matrix. The method of claim 1.

6. The step of generating the second defect matrix includes the steps of: calculating standard deviations for each local region from the target image to generate a matrix of standard deviations; and performing a weighted sum of the first defect matrix and the matrix of standard deviations. The method of claim 5.

7. generating a third defect matrix by removing elements corresponding to non-defective regions from the second defect matrix; The step of calculating the second defect scores is performed on the third defect matrix by replacing the second defect matrix. The method of claim 5.

8. generating the third defect matrix includes generating a non-defective matrix having a size corresponding to the target image and representing at least one of a background and padding in the target image; and performing a logical AND operation on the non-defective matrix and the first defect matrix. The method of claim 7.

9. The step of detecting the defective area includes the steps of generating boundary line value information in the target image, generating boundary line direction information in the target image, selecting boundary line value information equal to or greater than a predetermined threshold based on the boundary line value information and the boundary line direction information, and detecting the defective area based on the selected boundary line value information and the corresponding boundary line direction information. The method of claim 5.

10. The step of detecting the defect area or the step of generating the defect image are performed when at least one of the horizontal size and the vertical size of the target image is larger than that of the reference window. The method of claim 1.

11. If at least one of the horizontal size and the vertical size of the target image is smaller than the reference window, adding padding around the target image so that the target image corresponds to the reference window. The method of claim 1.

12. A recording medium for carrying out the method according to any one of claims 1 to 11, A computer program characterized by:

13. 1. An apparatus for detecting surface defects, comprising: a memory in which a program for detecting surface defects is stored; a processor that acquires a target image, detects at least one defect area from the target image, generates a first defect matrix having a size corresponding to the target image and including first defect scores calculated based on the number of the defect areas as components, calculates a plurality of second defect scores by adding up first defect scores within a range of the first defect matrix corresponding to a predetermined reference window while moving the reference window within the first defect matrix, and generates a defect image by extracting an image corresponding to the reference window from the target image based on the second defect scores. An apparatus characterized in that

14. The processor applies a standard deviation for each local region of the target image to the first defect matrix to generate a second defect matrix, and calculates a second defect score for the second defect matrix by replacing the first defect matrix.

14. The apparatus of claim 13.

15. The processor removes structures corresponding to non-defective regions in the second defect matrix to generate a third defect matrix; calculating second defect scores for the third defect matrix by substituting the second defect matrix; 15. The apparatus of claim 14.

Images