Defect detection method and device, electronic equipment and storage medium

Abstract

The invention discloses a flaw detection method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a flawless template image, carrying out the decomposition of the flawless template image based on discrete wavelet transform, and carrying out the arrangement based on a decomposition result, and obtaining a wavelet domain hidden Markov tree model; based on a wavelet domain hidden Markov tree model, training through an expectation maximization algorithm to obtain a model parameter set; constructing a classifier through log-likelihood estimation based on the model parameter set; and based on the classifier, extracting a flaw region of the flaw image to be detected. According to the cloth surface flaw detection method using the wavelet domain hidden Markov tree (HMT) model, the image logarithm likelihood distribution of the normal sample is obtained by using the HMT, the calculation efficiency is higher, noisy points are fewer, and the false detection rate is lower. The method can accurately and efficiently realize flaw detection, and can be widely applied to the technical field of data processing.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a defect detection method, device, electronic equipment and storage medium. Background Art

[0002] Currently, due to the highly competitive business and manufacturing environment, the manufacturing industry has to face the continuous challenge of producing high-quality products at low costs. As a key factor in ensuring product quality, visual inspection plays an increasingly important role in modern industry. Since inspection based on human vision is usually expensive, laborious, time-consuming and prone to errors due to operator subjectivity and fatigue. Summary of the invention

[0003] The present invention proposes a defect detection method, device, electronic device and storage medium, aiming to solve one of the technical problems in the related art at least to a certain extent. The embodiments of the present invention can realize defect detection accurately and efficiently.

[0004] In one aspect, an embodiment of the present invention provides a defect detection method, comprising:

[0005] Acquire a flawless template image, decompose the flawless template image based on discrete wavelet transform, and obtain a wavelet domain hidden Markov tree model based on the decomposition result;

[0006] Based on the wavelet domain hidden Markov tree model, the model parameter set is obtained through expectation maximization algorithm training;

[0007] A classifier is constructed based on the model parameter set through log-likelihood estimation;

[0008] Based on the classifier, the defect area of ​​the defect image to be detected is extracted.

[0009] Optionally, the flawless template image is decomposed based on discrete wavelet transform, and a wavelet domain hidden Markov tree model is obtained based on the decomposition result, including the following steps:

[0010] Perform discrete wavelet transform on the flawless template image to obtain multiple sub-bands;

[0011] Based on the marginal statistics of the sub-bands, an independent mixture model is obtained;

[0012] Among them, the independent mixture model associates the coefficients of each variable with discrete hidden state;

[0013] The marginal density of each coefficient associated with the independent mixed model was modeled as a two-density Gaussian mixture model;

[0014] Based on the double-density Gaussian mixture model, the distribution of highly populated small-valued coefficients is captured by small variance, and the distribution of sparsely populated large-valued wavelet coefficients is represented by large variance Gaussian, thus obtaining a hidden Markov tree model in wavelet domain.

[0015] Optionally, based on the wavelet domain hidden Markov tree model, a model parameter set is obtained by training an expectation maximization algorithm, including the following steps:

[0016] Perform the expectation step to maximize the likelihood function by calculating the posterior probability of the latent variables of the wavelet domain hidden Markov tree model;

[0017] Perform a maximum step to update the parameters by maximizing the likelihood function of the observed data and the latent variables;

[0018] Return to the step of executing the expected step, through repeated iterations until the preset convergence condition is reached, and the model parameter set is obtained according to the parameters of each round of iteration.

[0019] Optionally, constructing a classifier based on the model parameter set by log-likelihood estimation comprises the following steps:

[0020] The log-likelihood estimation is performed based on the model parameter set to obtain the log-likelihood graph;

[0021] The log-likelihood graph is constructed as a classifier.

[0022] Optionally, extracting a defect area of ​​a defect image to be detected based on a classifier includes the following steps:

[0023] Based on the classifier, the target wavelet coefficients of the defective area of ​​the defective image are extracted by applying the threshold process;

[0024] Based on the target wavelet coefficients, the defect image is corrected by inverse two-dimensional wavelet transformation to obtain the reconstructed image;

[0025] The defect image includes a regular texture background and a defect area; in the reconstructed image, the defect area is highlighted.

[0026] Optionally, applying a threshold process to extract target wavelet coefficients of the defective area of ​​the defective image comprises the following steps:

[0027] The wavelet coefficients corresponding to the regular texture background in the defect image are set to zero to retain the target wavelet coefficients corresponding to the defect area.

[0028] Optionally, before the step of correcting the defective image by inverse two-dimensional wavelet transformation, the method further comprises the following steps:

[0029] Image restoration is performed on defective images through a high-pass filter.

[0030] On the other hand, an embodiment of the present invention provides a defect detection device, comprising:

[0031] The first module is used to obtain a flawless template image, decompose the flawless template image based on discrete wavelet transform, and obtain a wavelet domain hidden Markov tree model based on the decomposition result;

[0032] The second module is used to obtain the model parameter set based on the wavelet domain hidden Markov tree model through the expectation maximization algorithm training;

[0033] The third module is used to construct a classifier based on the model parameter set through log-likelihood estimation;

[0034] The fourth module is used to extract the defect area of ​​the defect image to be detected based on the classifier.

[0035] Optionally, the device further comprises:

[0036] The fifth module is used to restore the defective image through a high-pass filter.

[0037] On the other hand, an embodiment of the present invention provides an electronic device, including: a processor and a memory; the memory is used to store programs; the processor executes the program to implement the above-mentioned defect detection method.

[0038] On the other hand, an embodiment of the present invention provides a computer storage medium, in which a program executable by a processor is stored. The program executable by the processor is used to implement the above-mentioned defect detection method when executed by the processor.

[0039] The embodiment of the present invention obtains a flawless template image, decomposes the flawless template image based on discrete wavelet transform, obtains a wavelet domain hidden Markov tree model based on the decomposition result; obtains a model parameter set based on the wavelet domain hidden Markov tree model through expectation maximization algorithm training; constructs a classifier based on the model parameter set through log-likelihood estimation; and extracts the defect area of ​​the defect image to be detected based on the classifier. The embodiment of the present invention uses a cloth surface defect detection method using a wavelet domain hidden Markov tree (HMT) model, and uses HMT to obtain the log-likelihood distribution of a normal sample image, which has higher computational efficiency, less noise, and lower false detection rate. The present invention can accurately and efficiently realize defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings are used to provide a further understanding of the technical solution of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the technical solution of the present invention and do not constitute a limitation on the technical solution of the present invention.

[0041] Figure 1A schematic diagram of an implementation environment for defect detection provided by an embodiment of the present invention;

[0042] Figure 2 A schematic diagram of a flow chart of a defect detection method provided by an embodiment of the present invention;

[0043] Figure 3 A schematic diagram of the expansion process of the wavelet domain hidden Markov tree modeling provided by an embodiment of the present invention;

[0044] Figure 4 A schematic diagram of the expansion flow of the expectation maximization algorithm provided in an embodiment of the present invention;

[0045] Figure 5 A schematic diagram of the expansion process of constructing a classifier provided in an embodiment of the present invention;

[0046] Figure 6 A schematic diagram of a process for extracting defective areas provided by an embodiment of the present invention;

[0047] Figure 7 A schematic diagram of an extended process of extracting defective areas provided by an embodiment of the present invention;

[0048] Figure 8 A schematic diagram of the overall process principle of the defect detection method provided by an embodiment of the present invention;

[0049] Fig. 9 A schematic diagram of the structure of a defect detection device provided by an embodiment of the present invention;

[0050] Fig.10 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0052] It should be noted that, although the functional modules are divided in the system schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the system or the order in the flowchart. The terms "first / S100", "second / S200", etc. in the specification, claims and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0053] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0054] To facilitate the understanding of the technical solution of the present invention, the technical feature terms that may appear in the embodiments of the present invention are first explained:

[0055] Hidden Markov Model: Hidden Markov Model (HMM) is used to describe a Markov process with hidden unknown parameters. In the Markov model, each state represents an observable event, so the Markov model is sometimes called a visible Markov model (VMM).

[0056] EM (Expectation-Maximum): The EM algorithm is the most common latent variable estimation method and is widely used in machine learning. For example, it is often used to learn the parameters of Gaussian mixture model (GMM); Hidden Markov algorithm (HMM), variational inference of LDA topic model, etc. The EM algorithm is an iterative optimization strategy. Since each iteration in its calculation method is divided into two steps, one is the expectation step (E step) and the other is the maximum step (M step), the algorithm is called the EM algorithm (Expectation-Maximization Algorithm). The basic idea is: first, estimate the value of the model parameters based on the given observed data; then estimate the value of the missing data based on the parameter value estimated in the previous step, and then re-estimate the parameter value based on the estimated missing data plus the previously observed data, and then iterate repeatedly until convergence and the iteration ends.

[0057] It is understandable that the defect detection method provided in the embodiment of the present invention can be applied to any computer device with data processing and computing capabilities, and this computer device can be various terminals or servers. When the computer device in the embodiment is a server, the server is an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Optionally, the terminal is a smart phone, tablet computer, laptop computer, desktop computer, etc., but is not limited to this.

[0058] like Figure 1 FIG. 1 is a schematic diagram of an implementation environment provided by an embodiment of the present invention. Figure 1 The implementation environment includes at least one terminal 102 and a server 101. The terminal 102 and the server 101 can be connected to a network wirelessly or wired to complete data transmission and exchange.

[0059] Server 101 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as big data and artificial intelligence platforms.

[0060] In addition, the server 101 can also be a node server in the blockchain network. Blockchain is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, and encryption algorithm.

[0061] The terminal 102 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto. The terminal 102 and the server 101 may be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiment of the present invention.

[0062] Based on the example Figure 1 In the implementation environment shown, an embodiment of the present invention provides a defect detection method. The following is an example of the defect detection method being applied in the terminal 102 . It can be understood that the defect detection method can also be applied in the server 101 .

[0063] Reference Figure 2 , Figure 2 The flowchart of the defect detection method applied to the terminal provided by the embodiment of the present invention, the execution subject of the defect detection method can be any of the aforementioned computer devices (including servers or terminals). Figure 2 , the method comprises the following steps:

[0064] S100, obtaining a flawless template image, decomposing the flawless template image based on discrete wavelet transform, and obtaining a wavelet domain hidden Markov tree model based on the decomposition result;

[0065] It should be noted that in some embodiments, Figure 3 As shown, step S100 may include the following steps: S101, performing discrete wavelet transform on the flawless template image to obtain multiple sub-bands; S102, performing marginal statistics based on the sub-bands to obtain an independent mixture model; wherein the independent mixture model associates each coefficient with a discrete hidden state variable; S103, modeling the marginal density of each coefficient associated with the independent mixture model as a double-density Gaussian mixture model; S104, based on the double-density Gaussian mixture model, capturing the distribution of highly filled small-value coefficients by small variance, and representing the distribution of sparsely filled large-value wavelet coefficients by large variance Gaussian, to obtain a wavelet domain hidden Markov tree model.

[0066] Exemplarily, in some specific embodiments, for the input flawless template image, the present invention performs a discrete wavelet transform (DWT), which has multi-resolution analysis characteristics and is an effective multi-scale image analysis tool that can represent different singular content of images at different scales and sub-bands. In general, DWT decomposes any input image into multiple sub-bands, each of which contains different image information. That is, one sub-band A corresponds to an approximation of the original image, and the other three detailed sub-bands (H, V, and D) correspond to high-frequency coefficients in the horizontal, vertical, and diagonal directions, respectively. When the high-order dependencies of the wavelet coefficients are ignored, the marginal statistics of the DWT can be characterized by a simple statistical model, called an independent mixture model (IMM). It associates each coefficient with a discrete hidden state variable, which can be "large" or "small", and models the marginal density of each coefficient as a double-density Gaussian mixture model (GMM), where the small variance Gaussian captures the distribution of highly populated small-valued coefficients, and the large variance Gaussian represents the distribution of sparsely populated large-valued wavelet coefficients. It can be parameterized as a wavelet domain hidden Markov tree model:

[0067]

[0068] Among them, j is the coarsest scale, and J is the maximum value of the coarsest scale; It is the transition probability of the Markov chain from scale j+1 to j, and it and the three elements before and after it all characterize the statistical characteristics of the joint probability density of the image; m, n represent the coordinates converted into two-dimensional space, and b represents the quantitative representation of the degree to which the independent variable affects the dependent variable.

[0069] Among them, in some specific application scenarios, the wavelet domain classification hidden Markov model is used to denoise the flawless image. In this model, the wavelet coefficients of each subband of the image at each scale are divided into several groups to highlight the characteristics of its non-stationary changes in the spatial domain, so that the original wavelet quadtree is divided into several wavelet quadtrees with different HMT parameters.

[0070] S200, based on the wavelet domain hidden Markov tree model, the model parameter set is obtained by training the expectation maximization algorithm;

[0071] It should be noted that in some embodiments, Figure 4 As shown, step S200 may include the following steps: S201, execute the expected step, maximize the likelihood function by calculating the posterior probability of the hidden variables of the wavelet domain hidden Markov tree model; S202, execute the maximum step, update the parameters by maximizing the likelihood function of the observed data and the hidden variables; S203, return to the step of executing the expected step, through repeated iterations until the preset convergence condition is reached, and obtain the model parameter set according to the parameters of each round of iteration.

[0072] Exemplarily, in some specific implementations, for this model, the present invention trains its parameters and uses the maximum likelihood estimation algorithm to determine the coefficients therein; wherein, the maximum likelihood estimation and the EM algorithm are closely related. In fact, the EM algorithm can be regarded as an iterative method for solving the maximum likelihood estimation of the parameters of a probability model containing hidden variables. In the E step, the present invention maximizes the likelihood function by calculating the posterior probability of the hidden variables. In the M step, the present invention updates the parameters by maximizing the likelihood function of the observed data and the hidden variables. Therefore, the EM algorithm can be regarded as a special form of maximum likelihood estimation.

[0073] Specifically, in some specific application scenarios, train HMT parameters and use the EM algorithm: for the given θ in the previous step HMT , the present invention uses the EM algorithm as follows:

[0074] Step E:

[0075] 1. Using the finest scale j=1, calculate m=0,1:

[0076]

[0077] in is the backward probability, that is, when the state at scale j and position (x, y) is m, the posterior probability of this state under the observed data. is the forward probability, that is, given a certain observed data, the probability that the state at scale j and position (x, y) is m. g[.] This is a likelihood function. represents the probability of the observed data given the current state and the observed value, which refers to the joint probability of the observed value and the hidden state. This is the variance parameter of the Gaussian distribution, which is usually used to define the noise level in the observation model. represents the square of the observed data.

[0078] 2. For 1 < j < J, calculate for m = 0, 1:

[0079]

[0080] where This is the state transition probability from the previous scale j - 1 to the current scale j, that is, the probability of transitioning from the state at position n of scale j - 1 to the state at position m of scale j.

[0081] 3. For the coarsest scale j = J, calculate for m = 0, 1:

[0082]

[0083] This is the posterior probability, representing the probability that the state at scale J and position (x, y) is m for the given observed data.

[0084] 4. For 1 < j < J, calculate for m = 0, 1:

[0085]

[0086] M step:

[0087] Sweep the quad - tree downward, calculate for m, n = 0, 1:

[0088]

[0089] Among them, training the HMT parameter set and using the EM method: After reasonable initialization and training of the parameters by the Expectation - Maximization (EM) algorithm, inverse transformation and recovery, this method significantly improves the quality (PSNR) of the recovered image without increasing the computational complexity.

[0090] S300. Construct a classifier through log - likelihood estimation based on the model parameter set;

[0091] It should be noted that in some embodiments, such as Figure 5As shown, step S300 may include the following steps: S301, performing log-likelihood estimation based on a model parameter set to obtain a log-likelihood graph; S302, constructing a classifier from the log-likelihood graph.

[0092] For example, in some embodiments, a log-likelihood map (LLM) consisting of the likelihood of each coefficient can be efficiently constructed using the trained HMT. This LLM provides a good classifier for distinguishing defects from regular textures.

[0093] Specifically, in some specific application scenarios, after obtaining a series of HMT parameters θ HMT Afterwards, his log-likelihood can be estimated as:

[0094]

[0095] The conditional likelihood It is calculated in steps 1 and 2 of the E step in the EM algorithm.

[0096] Based on the obtained likelihood, the defect detection method proposed by the method is very simple. When the present invention inputs a flawless cloth texture image and a defective cloth texture image, the present invention first performs a wavelet transform to obtain a wavelet transformed image, and then converts it into a logarithmic likelihood graph, and the difference between them can be clearly seen through the image, although the original image may have very small defects.

[0097] S400: extracting a defect region of a defect image to be detected based on a classifier.

[0098] It should be noted that in some embodiments, Figure 6 As shown, step S400 may include the following steps: S401, based on the classifier, applying a threshold process to extract the target wavelet coefficients of the defect area of ​​the defect image; S402, based on the target wavelet coefficients, correcting the inverse two-dimensional wavelet transformation of the defect image to obtain a reconstructed image; wherein the defect image includes a regular texture background and a defect area; in the reconstructed image, the defect area is highlighted.

[0099] In some embodiments, applying a threshold process to extract target wavelet coefficients of a defective area of ​​a defective image may include the following steps: setting the wavelet coefficients corresponding to the regular texture background in the defective image to zero to retain the target wavelet coefficients corresponding to the defective area.

[0100] Among them, in some embodiments, such as Figure 7 As shown, before the step of correcting the defective image by inverse two-dimensional wavelet transformation, the method may further include the following steps: S403, restoring the defective image by using a high-pass filter.

[0101] For example, in some embodiments, by comparing any detected defect sample with the defect sample of the template LLM, the threshold process can generally set the coefficients corresponding to the regular texture background to zero while retaining the coefficients corresponding to the defect area. Then, in the reconstructed image obtained by the inverse two-dimensional WT of the correction coefficient, the texture pattern will be significantly eliminated, while the defect area will be clearly highlighted.

[0102] Specifically, after obtaining this classifier composed of log-likelihood map, when a defect picture is input, the threshold process can be applied by comparing any inspected sample with the template, and the wavelet coefficients corresponding to the regular texture background are set to zero, while retaining the wavelet coefficients corresponding to the defect area.

[0103] Since the difference in values ​​between LLMs is mainly concentrated in the lower frequency channels. Therefore, defective pixels must be propagated downward through the scale until they reach the pixel level. Errors at the coarse resolution level may greatly affect the final pixel-level segmentation results. The present invention uses image restoration to overcome this problem. Specifically, a high-pass filter is directly used to find the edges of the image to highlight the detailed defects of the image. Finally, the defective parts can be distinguished by comparing the difference loss between the final image result and the template LLM and setting a suitable threshold.

[0104] Among them, in some specific application scenarios, by comparing the log-likelihood of each coefficient of the defective image with the average log-likelihood value of all coefficients at the same scale and the sub-band of the non-defective template image, the coefficients corresponding to the defective area can be separated from the coefficients corresponding to the regular texture background as follows:

[0105]

[0106] Where c is a control constant, and the differences between LLMs are mainly concentrated in the lower frequency channels. Therefore, defective pixels must be propagated downward through the scale until they reach the pixel level. Errors at the coarse resolution level may greatly affect the final pixel-level segmentation results. The present invention uses image restoration to overcome this problem. Specifically, the present invention directly uses a high-pass filter to search for edges of the image to highlight the detailed defects of the image.

[0107] Threshold segmentation and defect result output: The present invention can segment defects or not through some simple statistical process control restrictions.

[0108]

[0109] Among them, f b (x, y) is the reconstructed image, μ r are the mean and standard deviation gray values ​​of the entire image.

[0110] In order to explain the principle of the technical solution of the present invention in detail, the overall process of the present invention is described below in combination with some specific embodiments. It is easy to understand that the following is an explanation of the technical principle of the present invention and cannot be regarded as a limitation of the present invention.

[0111] First of all, it should be stated that although a large number of machine vision-based inspection algorithms and systems have been developed and reported in the past few decades. However, for textured materials such as cloth, it is a rather challenging task to develop an automatic defect detection system due to the large variation in texture features. The surfaces of such materials usually present complex patterns that show high similarity throughout the image, while the grayscale distribution, pattern shape and pattern size of the texture components may vary locally and individually. These local variations make detection more difficult compared to untextured surfaces such as paper, glass panels and film webs, where traditional inspection techniques based on intensity or edge detection are usually sufficient. For uniformly textured surface inspection, defects usually correspond to local anomalies in the texture pattern. Therefore, defect detection algorithms are usually used as anomaly detectors, aiming to identify all areas where the uniformity of the texture pattern is locally disrupted.

[0112] In view of this, the method used in defect detection is actually a semi-supervised learning scheme. It is assumed that a perfect sample pattern containing no defects can be obtained in advance. This image will then be used as a template to facilitate inspection. For controlled environments in industry, such a template image can be easily obtained from a defect-free area of ​​the inspection surface. In addition, it is assumed that no defective samples are available before inspection. That is, the knowledge of potential defects (such as their number, size, shape and distribution) is unknown before testing. In this case, methods based on the Bayesian segmentation framework are difficult to adopt directly because they rely on a complete training set containing all types of defect samples. In addition, for real-time inspection applications, it is not feasible to estimate the model parameters of each defect sample during the inspection process using an iterative electromagnetic algorithm due to the high computational burden. The method uses a hidden Markov model for the likelihood value of each coefficient to provide a good clustering representation of the uniform texture area, which can be conveniently used to distinguish defects from the background.

[0113] Reference Figure 8 The technical solution of the present invention mainly includes two major processes: training and reasoning. The training process includes wavelet decomposition of defect-free template images, training HMT parameter sets and using EM methods. The reasoning process includes wavelet decomposition of texture images, possibility calculation and image restoration and defect result output.

[0114] Wavelet decomposition of flawless template image: Use wavelet domain classification hidden Markov model to denoise the flawless image. In this model, the wavelet coefficients of each subband of the image at each scale are divided into several groups to highlight the characteristics of its non-stationary changes in the spatial domain, so that the original wavelet quadtree is divided into several wavelet quadtrees with different HMT parameters.

[0115] Training HMT parameter set and using EM method: After reasonable initialization and expectation maximization (EM) algorithm training parameters, inverse transformation and restoration, this method significantly improves the quality (PSNR) of the restored image without increasing the amount of computation.

[0116] Texture image inspection based on wavelet decomposition: The texture is analyzed by using the wavelet domain hidden Markov tree model (HMT), and the requirements and characteristics of specific cloth images are organically combined for in-depth research. Since the wavelet coefficients do not satisfy the Gaussian distribution, they show a potential dependency relationship within the same scale and between scales. Therefore, the wavelet domain HMT model accurately reveals these dependencies between wavelet coefficients.

[0117] Likelihood Calculation and Image Restoration: By comparing any detected defect sample with the defect sample of the template LLM, the thresholding process can typically set the coefficients corresponding to the regular texture background to zero while retaining the coefficients corresponding to the defect area.

[0118] Defect result output: In the reconstructed image obtained by the inverse 2D WT of the correction coefficients, the texture pattern will be significantly eliminated, while the defect area will be clearly highlighted.

[0119] Exemplarily, in some specific implementations, the present invention may include the following process:

[0120] Wavelet decomposition of flawless template image: For the input flawless template image, a discrete wavelet transform (DWT) is performed, which has multi-resolution analysis characteristics and is an effective multi-scale image analysis tool that can represent different singular content of images at different scales and subbands. In general, DWT decomposes any input image into multiple subbands, each of which contains different image information. That is, one subband A corresponds to an approximation of the original image, and the other three detailed subbands (H, V, and D) correspond to high-frequency coefficients in the horizontal, vertical, and diagonal directions, respectively. When the high-order dependencies of the wavelet coefficients are ignored, the marginal statistics of the DWT can be characterized by a simple statistical model called the independent mixture model (IMM). It associates each coefficient with a discrete hidden state variable that can be "large" or "small", and models the marginal density of each coefficient as a double-density Gaussian mixture model (GMM), where the small variance Gaussian captures the distribution of highly populated small-valued coefficients, and the large variance Gaussian represents the distribution of sparsely populated large-valued wavelet coefficients. It can be parameterized as:

[0121]

[0122] where j is the coarsest scale, is the transition probability of the Markov chain from scale j + 1 to j.

[0123] Training HMT parameters and using the EM algorithm: For θ given in the previous step HMT , use the EM algorithm as follows,

[0124] E-step:

[0125] 1. Set j = 1 for the finest scale and calculate for m = 0, 1:

[0126]

[0127] where is the backward probability, i.e., the posterior probability of the state at scale j, position (x, y) being m under the observed data. is the forward probability, i.e., the probability that the state at scale j, position (x, y) is m given a certain observed data. g[.] is a likelihood function, represents the probability of the observed data given the current state and observation value, referring to the joint probability of the observation value and the hidden state. This is the variance parameter of the Gaussian distribution, usually used to define the noise level in the observation model, represents the square of the observed data.

[0128] 2. For 1 < j < J, calculate for m = 0, 1:

[0129]

[0130] where, this is the state transition probability from the previous scale j - 1 to the current scale j, i.e., the probability of transitioning from the state at position n in scale j - 1 to the state at position m in scale j.

[0131] 3. For the coarsest scale j = J, calculate for m = 0, 1:

[0132]

[0133] This is the posterior probability, representing the probability that the state at scale J, position (x, y) is m given the observed data.

[0134] 4. For 1 < j < J, calculate for m = 0, 1:

[0135]

[0136] M-step:

[0137] Scan down the quad tree and calculate m, n = 0, 1:

[0138]

[0139] Wavelet decomposition texture image inspection: After obtaining a series of HMT parameters θ HMT Afterwards, his log-likelihood can be estimated as:

[0140]

[0141] The conditional likelihood This is calculated in steps 1 and 2 of the previous step.

[0142] Based on the obtained likelihood, the defect detection method proposed by this method is very simple. When a flawless cloth texture image and a defective cloth texture image are input, wavelet transform is first performed to obtain a wavelet transformed image, which is then converted into a logarithmic likelihood graph. The difference between them can be clearly seen through the image, although the original image may have very small defects.

[0143] Likelihood calculation and image restoration: Now by comparing the log-likelihood of each coefficient of the defect image with the average log-likelihood value of all coefficients at the same scale and a subband of the non-defect template image, the coefficients corresponding to the defect area can be separated from the coefficients corresponding to the regular texture background as follows:

[0144]

[0145] Where c is the control constant, and the differences between LLMs are mainly concentrated in the lower frequency channels. Therefore, the defective pixels must be propagated downward through the scale until they reach the pixel level. Errors at the coarse resolution level may greatly affect the final pixel-level segmentation results. Image restoration is used to overcome this problem. Specifically, a high-pass filter is directly used to find the edges of the image to highlight the detailed defects of the image.

[0146] Threshold segmentation and defect result output: Some simple statistical process control limits can be used to determine whether defects are segmented or not.

[0147]

[0148] Among them, f b (x, y) is the reconstructed image, μ r are the mean and standard deviation gray values ​​of the entire image.

[0149] Specifically, taking an industrial quality inspection task as an example, in an actual application scenario, the present invention can be implemented as follows:

[0150] 1. For the input, some normal parts samples are collected from actual industrial production, that is, defect-free template images, and discrete wavelet transform (DWT) is performed. It has multi-resolution analysis characteristics and is an effective multi-scale image analysis tool that can represent different singular content of images at different scales and sub-bands. In general, DWT decomposes any input image into multiple sub-bands, each of which contains different image information. That is, one sub-band A corresponds to the approximation of the original image, and the other three detailed sub-bands (H, V, and D) correspond to high-frequency coefficients in the horizontal, vertical, and diagonal directions, respectively. When the high-order dependence of the wavelet coefficients is ignored, the marginal statistics of DWT can be characterized by a simple statistical model called the independent mixture model (IMM). It associates each coefficient with a discrete hidden state variable, which can be "large" or "small", and models the marginal density of each coefficient as a double-density Gaussian mixture model (GMM), where the small variance Gaussian captures the distribution of highly populated small-valued coefficients, and the large variance Gaussian represents the distribution of sparsely populated large-valued wavelet coefficients. It can be parameterized as a wavelet domain hidden Markov tree model.

[0151] 2. For this model, its parameters need to be trained and the maximum likelihood estimation algorithm is used to determine the coefficients; among them, the maximum likelihood estimation and the EM algorithm are closely related. In fact, the EM algorithm can be regarded as an iterative method for solving the maximum likelihood estimation of the parameters of a probability model containing hidden variables. In the E step, the likelihood function is maximized by calculating the posterior probability of the hidden variables. In the M step, the parameters are updated by maximizing the likelihood function of the observed data and the hidden variables. Therefore, the EM algorithm can be regarded as a special form of maximum likelihood estimation. Using the trained HMT, a log-likelihood map (LLM) consisting of the likelihood of each coefficient can be effectively constructed. This LLM provides a good classifier for distinguishing defects from regular textures.

[0152] 3. After obtaining this classifier composed of log-likelihood map, when the defect picture is input, the threshold process can be applied by comparing any inspected sample with the template, setting the wavelet coefficients corresponding to the regular texture background to zero while retaining the wavelet coefficients corresponding to the defect area.

[0153] 4. Since the difference in values ​​between LLMs is mainly concentrated in the lower frequency channels, the defective pixels must be propagated downward through the scale until they reach the pixel level. The error at the coarse resolution level may greatly affect the final pixel-level segmentation result. Image restoration is used to overcome this problem. Specifically, a high-pass filter is directly used to find the edges of the image to highlight the details of the image.

[0154] 5. By comparing the final image result with the difference loss of the template LLM, a suitable threshold can be set to distinguish defective parts.

[0155] In summary, the present invention provides a cloth surface defect detection method based on a wavelet domain hidden Markov tree (HMT) model, which is a detection method based on computer vision, i.e., image pixel level, which includes two consecutive stages, namely, training and inference. During the training process, the HMT of the wavelet transform (WT) of the defect-free template image acquired a priori is modeled using the expectation maximization (EM) algorithm. Using the trained HMT, a log-likelihood map (LLM) consisting of the likelihood of each coefficient can be effectively constructed. This LLM provides a good classifier for distinguishing defects from regular textures. During the inference process, by comparing any detected defect sample with the defect sample of the template using the LLM, the threshold process can usually set the coefficient corresponding to the regular texture background to zero while retaining the coefficient corresponding to the defect area. Therefore, in the reconstructed image obtained by the inverse two-dimensional WT of the corrected coefficients, the texture pattern will be significantly eliminated, while the defect area will be clearly highlighted.

[0156] Compared with the prior art, the present invention has at least the following beneficial effects:

[0157] 1. It solves the three problems of small number of defective samples, unknowns and a wide variety of defective samples in industrial production. The defect detection solution provided by this patent does not require a large number of defective samples for marking detection. On the premise of providing only normal samples, it can detect all industrial parts defects as comprehensively and quickly as possible.

[0158] 2. A cloth surface defect detection method using a wavelet domain hidden Markov tree (HMT) model is proposed. HMT is used to obtain the log-likelihood distribution of normal sample images, which has higher computational efficiency, less noise, and lower false detection rate.

[0159] 3. The obvious differences in LLM between the template and the defective sample are mainly concentrated in the lower frequency channels, and the coefficients at the fine resolution level usually provide little information for reconstruction and detection. The present invention uses high-pass filtering to deal with this, thereby solving the problem of requiring a sufficient number of decomposition levels, especially for images with small-sized defects.

[0160] On the other hand, Fig. 9 As shown, an embodiment of the present invention provides a defect detection device 900, comprising:

[0161] The first module 901 is used to obtain a flawless template image, decompose the flawless template image based on discrete wavelet transform, and obtain a wavelet domain hidden Markov tree model based on the decomposition result;

[0162] The second module 902 is used to obtain a model parameter set by training the wavelet domain hidden Markov tree model through an expectation maximization algorithm;

[0163] The third module 903 is used to construct a classifier based on the model parameter set through log-likelihood estimation;

[0164] The fourth module 904 is used to extract the defect area of ​​the defect image to be detected based on the classifier.

[0165] In some embodiments, the apparatus may further include:

[0166] The fifth module is used to restore the defective image through a high-pass filter.

[0167] The contents of the method embodiments of the present invention are all applicable to the device embodiments. The functions specifically implemented by the device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0168] On the other hand, an embodiment of the present invention further provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above sensitive information method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0169] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0170] like Fig.10 As shown, Fig.10 A specific example of the hardware structure of an electronic device 1000 according to an embodiment is shown. The electronic device 1000 includes:

[0171] The processor 1001 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention.

[0172] The memory 1002 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1002 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1002, and the processor 1001 calls and executes the network node population optimization method of the embodiment of the present invention;

[0173] Input / output interface 1003, used to implement information input and output;

[0174] The communication interface 1004 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0175] A bus 1005 , which transmits information between various components of the device (e.g., the processor 1001 , the memory 1002 , the input / output interface 1003 , and the communication interface 1004 );

[0176] The processor 1001 , the memory 1002 , the input / output interface 1003 and the communication interface 1004 are connected to each other in communication within the device via the bus 1005 .

[0177] The electronic device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0178] The contents of the method embodiments of the present invention are all applicable to the electronic device embodiments. The functions specifically implemented by the electronic device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0179] Another aspect of an embodiment of the present invention further provides a computer-readable storage medium, wherein the storage medium stores a program, and the program is executed by a processor to implement the above method.

[0180] It should be noted that the computer-readable medium shown in the embodiment of the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (Compact Disc Read to Only Memory, CD to ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0181] The contents of the method embodiments of the present invention are all applicable to the computer-readable storage medium embodiments. The functions specifically implemented by the computer-readable storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0182] The embodiment of the present invention also discloses a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device can read the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the above method.

[0183] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0184] It should be noted that, although several modules of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to an embodiment of the present invention, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0185] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD to ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation of the present invention.

[0186] In some selectable embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part for which is described as a larger operation is performed independently.

[0187] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise specified, one or more of the functions and / or features can be integrated into a single physical device and / or software module, or one or more functions and / or features can be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present invention. More specifically, in view of the properties, functions and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional skills of the engineer. Therefore, those skilled in the art can implement the present invention set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0188] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0189] The logic and / or steps represented in the flowchart or otherwise described herein, for example, may be considered as an ordered list of executable instructions for implementing logical functions, and may be embodied in any computer-readable medium for use by an instruction execution device, apparatus or device (such as a computer-based device, a device including a processor, or other device that can fetch instructions from an instruction execution device, apparatus or device and execute the instructions), or in conjunction with such instruction execution device, apparatus or device. For purposes of this specification, "computer-readable medium" may be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution device, apparatus or device, or in conjunction with such instruction execution device, apparatus or device.

[0190] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0191] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution device. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0192] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0193] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

[0194] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present invention.

Claims

1. A defect detection method, characterized in that: The following steps are involved: Acquire a flawless template image, decompose the flawless template image based on discrete wavelet transform, and obtain a wavelet domain hidden Markov tree model based on the decomposition result; Based on the wavelet domain hidden Markov tree model, a model parameter set is obtained by training through an expectation maximization algorithm; Constructing a classifier based on the model parameter set by log-likelihood estimation; Based on the classifier, a defect area of ​​the defect image to be detected is extracted.

2. The defect detection method according to claim 1, characterized in that: The method of decomposing the flawless template image based on discrete wavelet transform and obtaining a wavelet domain hidden Markov tree model based on the decomposition result comprises the following steps: Performing the discrete wavelet transform on the flawless template image to obtain a plurality of sub-bands; Performing marginal statistics based on the sub-bands to obtain an independent mixture model; wherein the independent mixture model associates coefficients of each discrete hidden state variable; modeling the marginal density of each of said coefficients associated with said independent mixture models as a two-density Gaussian mixture model; Based on the dual-density Gaussian mixture model, the distribution of highly filled small-value coefficients is captured by small variance, and the distribution of sparsely filled large-value wavelet coefficients is represented by large variance Gaussian, thereby obtaining the wavelet domain hidden Markov tree model.

3. The defect detection method according to claim 1, characterized in that: The method of obtaining a model parameter set based on the wavelet domain hidden Markov tree model by training with an expectation maximization algorithm comprises the following steps: Executing an expectation step to maximize the likelihood function by calculating the posterior probability of the latent variables of the wavelet domain hidden Markov tree model; Perform a maximum step to update the parameters by maximizing the likelihood function of the observed data and the latent variables; Return to the step of executing the expected step, and iterate repeatedly until a preset convergence condition is reached, and obtain the model parameter set according to the parameters of each round of iteration.

4. The defect detection method according to claim 1, characterized in that: The step of constructing a classifier based on the model parameter set by log-likelihood estimation comprises the following steps: Performing the log-likelihood estimation based on the model parameter set to obtain a log-likelihood graph; The log-likelihood map is used to form the classifier.

5. The defect detection method according to claim 1, characterized in that: The step of extracting the defect area of ​​the defect image to be detected based on the classifier comprises the following steps: Based on the classifier, applying a threshold process to extract target wavelet coefficients of the defective area of ​​the defective image; Based on the target wavelet coefficients, the defect image is corrected by inverse two-dimensional wavelet transformation to obtain a reconstructed image; The defect image includes a regular texture background and the defect area; in the reconstructed image, the defect area is highlighted.

6. The defect detection method according to claim 5, characterized in that: The step of applying a threshold process to extract target wavelet coefficients of the defective area of ​​the defective image comprises the following steps: The wavelet coefficients corresponding to the regular texture background in the defect image are set to zero to retain the target wavelet coefficients corresponding to the defect area.

7. The defect detection method according to claim 5, characterized in that: Before the step of correcting the defective image by inverse two-dimensional wavelet transformation, the method further comprises the following steps: The defective image is restored by using a high-pass filter.

8. A defect detection device, characterized in that: include: The first module is used to obtain a flawless template image, decompose the flawless template image based on discrete wavelet transform, and obtain a wavelet domain hidden Markov tree model based on the decomposition result; The second module is used to obtain a model parameter set by training the wavelet domain hidden Markov tree model through an expectation maximization algorithm; A third module is used to construct a classifier based on the model parameter set by log-likelihood estimation; The fourth module is used to extract the defect area of ​​the defect image to be detected based on the classifier.

9. An electronic device, characterized in that: including a processor and a memory; The memory is used to store programs; The processor executes the program to implement the method according to any one of claims 1 to 7.

10. A computer storage medium storing a program executable by a processor, characterized in that: The program executable by the processor is used to implement the method according to any one of claims 1 to 7 when executed by the processor.

Images