Industrial defect detection method and device, storage medium and electronic equipment

Abstract

The present disclosure belongs to the technical field of image processing, and relates to an industrial flaw detection method and device, a storage medium and an electronic device. The method comprises: acquiring a flaw detection image, inputting the flaw detection image into a pre-trained flaw detection model, and enabling the pre-trained flaw detection model to output a mask image; performing flaw contour detection on the mask image to obtain a flaw contour, and performing key degree calculation on the flaw contour to obtain a flaw key degree; and marking the flaw contour in the flaw detection image according to the flaw key degree to obtain a flaw marking image. The present disclosure does not need to collect flaw data as a training set, and does not need to mark the training set. The present disclosure is suitable for various industrial flaw detection, effectively adds frequency domain information in the mask image, improves the accuracy of flaw detection by using the sensitivity of the edge information of the flaw in the frequency domain, can filter non-key flaws, unifies the standards of flaw screening and determination, and makes the detection result more reasonable and accurate.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and more particularly to a method and apparatus for detecting industrial defects, a computer-readable storage medium, and an electronic device. Background Technology

[0002] In automated industrial production, product appearance defects are inevitable. Common types of defects include scratches, dirt, pits, and paint peeling. Therefore, industrial product quality inspection has become a crucial step in the entire manufacturing process. Due to the variety and morphology of defects, many manufacturing plants still rely on manual visual inspection. This method has many problems, such as high rates of false positives and false negatives, inconsistent evaluation standards, and rising labor costs year by year.

[0003] Existing deep learning-based industrial defect detection methods mostly employ supervised learning. First, a large number of defect images are collected. Then, the locations or regions of defects, as well as their types, are manually labeled to construct a training set. Next, a convolutional neural network is built, and the defect images and labeled information are fed into the network for training. In application, once a defect is detected, an alarm is triggered, and all detection results are output for manual judgment. The main problem with this industrial defect detection method is the difficulty in acquiring industrial defect images; it is hard to collect tens or even hundreds of thousands of diverse defect data points in a short period of time. Furthermore, manually labeling such a large amount of data is time-consuming and lacks accuracy.

[0004] Therefore, there is an urgent need in this field to develop a new method and apparatus for detecting industrial defects.

[0005] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] The purpose of this disclosure is to provide a method for detecting industrial defects, an apparatus for detecting industrial defects, a computer-readable storage medium, and an electronic device, thereby overcoming, to at least some extent, the technical problems of difficulty in obtaining samples and poor detection results caused by limitations in related technologies.

[0007] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0008] According to a first aspect of the present invention, a method for detecting industrial defects is provided, the method comprising:

[0009] A defect detection image is acquired and input into a pre-trained defect detection model so that the pre-trained defect detection model outputs a mask image.

[0010] The mask image is subjected to defect contour detection to obtain defect contours, and the defect contours are subjected to keyness calculation to obtain defect keyness.

[0011] The defect outline is marked in the defect detection image according to the defect criticality to obtain a defect-marked image.

[0012] In an exemplary embodiment of the present invention, before inputting the defect detection image into a pre-trained defect detection model so that the pre-trained defect detection model outputs a mask image, the method further includes:

[0013] Obtain a flawless image and extract spatial features from the flawless image to obtain spatial information features;

[0014] The flawless image is subjected to spatial transformation processing to obtain frequency domain information features, and the spatial information features and the frequency domain information features are subjected to feature concatenation processing to obtain a first reconstructed image;

[0015] A target loss function is obtained by calculating the loss function of the flawless image, the spatial information features, the frequency domain information features, and the first reconstructed image. The target loss function is then used to train the flaw detection model to be trained, resulting in a pre-trained flaw detection model.

[0016] In an exemplary embodiment of the present invention, the step of performing spatial transformation processing on the flawless image to obtain frequency domain information features includes:

[0017] The flawless image is transformed to obtain initial frequency domain features, and the initial frequency domain features are then subjected to feature extraction processing to obtain sampling features;

[0018] The sampled features are numerically corrected to obtain corrected features, and the corrected features are inversely transformed to obtain frequency domain information features.

[0019] In an exemplary embodiment of the present invention, the step of calculating the target loss function by performing loss function calculation on the flawless image, the spatial information features, the frequency domain information features, and the first reconstructed image includes:

[0020] A first loss function is obtained by performing loss function calculation on the flawless image and the spatial information features, and a second loss function is obtained by performing loss function calculation on the frequency domain information features and the flawless image;

[0021] A third loss function is obtained by calculating the loss function of the first reconstructed image and the flawless image, and a target loss function is obtained by weighting the first loss function, the second loss function and the third loss function.

[0022] In an exemplary embodiment of the present invention, the step of inputting the defect detection image into a pre-trained defect detection model, so that the pre-trained defect detection model outputs a mask image, includes:

[0023] The defect detection image is input into a pre-trained defect detection model so that the pre-trained defect detection model outputs a second reconstructed image;

[0024] The defect detection image and the second reconstructed image are subjected to pixel comparison processing to obtain the defect area, and the defect area is subjected to noise removal processing to obtain a mask image.

[0025] In an exemplary embodiment of the present invention, the step of calculating the criticality of the defect contour to obtain the defect criticality includes:

[0026] The defect parameters are obtained by performing parameter calculation on the defect contour, and the defect clarity is obtained by performing gradient calculation on the defect contour.

[0027] The criticality of a defect is obtained by calculating the criticality of the defect parameters and the defect clarity.

[0028] In an exemplary embodiment of the present invention, the step of annotating the defect contour in the defect detection image according to the defect criticality to obtain a defect-annotated image includes:

[0029] The defect outline is determined as a critical defect based on the criticality threshold corresponding to the criticality of the defect, and the number of defects is obtained by statistical processing of the critical defects.

[0030] Obtain a quantity threshold corresponding to the number of defects, and compare the number of defects with the quantity threshold to obtain a comparison result;

[0031] If the comparison result indicates that the number of defects is greater than the number threshold, the defect outline is marked in the defect detection image to obtain a defect-marked image.

[0032] According to a second aspect of the present invention, an apparatus for detecting industrial defects is provided, comprising:

[0033] The defect detection module is configured to acquire a defect detection image and input the defect detection image into a pre-trained defect detection model so that the pre-trained defect detection model outputs a mask image;

[0034] The parameter calculation module is configured to perform defect contour detection on the mask image to obtain defect contours, and to perform criticality calculation on the defect contours to obtain defect criticality.

[0035] The defect annotation module is configured to annotate the defect outline in the defect detection image according to the defect criticality to obtain a defect annotation image.

[0036] According to a third aspect of the present invention, an electronic device is provided, comprising: a processor and a memory; wherein the memory stores computer-readable instructions, which, when executed by the processor, implement the method for detecting industrial defects in any of the above exemplary embodiments.

[0037] According to a fourth aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method for detecting industrial defects in any of the above exemplary embodiments.

[0038] As can be seen from the above technical solutions, the industrial defect detection method, industrial defect detection device, computer storage medium, and electronic device in the exemplary embodiments of this disclosure have at least the following advantages and positive effects:

[0039] In the methods and apparatus provided by the exemplary embodiments of this disclosure, the pre-trained defect detection model does not require the collection of a large amount of defect data as a training set, nor does it require annotation of the training set. This solves the problems of high time costs and frequent false positives and false negatives caused by manual annotation, making it suitable for various types of industrial defect detection. Furthermore, by using the pre-trained defect detection model to output a mask image, frequency domain information is effectively added to the mask image, and the sensitivity of defect edge information in the frequency domain is used to improve the accuracy of defect detection. In addition, by annotating the defect contour in the defect detection image according to the criticality of the defect, non-critical defects can be filtered out, replacing traditional manual analysis, unifying the standards for defect screening and judgment, and making the detection results more reasonable and accurate.

[0040] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0041] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0042] Figure 1 The illustration schematically shows a flowchart of a method for detecting industrial defects according to an exemplary embodiment of the present disclosure;

[0043] Figure 2 This schematic diagram illustrates a process flow of a method for obtaining a pre-trained defect detection model in an exemplary embodiment of this disclosure.

[0044] Figure 3 The schematic diagram illustrates a flow chart of a spatial transformation processing method in an exemplary embodiment of the present disclosure;

[0045] Figure 4 The schematic diagram illustrates a flowchart of a method for calculating a loss function in an exemplary embodiment of this disclosure;

[0046] Figure 5 The schematic diagram illustrates a flowchart of a method for outputting a masked image in an exemplary embodiment of the present disclosure;

[0047] Figure 6 The schematic diagram illustrates a flowchart of a method for calculating keyness in an exemplary embodiment of this disclosure;

[0048] Figure 7 This schematically illustrates a flowchart of a method for marking defect outlines in a defect detection image according to an exemplary embodiment of the present disclosure;

[0049] Figure 8 This diagram illustrates the modules of an industrial defect detection method in an application scenario of an exemplary embodiment of this disclosure.

[0050] Figure 9 This schematic diagram illustrates the network structure of a defect detection model in an exemplary embodiment of the present disclosure.

[0051] Figure 10 This schematic diagram illustrates the network structure for defect testing in an exemplary embodiment of this disclosure.

[0052] Figure 11 The schematic diagram illustrates a flowchart of the defect analysis module in an exemplary embodiment of this disclosure;

[0053] Figure 12 This diagram illustrates a network model of an industrial defect detection method in an application scenario of an exemplary embodiment of this disclosure.

[0054] Figure 13 This schematic diagram illustrates the structure of an industrial defect detection apparatus according to an exemplary embodiment of the present disclosure;

[0055] Figure 14 This illustration schematically depicts an electronic device for implementing a method for detecting industrial defects according to an exemplary embodiment of the present disclosure;

[0056] Figure 15 The illustration schematically depicts a computer-readable storage medium for implementing a method for detecting industrial defects according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0057] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this disclosure more comprehensive and complete, and to fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a full understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more of the specific details omitted, or other methods, components, apparatus, steps, etc., can be employed. In other instances, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0058] The terms “a,” “an,” “the,” and “the” are used in this specification to indicate the presence of one or more elements / components / etc.; the terms “including” and “having” are used to indicate an open-ended inclusion and to mean that there may be other elements / components / etc. in addition to the listed elements / components / etc.; the terms “first” and “second” are used only as markings and are not a limitation on the number of objects.

[0059] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities.

[0060] In automated industrial production, product appearance defects are inevitable. Common types of defects include scratches, dirt, pits, and paint peeling. Therefore, industrial product quality inspection has become a crucial step in the entire manufacturing process. Due to the variety and morphology of defects, many manufacturing plants still rely on manual visual inspection. This method has many problems, such as high rates of false positives and false negatives, inconsistent evaluation standards, and rising labor costs year by year.

[0061] Existing deep learning-based industrial defect detection methods mostly employ supervised learning. First, a large number of defect images are collected. Then, the locations or regions of defects and their types are manually labeled to construct a training set. Next, a convolutional neural network is built, and the defect images and labeled information are fed into the network for training. In application, once a defect is detected, an alarm is triggered, and all detection results are output for manual judgment. The main problem with this industrial defect detection method is the difficulty in acquiring industrial defect images; it is hard to collect tens or even hundreds of thousands of diverse defect data points in a short period of time. Furthermore, manually labeling such a large amount of data is time-consuming and lacks accuracy.

[0062] In addition, current neural networks used for industrial defect detection only utilize spatial domain information of images, neglecting frequency domain information. For some defects with prominent edges, the frequency domain is more sensitive than the spatial domain, indicating significant room for improvement in algorithm detection accuracy. Furthermore, current defect detection results are judged manually, lacking automated filtering methods for non-critical defects.

[0063] In view of the problems existing in the related technologies, this disclosure proposes a method for detecting industrial defects. Figure 1 A flowchart illustrating a method for detecting industrial defects is shown, such as... Figure 1 As shown, the method for detecting industrial defects includes at least the following steps:

[0064] Step S110. Obtain the defect detection image and input the defect detection image into the pre-trained defect detection model so that the pre-trained defect detection model outputs a mask image.

[0065] Step S120. Perform defect contour detection on the mask image to obtain defect contours, and calculate the criticality of the defect contours to obtain defect criticality.

[0066] Step S130. Mark the defect outline in the defect detection image according to the defect criticality to obtain the defect annotation image.

[0067] In the exemplary embodiments of this disclosure, the pre-trained defect detection model does not require collecting a large amount of defect data as a training set, nor does it require labeling the training set. This solves the drawbacks of high time costs and frequent false positives and false negatives caused by manual labeling, making it suitable for various types of industrial defect detection. Furthermore, by using the pre-trained defect detection model to output a mask image, frequency domain information is effectively added to the mask image. The sensitivity of defect edge information in the frequency domain is utilized to improve the accuracy of defect detection. In addition, by labeling the defect contour in the defect detection image according to the defect criticality, non-critical defects can be filtered out, replacing traditional manual analysis, unifying the standards for defect screening and judgment, and making the detection results more reasonable and accurate.

[0068] The following section provides a detailed explanation of each step in the method for detecting industrial defects.

[0069] In step S110, a defect detection image is acquired and input into a pre-trained defect detection model so that the pre-trained defect detection model outputs a mask image.

[0070] In an exemplary embodiment of this disclosure, the defect detection image may be determined from an image to be processed acquired frame by frame using an industrial camera. Since there is a large amount of repetitive information in the image to be processed acquired frame by frame, keyframes that may contain defects can be extracted based on the production cycle as defect detection images for subsequent industrial defect detection.

[0071] For example, when a new product is delivered to the industrial production line every 10 seconds, a key frame that may have industrial defects can be captured every 10 seconds as a defect detection image D.

[0072] Furthermore, the defect detection model to be trained can be trained to obtain a pre-trained defect detection model.

[0073] In an optional embodiment, Figure 2 The flowchart illustrating the method for obtaining a pre-trained defect detection model is shown, such as... Figure 2 As shown, the method includes at least the following steps: in step S210, a flawless image is acquired, and spatial features are extracted from the flawless image to obtain spatial information features.

[0074] The flawless image can also be determined from images to be processed, captured frame by frame using an industrial camera. Furthermore, the acquired flawless image I... s As a training set, the flawless images I in the training set s The image size is W×H.

[0075] It is worth noting that the flawless images in this training set do not require manual annotation.

[0076] Furthermore, the flawless images in the training set can be subjected to image enhancement processing such as flipping, translating, rotating, and brightness adjustment, and then randomly cropped into a fixed size w×h to form a dataset I of processed flawless images.

[0077] The dataset I of processed flawless images is input into the designed flaw detection model for training.

[0078] The defect detection model to be trained is divided into two paths to process the defect-free image.

[0079] The lower-level path can extract and recover the spatial domain information of a flawless image. Specifically, after the flawless image undergoes spatial domain encoding and decoding, the spatial domain encoder and decoder can be used to extract spatial features from the flawless image to obtain spatial information features F. s .

[0080] The spatial domain codec can be a Unet network.

[0081] The Unet network comprises a preset number of downsampling modules and a preset number of upsampling modules. The preset number of downsampling modules downsample the input image / input image features layer by layer to obtain downsampled images at different scales. The preset number of upsampling modules upsample the downsampled image features output by the last downsampling module layer by layer to obtain upsampled image features at different scales. Upsampled image features belonging to the same scale are then connected across layers to obtain cross-layer connected image features. The scales of the image features output by the downsampling and upsampling modules correspond one-to-one.

[0082] Subsampling, also known as downsampling, has two main purposes: to make the image fit the size of the display area and to generate a thumbnail of the corresponding image. The principle of subsampling is to take an image of size M*N and downsample it by a factor of s, resulting in a resolution image of size (M / s)*(N / s), where s is a common divisor of M and N. If considering a matrix image, it means transforming the image within an s*s window of the original image into a single pixel, and the value of this pixel is the average value of all pixels within the window.

[0083] The main purpose of upsampling is to enlarge the original image so that it can be displayed on a higher resolution display device. The principle of upsampling is that image enlargement almost always uses interpolation methods, that is, inserting new elements between pixels of the original image using an appropriate interpolation algorithm.

[0084] In step S220, the flawless image is subjected to spatial transformation processing to obtain frequency domain information features, and the spatial information features and frequency domain information features are subjected to feature connection processing to obtain the first reconstructed image.

[0085] In an optional embodiment, Figure 3 A flowchart illustrating the spatial transformation processing method is shown, such as... Figure 3 As shown, the method includes at least the following steps: in step S310, the flawless image is transformed to obtain initial frequency domain features, and the initial frequency domain features are extracted to obtain sampling features.

[0086] The uplink of the defect detection model to be trained can extract and recover frequency domain information.

[0087] Specifically, the DCT (Discrete Cosine Transform) can be used to transform flawless images into initial frequency domain features.

[0088] DCT is widely used in image coding and is a standard for JPEG compression and MPEG-1 / 2. DCT is a second-best orthogonal transform derived under the minimum mean square error condition, second only to the KL transform, and is a lossless orthogonal transform. It is known for its fast computation speed and high accuracy, achieving an optimal balance between its ability to extract feature components and its computational speed.

[0089] Furthermore, a frequency domain codec is used to extract features from the initial frequency domain features to obtain the corresponding sampled features. This frequency domain codec can also be a Unet network. Therefore, the initial frequency domain features are input into the Unet network, which downsamples and extracts features to obtain the sampled features.

[0090] In step S320, the sampled features are numerically corrected to obtain corrected features, and the corrected features are inversely transformed to obtain frequency domain information features.

[0091] After obtaining the sampling features, the DCT correction unit is used to perform numerical correction processing on the sampling features.

[0092] For example, the DCT correction unit can be used to perform feature decoding and dynamic convolution on the sampled features to obtain the corresponding corrected features.

[0093] Furthermore, the frequency domain information feature F is obtained by performing an inverse transform on the modified features using IDCT (inverse DCT). f .

[0094] The IDCT transform is the inverse of the DCT transform.

[0095] In this exemplary embodiment, in the frequency domain, a deep learning algorithm is used to perform spatial transformation processing on the flawless image, which leverages the frequency domain features of flaws with significant edges and improves the accuracy of flaw detection.

[0096] After obtaining the frequency domain information features, feature concatenation processing can be performed on the spatial information features and the frequency domain information features.

[0097] Specifically, after a series of convolution operations on the output units of the defect detection model to be trained, the frequency domain information feature F from the upper path can be combined using the concat (feature map fusion function) function. f Spatial information features F of the lower paths Perform a connection and output the first reconstructed image R.

[0098] In step S230, a target loss function is calculated by performing loss function calculation on the flawless image, spatial information features, frequency domain information features, and the first reconstructed image. The target loss function is then used to train the flaw detection model to be trained, resulting in a pre-trained flaw detection model.

[0099] In an optional embodiment, Figure 4 A flowchart illustrating the method for calculating the loss function is shown, as follows: Figure 4 As shown, the method includes at least the following steps: in step S410, a first loss function is obtained by calculating the loss function of the flawless image and spatial information features, and a second loss function is obtained by calculating the loss function of the frequency domain information features and the flawless image.

[0100] The target loss function consists of three parts: loss1, loss2, and loss3.

[0101] loss1 is the first loss function, which is the sum of the flawless image I and the spatial information features F. s The difference between them can be calculated by comparing the flawless image I with the spatial information features F. s The mean square error between the two can also be obtained through other calculation methods, and this exemplary embodiment does not impose any special limitations on this.

[0102] loss2 is the second loss function, which is the frequency domain information feature F. f The difference between the flawless image I and the image F. This difference can also be calculated by computing the frequency domain information feature F. f The mean square error between the image and the flawless image I is obtained, but it can also be obtained through other calculation methods. This exemplary embodiment does not impose any special limitations on this.

[0103] In step S420, a third loss function is obtained by calculating the loss function of the first reconstructed image and the flawless image, and a weighted loss calculation is performed on the first loss function, the second loss function and the third loss function to obtain the target loss function.

[0104] loss3 is the third loss function, which is the difference between the first reconstructed image R and the flawless image I. This difference can also be obtained by calculating the mean squared error between the first reconstructed image R and the flawless image I as a similarity loss.

[0105] After obtaining the first loss function, the second loss function, and the third loss function, we can also obtain the weights α, β, and γ corresponding to the first loss function, the second loss function, and the third loss function, respectively. Where α + β + γ = 1.0.

[0106] Therefore, the target loss function is loss = αloss1 + βloss2 + γloss3.

[0107] In this exemplary embodiment, the target loss function is obtained by calculating the loss function of the flawless image, spatial information features, frequency domain information features, and the first reconstructed image. The calculation method is simple and accurate, and provides a data foundation and theoretical basis for training the flaw detection model to be trained.

[0108] After calculating the target loss function, the defect detection model to be trained can be trained using the target loss function to obtain a pre-trained defect detection model.

[0109] Specifically, an optimization function is selected to perform backpropagation on the defect detection model to be trained.

[0110] The optimization function can be the Adam optimizer.

[0111] For example, based on the calculated target loss function, the gradient descent method based on the Adam optimizer is used to optimize the hyperparameters such as the number of iterations and the learning rate in the defect detection model to be trained, and the model is trained until it converges to obtain the pre-trained defect detection model, which is then saved.

[0112] In this exemplary embodiment, a pre-trained defect detection model is obtained by training the detection model to be trained using a flawless image. This effectively incorporates frequency domain information and utilizes the sensitivity of defect edge information in the frequency domain, enabling the defect detection model to combine spatial and frequency domain information for defect detection, thereby improving the accuracy of defect detection.

[0113] After obtaining a pre-trained defect detection model, the defect detection image can be input into the pre-trained defect detection model so that the defect detection model outputs the corresponding mask image.

[0114] In an optional embodiment, Figure 5 A flowchart illustrating the method for outputting a masked image is shown, such as... Figure 5 As shown, the method includes at least the following steps: In step S510, the defect detection image is input into a pre-trained defect detection model so that the pre-trained defect detection model outputs a second reconstructed image.

[0115] The defect detection image D is input into the pre-trained defect detection model, and after the network calculation of the pre-trained defect detection model, the second reconstructed image R′ is output.

[0116] In step S520, pixel comparison processing is performed on the defect detection image and the second reconstructed image to obtain the defect area, and noise removal processing is performed on the defect area to obtain the mask image.

[0117] Since the pre-trained defect detection model has not learned the reconstruction of defective image samples, there is a large difference between the second reconstructed image R′ and the defect detection image D.

[0118] Therefore, pixel comparison processing can be performed on the second reconstructed image R′ and the defect detection image D.

[0119] Specifically, the second reconstructed image R′ and the defect detection image D are compared at the pixel level or block level.

[0120] For example, when comparing the second reconstructed image R′ with the defect detection image D at the pixel level, a single-channel mask image M with the same dimensions as the second reconstructed image R′ is constructed. The initial values ​​of this single-channel mask image M are all 0, making it a black image.

[0121] Furthermore, the pixel difference between the second reconstructed image R′ and the single-channel mask image M is calculated for each pixel. When the corresponding pixel difference between the second reconstructed image R′ and the single-channel mask image M is greater than the threshold Th1, the pixel is determined to be a defective region. The pixel value of this pixel is then marked as 255 in the single-channel mask image M. Subsequently, the obtained image is processed by erosion followed by dilation using a convolution kernel to obtain the corresponding mask image M. The size of the convolution kernel can be 3×3 or other sizes; this exemplary embodiment does not impose any special limitations on this.

[0122] For example, when comparing the second reconstructed image R′ and the defect detection image D at the block level, the second reconstructed image R′ and the defect detection image D can be divided into image blocks of the same size, such as 8*8 or 16*16 image blocks.

[0123] Furthermore, the mean value of pixels in the pixel block between the second reconstructed image R′ and the defect detection image D is calculated for each pixel block. When the mean value of pixels in the pixel block between the second reconstructed image R′ and the defect detection image D is greater than the threshold Th2, the pixel block is determined to be a defective block. Then, after removing defect marker noise by erosion and dilation of the image with the identified defective blocks, a binarized mask image M can be output.

[0124] In this exemplary embodiment, a pre-trained defect detection model can output a corresponding mask image, providing an image basis and theoretical support for detecting industrial defects, and also improving the accuracy of defect detection.

[0125] In step S120, defect contour detection is performed on the mask image to obtain defect contours, and the criticality of the defect contours is calculated to obtain defect criticality.

[0126] In an exemplary embodiment of this disclosure, after obtaining the mask image, the defect contours therein can be detected based on the binarized mask image M.

[0127] Specifically, it can be implemented using OpenCV or other graphics processing methods, and this exemplary embodiment does not impose any special limitations on it.

[0128] Furthermore, the criticality of the obtained defect contour can be calculated to obtain the defect criticality.

[0129] In an optional embodiment, Figure 6 A flowchart illustrating the criticality calculation method is shown, such as... Figure 6 As shown, the method includes at least the following steps: in step S610, the defect contour is parameterized to obtain defect parameters, and the defect contour is gradient-calculated to obtain defect clarity.

[0130] The defect parameters may include the outline length L and the inner area A of each defect contour, or other parameters, which are not specifically limited in this exemplary embodiment.

[0131] In addition, gradient calculations can be performed on the blemish outlines to obtain the blemish clarity of each blemish outline.

[0132] Specifically, for each detected defect contour, the edge sharpness of the defect, i.e., the defect sharpness E, is obtained by calculating the gradient. The minimum bounding rectangle of each defect contour is obtained on the mask image, and the corresponding rectangular image at the same position is obtained on the defect detection image D. The average gradient E is then calculated using the Sobel operator as the defect sharpness.

[0133] In step S620, the criticality of the defect is calculated by performing a criticality calculation on the defect parameters and defect clarity.

[0134] After obtaining the contour length L, the inner area A, and the sharpness E of the defect parameters, the criticality of the defect parameters and the sharpness can be calculated.

[0135] Specifically, the defect criticality K is calculated according to formulas (1) and (2):

[0136] K = αL K L +β A K A +γ E K E (1)

[0137] α L +β A +γ E =1.0 (2)

[0138] Among them, K L =|L / H-Th L | / Th L ,K A =|A / (W×H)-Th A | / Th A ,K E =|E-Th E | / Th E .

[0139] K L For information about the contour length L, K A Information about the area A within the defect.

[0140] Th L This is the threshold corresponding to the length of the defect outline, and its physical meaning is the maximum allowable value of the defect perimeter as a percentage of the image width;

[0141] Th A The threshold value for the area within the defect outline is the maximum allowable value of the defect area as a percentage of the image area.

[0142] Th E This is a threshold related to the clarity of the edge of the defect in the contour position. Its physical meaning is the maximum allowable value of the gradient of the defect area.

[0143] The L ,Th A and Th E It can be set based on experience or actual needs, and this exemplary embodiment does not impose any special limitations on it.

[0144] Furthermore, α can be adjusted based on the importance of different dimensions of evaluation. L β A γ E The value of , where α L β A γ E The larger the value, the more important the parameter of the corresponding dimension.

[0145] In this exemplary embodiment, the criticality of a defect can be calculated using defect parameters and defect clarity, providing data reference for judging whether a defect area is a defect of interest and ensuring the accuracy of defect labeling.

[0146] In step S130, the defect outline is marked on the defect detection image according to the defect criticality to obtain the defect marking image.

[0147] In an exemplary embodiment of this disclosure, after determining the defect criticality of each defect contour, the corresponding defect contour can be marked on the defect detection image according to the defect criticality.

[0148] In an optional embodiment, Figure 7 A flowchart illustrating a method for outlining defects in a defect detection image is shown, such as... Figure 7 As shown, the method includes at least the following steps: In step S710, the defect contour is determined as a critical defect according to the criticality threshold corresponding to the defect criticality, and the number of defects is obtained by statistical processing of the critical defects.

[0149] For the criticality K of a defect, a corresponding criticality threshold Th can be set. K When the defect criticality K is greater than the criticality threshold Th K When the defect outline is identified as a critical defect, the criticality threshold Th is determined. K The settings can be based on experience or actual conditions, and this exemplary embodiment does not impose any special limitations on this.

[0150] Furthermore, the number of identified critical defects can be statistically processed to obtain the total number of critical defects.

[0151] In step S720, a quantity threshold corresponding to the number of defects is obtained, and the number of defects and the quantity threshold are compared to obtain a comparison result.

[0152] After calculating the number of critical defects n, we can obtain the quantity threshold corresponding to the number of defects n, which is the preset standard number of defects N.

[0153] Furthermore, the number of defects n and the quantity threshold N can be compared to obtain the corresponding comparison results.

[0154] In step S730, if the comparison result shows that the number of defects is greater than the number threshold, the defect outline is marked on the defect detection image to obtain a defect marking image.

[0155] When the comparison between the number of defects n and the quantity threshold N results in the number of defects n being greater than the quantity threshold N, the corresponding defect contour is marked on the defect detection image D, and the corresponding defect annotation image D′ is output.

[0156] In this exemplary embodiment, the defect outline is marked on the defect detection image according to the defect criticality, which can filter non-critical defects from multiple dimensions, making the results of the detected defect marking image more reasonable and accurate.

[0157] The following describes in detail the industrial defect detection method in this disclosure embodiment with reference to an application scenario.

[0158] Figure 8 The diagram illustrates the modules of an industrial defect detection method in an application scenario, such as... Figure 8 As shown, the industrial defect detection method mainly includes an image acquisition module, a keyframe extraction module, a defect detection module, a defect analysis module, and an alarm module.

[0159] The image acquisition module can be an industrial camera that acquires images to be processed frame by frame.

[0160] Since there is a lot of repetitive information in the images to be processed acquired frame by frame, the keyframe extraction module can extract keyframes from the images to be processed according to the production cycle as defect detection images for detection.

[0161] The defect detection module can feed the defect detection image of the key frame into a pre-trained defect detection neural network for prediction.

[0162] The defect analysis module analyzes the output of the defect detection module and filters out non-critical defects. Then, it determines whether an alarm should be triggered based on the remaining critical defect areas.

[0163] The alarm module is used to trigger an alarm for manual secondary review or to automatically remove defective products when the defect analysis module determines that they are abnormal.

[0164] Specifically, the industrial defect detection method in the application scenario is implemented using an unsupervised dual-domain deep learning approach. This unsupervised dual-domain deep learning method consists of two parts: model training and testing phases.

[0165] During the training phase, flawless images I without any defects are collected. s As a training set, this data does not require manual labeling.

[0166] The flawless image can also be determined from images to be processed, captured frame by frame using an industrial camera. Furthermore, the acquired flawless image I... s As a training set, the flawless images I in the training set s The image size is W×H.

[0167] It is worth noting that the flawless images in this training set do not require manual annotation.

[0168] Furthermore, the flawless image I s Image enhancement processes such as flipping, translation, rotation, and brightness adjustment are performed, and the images are randomly cropped to a fixed size w×h to form a dataset I of processed, flawless images.

[0169] The dataset I of processed flawless images is input into the designed flaw detection model for training.

[0170] Figure 9 A schematic diagram of the network structure of the defect detection model is shown, such as... Figure 9 As shown, the defect detection model to be trained is divided into upper and lower paths to process the defect-free image.

[0171] The lower-level path can extract and recover the spatial domain information of a flawless image. Specifically, after the flawless image undergoes spatial domain encoding and decoding, the spatial domain encoder and decoder can be used to extract spatial features from the flawless image to obtain spatial information features F. s .

[0172] The spatial domain codec can be a Unet network.

[0173] The uplink of the defect detection model to be trained can extract and recover frequency domain information.

[0174] Specifically, DCT transformation can be used to transform flawless images into initial frequency domain features.

[0175] Furthermore, a frequency domain codec is used to extract features from the initial frequency domain features to obtain the corresponding sampled features. This frequency domain codec can also be a Unet network. Therefore, the initial frequency domain features are input into the Unet network, which downsamples and extracts features to obtain the sampled features.

[0176] The sampled features are numerically corrected to obtain corrected features, and the corrected features are then inversely transformed to obtain frequency domain information features.

[0177] After obtaining the sampling features, the DCT correction unit is used to perform numerical correction processing on the sampling features.

[0178] For example, the DCT correction unit can be used to perform feature decoding and dynamic convolution on the sampled features to obtain the corresponding corrected features.

[0179] Furthermore, the frequency domain information feature F is obtained by performing an inverse transform on the modified features using IDCT. f .

[0180] After obtaining the frequency domain information features, feature concatenation processing can be performed on the spatial information features and the frequency domain information features.

[0181] Specifically, after a series of convolution operations on the output units of the defect detection model to be trained, the concat function can be used to combine the frequency domain information features F from the upper path. f Spatial information features F of the lower path s Perform a connection and output the first reconstructed image R.

[0182] The first loss function is obtained by calculating the loss function of the flawless image and spatial information features, and the second loss function is obtained by calculating the loss function of the frequency domain information features and the flawless image.

[0183] The target loss function consists of three parts: loss1, loss2, and loss3.

[0184] loss1 is the first loss function, which is the sum of the flawless image I and the spatial information features F. s The difference between them can be calculated by comparing the flawless image I with the spatial information features F. s The mean square error between the two can also be obtained through other calculation methods, and this exemplary embodiment does not impose any special limitations on this.

[0185] loss2 is the second loss function, which is the frequency domain information feature F. f The difference between the flawless image I and the image F. This difference can also be calculated by computing the frequency domain information feature F. f The mean square error between the image and the flawless image I is obtained, but it can also be obtained through other calculation methods. This exemplary embodiment does not impose any special limitations on this.

[0186] loss3 is the third loss function, which is the difference between the first reconstructed image R and the flawless image I. This difference can also be obtained by calculating the mean squared error between the first reconstructed image R and the flawless image I as a similarity loss.

[0187] After obtaining the first loss function, the second loss function, and the third loss function, we can also obtain the weights α, β, and γ corresponding to the first loss function, the second loss function, and the third loss function, respectively. Where α + β + γ = 1.0.

[0188] Therefore, the target loss function is loss = αloss1 + βloss2 + γloss3.

[0189] After calculating the target loss function, the defect detection model to be trained can be trained using the target loss function to obtain a pre-trained defect detection model.

[0190] Specifically, an optimization function is selected to perform backpropagation on the defect detection model to be trained.

[0191] The optimization function can be the Adam optimizer.

[0192] For example, based on the calculated target loss function, the gradient descent method based on the Adam optimizer is used to optimize the hyperparameters such as the number of iterations and the learning rate in the defect detection model to be trained, and the model is trained until it converges to obtain the pre-trained defect detection model, which is then saved.

[0193] During the testing phase, Figure 10 A schematic diagram of the network structure for defect testing is shown, such as... Figure 10 As shown, after obtaining the pre-trained defect detection model, the defect detection image can be input into the pre-trained defect detection model so that the defect detection model outputs the corresponding mask image.

[0194] The defect detection image D is input into the pre-trained defect detection model, and after the network calculation of the pre-trained defect detection model, the second reconstructed image R′ is output.

[0195] Since the pre-trained defect detection model has not learned the reconstruction of defective image samples, there is a large difference between the second reconstructed image R′ and the defect detection image D.

[0196] Therefore, pixel comparison processing can be performed on the second reconstructed image R′ and the defect detection image D.

[0197] Specifically, the second reconstructed image R′ and the defect detection image D are compared at the pixel level or block level.

[0198] For example, when comparing the second reconstructed image R′ with the defect detection image D at the pixel level, a single-channel mask image M of the same length as the second reconstructed image R′ is constructed. The initial values ​​of this single-channel mask image M are all 0, making it a black image.

[0199] Furthermore, the pixel difference between the second reconstructed image R′ and the single-channel mask image M is calculated for each pixel. When the corresponding pixel difference between the second reconstructed image R′ and the single-channel mask image M is greater than the threshold Th1, the pixel is determined to be a defective region. The pixel value of this pixel is then marked as 255 in the single-channel mask image M. Subsequently, the obtained image is processed by erosion followed by dilation using a convolution kernel to obtain the corresponding mask image M. The size of the convolution kernel can be 3×3 or other sizes; this exemplary embodiment does not impose any special limitations on this.

[0200] For example, when comparing the second reconstructed image R′ and the defect detection image D at the block level, the second reconstructed image R′ and the defect detection image D can be divided into image blocks of the same size, such as 8*8 or 16*16 image blocks.

[0201] Furthermore, the mean value of pixels in the pixel block between the second reconstructed image R′ and the defect detection image D is calculated for each pixel block. When the mean value of pixels in the pixel block between the second reconstructed image R′ and the defect detection image D is greater than the threshold Th2, the pixel block is identified as a defect block. After removing defect marker noise by erosion and dilation of the image containing the identified defect blocks, a binarized mask image M can be output for subsequent defect analysis.

[0202] Specifically, the defect analysis module analyzes the results of the defect detection module.

[0203] Figure 11 A flowchart of the defect analysis module is shown, such as... Figure 11 As shown, the defect analysis module analyzes defects from dimensions such as length, area, edge clarity, and quantity, and filters out non-critical defects.

[0204] In step S1110, the flawed image D and the flaw detection mask image M are input.

[0205] The input to the defect analysis module is a defective image D and a defect detection mask image M, both with dimensions W×H. The defective image D is the defect detection image D, and the defect detection mask image M is the binarized mask image M.

[0206] In step S1120, the defect contour is detected based on the mask image M.

[0207] After obtaining the mask image, the defect contours can be detected based on the binarized mask image M.

[0208] Specifically, it can be implemented using OpenCV or other graphics processing methods, and this exemplary embodiment does not impose any special limitations on it.

[0209] In step S1130, the perimeter of the contour, the area inside the contour, and the edge sharpness are calculated for each defect contour.

[0210] The defect parameters are obtained by performing parameter calculations on the defect outline, and the defect clarity is obtained by performing gradient calculations on the defect outline.

[0211] The defect parameters may include the outline length L and the inner area A of each defect contour, or other parameters, which are not specifically limited in this exemplary embodiment.

[0212] In addition, gradient calculations can be performed on the blemish outlines to obtain the blemish clarity of each blemish outline.

[0213] In step S1131, the minimum bounding rectangle of the outline is obtained.

[0214] Specifically, for each detected defect contour, the edge sharpness of the defect, i.e., the defect sharpness E, is obtained by calculating the gradient. The minimum bounding rectangle of each defect contour is then obtained on the mask image.

[0215] In step S1132, the average gradient of the corresponding region on image D is calculated.

[0216] Furthermore, a rectangular image at the same position is obtained on the defect detection image D, and the average gradient E is calculated as the defect sharpness using the Sobel operator.

[0217] In step S1140, the defect criticality K is calculated.

[0218] After obtaining the contour length L, the inner area A, and the sharpness E of the defect parameters, the criticality of the defect parameters and the sharpness can be calculated.

[0219] Specifically, the defect criticality K is calculated according to formulas (1) and (2).

[0220] In step S1150, K>Th K .

[0221] For the criticality K of a defect, a corresponding criticality threshold Th can be set. K When the defect criticality K is greater than the criticality threshold Th K When the defect outline is identified as a critical defect, the criticality threshold Th is determined. K The settings can be based on experience or actual conditions, and this exemplary embodiment does not impose any special limitations on this.

[0222] In step S1160, the number of critical defects n = n + 1.

[0223] Furthermore, the number of identified critical defects can be statistically processed to obtain the total number of critical defects.

[0224] In step S1170, the number of critical defects n>N.

[0225] After calculating the number of critical defects n, we can obtain the quantity threshold corresponding to the number of defects n, which is the preset standard number of defects N.

[0226] Furthermore, the number of defects n and the quantity threshold N can be compared to obtain the corresponding comparison results.

[0227] In step S1180, the key defect contour image D′ and alarm signal are output.

[0228] When the comparison between the number of defects n and the quantity threshold N results in the number of defects n being greater than the quantity threshold N, the corresponding defect contour is marked on the defect detection image D, and the corresponding defect annotation image D′ is output.

[0229] Furthermore, defective signals are transmitted to the alarm module, where the alarm is manually reviewed or automatically removed from the repair line.

[0230] Figure 12 A schematic diagram of a network model for an industrial defect detection method in an application scenario is shown, such as... Figure 12 As shown, flawless washing machine panel images acquired frame-by-frame by an industrial camera are used as the training set, denoted as I. s .

[0231] Training set I s Images of flawless washing machine panels were randomly flipped, translated, rotated, and had their brightness adjusted for image enhancement. Then, the images were randomly cropped to 256×256 pixels to form dataset I of the processed flawless images.

[0232] A neural network for defect detection is constructed, and the encoders and decoders in both the spatial and frequency domains adopt the Unet structure.

[0233] The input is a processed, flawless image (256, 256, 3). The network performs an 8×8 block DCT transformation on I, resulting in an initial frequency domain feature of (256, 256, 1).

[0234] Furthermore, the input is fed into Unet, where downsampling extracts features into a (16, 16, 1024) sampled feature. After feature decoding, the input size is restored to (256, 256, 1). Then, after dynamic convolution, the output undergoes an 8x8 block inverse DCT transformation, transforming the frequency domain into the spatial domain (256, 256, 1) frequency domain information features.

[0235] In the lower network path, after I passes through Unet, the output spatial information feature F is obtained. s With the dimensions unchanged, the output is (256, 256, 3).

[0236] The network outputs are concatenated to form (256, 256, 4). After passing through two residual blocks and one convolutional layer, the first reconstructed image R is output.

[0237] The target loss function consists of three parts: loss1, loss2, and loss3.

[0238] loss1 is the first loss function, which is the sum of the flawless image I and the spatial information features F. s The difference between them can be calculated by comparing the flawless image I with the spatial information features F. s The mean square error between the two can also be obtained through other calculation methods, and this exemplary embodiment does not impose any special limitations on this.

[0239] loss2 is the second loss function, which is the frequency domain information feature F. f The difference between the flawless image I and the image F. This difference can also be calculated by computing the frequency domain information feature F. f The mean square error between the image and the flawless image I is obtained, but it can also be obtained through other calculation methods. This exemplary embodiment does not impose any special limitations on this.

[0240] loss3 is the third loss function, which is the difference between the first reconstructed image R and the flawless image I. This difference can also be obtained by calculating the mean squared error between the first reconstructed image R and the flawless image I as a similarity loss.

[0241] After obtaining the first loss function, the second loss function, and the third loss function, we can also obtain the weights α, β, and γ corresponding to the first loss function, the second loss function, and the third loss function, respectively. Where α + β + h = 1.0.

[0242] Therefore, the target loss function is loss = αloss1 + βloss2 + γloss3.

[0243] In this case, α = 0.2, β = 0.2, and γ = 0.6 are set.

[0244] The Adam optimizer is selected to perform backpropagation on the defect detection model to be trained.

[0245] For example, based on the calculated target loss function, the gradient descent method based on the Adam optimizer is used to optimize the hyperparameters such as the number of iterations and the learning rate in the defect detection model to be trained, and the model is trained until it converges to obtain the pre-trained defect detection model, which is then saved.

[0246] The keyframes of the image to be detected are extracted based on the production cycle. When a keyframe is a flawed image, that flawed image is the flaw detection image.

[0247] The defect detection image D is input into the pre-trained defect detection model, and after the network calculation of the pre-trained defect detection model, the second reconstructed image R′ is output.

[0248] The second reconstructed image R′ and the defect detection image D are compared at the pixel level.

[0249] Construct a single-channel mask image M with the same length as the second reconstructed image R′. The initial values ​​of this single-channel mask image M are all 0, making it a black image.

[0250] Furthermore, the pixel difference between the second reconstructed image R′ and the single-channel mask image M is calculated for each pixel. When the corresponding pixel difference between the second reconstructed image R′ and the single-channel mask image M is greater than the threshold Th1, the pixel is determined to be a defective region. The pixel value of this pixel is then marked as 255 in the single-channel mask image M. Subsequently, the obtained image is processed by erosion followed by dilation using a convolution kernel to obtain the corresponding mask image M. The size of the convolution kernel can be 3×3 or other sizes; this exemplary embodiment does not impose any special limitations on this.

[0251] After obtaining the mask image, the defect contours can be detected based on the binarized mask image M. Each defect contour represents a defect region.

[0252] The defect parameters are obtained by performing parameter calculations on the defect outline, and the defect clarity is obtained by performing gradient calculations on the defect outline.

[0253] The defect parameters may include the outline length L and the inner area A of each defect contour, or other parameters, which are not specifically limited in this exemplary embodiment.

[0254] In addition, gradient calculations can be performed on the blemish outlines to obtain the blemish clarity of each blemish outline.

[0255] Specifically, for each detected defect contour, the edge sharpness of the defect, i.e., the defect sharpness E, is obtained by calculating the gradient. The minimum bounding rectangle of each defect contour is obtained on the mask image, and the corresponding rectangular image at the same position is obtained on the defect detection image D. The average gradient E is then calculated using the Sobel operator as the defect sharpness.

[0256] After obtaining the contour length L, the inner area A, and the sharpness E of the defect parameters, the criticality of the defect parameters and the sharpness can be calculated.

[0257] Specifically, the defect criticality K is calculated according to formulas (1) and (2).

[0258] Among them, Th can be set L =0.02, Th A =0.005, Th E =15, α L =0.5, β A =0.3, γ E =0.2, but other values ​​can also be set. This exemplary embodiment does not impose any special limitations on this.

[0259] For the criticality K of a defect, a corresponding criticality threshold Th can be set. K The criticality threshold Th K It can be set to 0.2. When the defect criticality K is greater than the criticality threshold of 0.2, it is determined that a critical defect exists.

[0260] When a critical defect exists, that is, when the number of critical defects n>1, the defect outline of the critical defect is marked in the defect detection image, and the corresponding defect annotation image D′ is output.

[0261] The defective signal and defective label image D′ are transmitted to the alarm module, which then prompts the module to conduct a second manual review or automatically remove the defective washing machine panel to the repair line.

[0262] In this application scenario, the industrial defect detection method utilizes a pre-trained defect detection model. This eliminates the need for collecting large amounts of defect data as a training set or for labeling the training set, thus overcoming the drawbacks of high time costs and frequent false positives and false negatives associated with manual labeling. This method is applicable to various types of industrial defect detection. Furthermore, the pre-trained defect detection model outputs a mask image, effectively incorporating frequency domain information. The sensitivity of defect edge information in the frequency domain improves the accuracy of defect detection. In addition, labeling defect contours in the defect detection image based on their criticality allows for the filtering of non-critical defects, replacing traditional manual analysis and standardizing defect screening and judgment, resulting in more reasonable and accurate detection results.

[0263] Furthermore, in an exemplary embodiment of this disclosure, an apparatus for detecting industrial defects is also provided. Figure 13 A schematic diagram of an industrial defect detection device is shown, such as... Figure 13 As shown, the industrial defect detection device 1300 may include: a defect detection module 1310, a parameter calculation module 1320, and a defect marking module 1330. Wherein:

[0264] The defect detection module 1310 is configured to acquire a defect detection image and input the defect detection image into a pre-trained defect detection model so that the pre-trained defect detection model outputs a mask image.

[0265] The parameter calculation module 1320 is configured to perform defect contour detection on the mask image to obtain defect contours, and to perform criticality calculation on the defect contours to obtain defect criticality.

[0266] The defect annotation module 1330 is configured to annotate the defect outline in the defect detection image according to the defect criticality to obtain a defect annotation image.

[0267] In an exemplary embodiment of the present invention, before inputting the defect detection image into a pre-trained defect detection model so that the pre-trained defect detection model outputs a mask image, the method further includes:

[0268] Obtain a flawless image and extract spatial features from the flawless image to obtain spatial information features;

[0269] The flawless image is subjected to spatial transformation processing to obtain frequency domain information features, and the spatial information features and the frequency domain information features are subjected to feature concatenation processing to obtain a first reconstructed image;

[0270] A target loss function is obtained by calculating the loss function of the flawless image, the spatial information features, the frequency domain information features, and the first reconstructed image. The target loss function is then used to train the flaw detection model to be trained, resulting in a pre-trained flaw detection model.

[0271] In an exemplary embodiment of the present invention, the step of performing spatial transformation processing on the flawless image to obtain frequency domain information features includes:

[0272] The flawless image is transformed to obtain initial frequency domain features, and the initial frequency domain features are then subjected to feature extraction processing to obtain sampling features;

[0273] The sampled features are numerically corrected to obtain corrected features, and the corrected features are inversely transformed to obtain frequency domain information features.

[0274] In an exemplary embodiment of the present invention, the step of calculating the target loss function by performing loss function calculation on the flawless image, the spatial information features, the frequency domain information features, and the first reconstructed image includes:

[0275] A first loss function is obtained by performing loss function calculation on the flawless image and the spatial information features, and a second loss function is obtained by performing loss function calculation on the frequency domain information features and the flawless image;

[0276] A third loss function is obtained by calculating the loss function of the first reconstructed image and the flawless image, and a target loss function is obtained by weighting the first loss function, the second loss function and the third loss function.

[0277] In an exemplary embodiment of the present invention, the step of inputting the defect detection image into a pre-trained defect detection model, so that the pre-trained defect detection model outputs a mask image, includes:

[0278] The defect detection image is input into a pre-trained defect detection model so that the pre-trained defect detection model outputs a second reconstructed image;

[0279] The defect detection image and the second reconstructed image are subjected to pixel comparison processing to obtain the defect area, and the defect area is subjected to noise removal processing to obtain a mask image.

[0280] In an exemplary embodiment of the present invention, the step of calculating the criticality of the defect contour to obtain the defect criticality includes:

[0281] The defect parameters are obtained by performing parameter calculation on the defect contour, and the defect clarity is obtained by performing gradient calculation on the defect contour.

[0282] The criticality of a defect is obtained by calculating the criticality of the defect parameters and the defect clarity.

[0283] In an exemplary embodiment of the present invention, the step of annotating the defect contour in the defect detection image according to the defect criticality to obtain a defect-annotated image includes:

[0284] The defect outline is determined as a critical defect based on the criticality threshold corresponding to the criticality of the defect, and the number of defects is obtained by statistical processing of the critical defects.

[0285] Obtain a quantity threshold corresponding to the number of defects, and compare the number of defects with the quantity threshold to obtain a comparison result;

[0286] If the comparison result indicates that the number of defects is greater than the number threshold, the defect outline is marked in the defect detection image to obtain a defect-marked image.

[0287] The specific details of the aforementioned industrial defect detection device 1300 have been described in detail in the corresponding industrial defect detection method, so they will not be repeated here.

[0288] It should be noted that although several modules or units of the industrial defect detection device 1300 have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0289] Furthermore, in an exemplary embodiment of this disclosure, an electronic device capable of implementing the above-described method is also provided.

[0290] The following reference Figure 14 To describe an electronic device 1400 according to such an embodiment of the present invention. Figure 14 The electronic device 1400 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0291] like Figure 14 As shown, the electronic device 1400 is manifested in the form of a general-purpose computing device. The components of the electronic device 1400 may include, but are not limited to: at least one processing unit 1410, at least one storage unit 1420, a bus 1430 connecting different system components (including storage unit 1420 and processing unit 1410), and a display unit 1440.

[0292] The storage unit stores program code that can be executed by the processing unit 1410, causing the processing unit 1410 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of the present invention.

[0293] Storage unit 1420 may include readable media in the form of volatile storage units, such as random access memory (RAM) 1421 and / or cache memory 1422, and may further include read-only memory (ROM) 1423.

[0294] Storage unit 1420 may also include a program / utility 1424 having a set (at least one) of program modules 1425, such program modules 1425 including but not limited to: an operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0295] Bus 1430 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0296] Electronic device 1400 can also communicate with one or more external devices 1600 (e.g., keyboard, pointing device, Bluetooth device, etc.), one or more devices that enable a user to interact with electronic device 1400, and / or any device that enables electronic device 1400 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 1450. Furthermore, electronic device 1400 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 1460. As shown, network adapter 1460 communicates with other modules of electronic device 1400 via bus 1430. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 1400, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0297] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0298] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the methods described above is stored. In some possible embodiments, various aspects of the invention may also be implemented as a program product comprising program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps of the various exemplary embodiments of the invention described in the "Exemplary Methods" section above.

[0299] refer to Figure 15 As shown, a program product 1500 for implementing the above-described method according to an embodiment of the present invention is described. It may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0300] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0301] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0302] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0303] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0304] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

Claims

1. A method for detecting industrial defects, characterized in that, The method includes: A defect detection image is acquired and input into a pre-trained defect detection model so that the pre-trained defect detection model outputs a second reconstructed image; The defect detection image and the second reconstructed image are subjected to pixel comparison processing to obtain the defect region, and the defect region is subjected to noise removal processing to obtain a mask image; The mask image is subjected to defect contour detection to obtain defect contours, and the defect contours are subjected to keyness calculation to obtain defect keyness. Based on the severity of the defect, the defect outline is marked in the defect detection image to obtain a defect-marked image; The pre-trained defect detection model is obtained by training the defect detection model to be trained. The defect detection model to be trained is divided into two paths: the lower path is used to extract and recover the spatial domain information of the defect-free image, and the upper path is used to extract and recover the frequency domain information of the defect-free image. The training process is as follows: Obtain a flawless image and extract spatial features from the flawless image to obtain spatial information features; The flawless image is subjected to spatial transformation processing to obtain frequency domain information features, and the spatial information features and the frequency domain information features are subjected to feature concatenation processing to obtain a first reconstructed image; A first loss function is obtained by performing loss function calculation on the flawless image and the spatial information features, and a second loss function is obtained by performing loss function calculation on the frequency domain information features and the flawless image; A third loss function is obtained by calculating the loss function of the first reconstructed image and the flawless image. The first loss function, the second loss function and the third loss function are then weighted to obtain the target loss function. The target loss function is then used to train the flaw detection model to be trained to obtain the pre-trained flaw detection model.

2. The method for detecting industrial defects according to claim 1, characterized in that, The step of performing spatial transformation processing on the flawless image to obtain frequency domain information features includes: The flawless image is transformed to obtain initial frequency domain features, and the initial frequency domain features are then subjected to feature extraction processing to obtain sampling features; The sampled features are numerically corrected to obtain corrected features, and the corrected features are inversely transformed to obtain frequency domain information features.

3. The method for detecting industrial defects according to claim 1, characterized in that, The criticality calculation of the defect contour to obtain the defect criticality includes: The defect parameters are obtained by performing parameter calculation on the defect contour, and the defect clarity is obtained by performing gradient calculation on the defect contour. The criticality of a defect is obtained by calculating the criticality of the defect parameters and the defect clarity.

4. The method for detecting industrial defects according to claim 1, characterized in that, The step of annotating the defect contour in the defect detection image according to the defect criticality to obtain a defect-annotated image includes: The defect outline is determined as a critical defect based on the criticality threshold corresponding to the criticality of the defect, and the number of defects is obtained by statistical processing of the critical defects. Obtain a quantity threshold corresponding to the number of defects, and compare the number of defects with the quantity threshold to obtain a comparison result; If the comparison result indicates that the number of defects is greater than the number threshold, the defect outline is marked in the defect detection image to obtain a defect-marked image.

5. An industrial defect detection device, characterized in that, include: The defect detection module is configured to acquire a defect detection image and input the defect detection image into a pre-trained defect detection model so that the pre-trained defect detection model outputs a second reconstructed image; The defect detection image and the second reconstructed image are subjected to pixel comparison processing to obtain the defect region, and the defect region is subjected to noise removal processing to obtain a mask image; wherein, the pre-trained defect detection model is obtained by training the defect detection model to be trained, the defect detection model to be trained is divided into upper and lower paths, the lower path is used to extract and restore the spatial domain information of the defect-free image, and the upper path is used to extract and restore the frequency domain information of the defect-free image. The training process is as follows: obtain the defect-free image, and extract spatial features from the defect-free image to obtain spatial information features; perform spatial transformation processing on the defect-free image to obtain frequency domain information features, and then perform noise removal processing on the defect region to obtain the mask image. A first reconstructed image is obtained by performing feature concatenation processing on spatial information features and frequency domain information features; a first loss function is obtained by calculating a loss function on the flawless image and the spatial information features, and a second loss function is obtained by calculating a loss function on the frequency domain information features and the flawless image; a third loss function is obtained by calculating a loss function on the first reconstructed image and the flawless image, and a target loss function is obtained by weighting the first loss function, the second loss function and the third loss function; and a pre-trained flaw detection model is obtained by training the flaw detection model to be trained using the target loss function. The parameter calculation module is configured to perform defect contour detection on the mask image to obtain defect contours, and to perform criticality calculation on the defect contours to obtain defect criticality. The defect annotation module is configured to annotate the defect outline in the defect detection image according to the defect criticality to obtain a defect annotation image.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the industrial defect detection method according to any one of claims 1-4.

7. An electronic device, characterized in that, include: processor; Memory for storing the executable instructions of the processor; The processor is configured to execute the industrial defect detection method according to any one of claims 1-4 by executing the executable instructions.

Images