Deep learning image defect detection method based on zernike inverse transform

Abstract

The application discloses a deep learning image defect detection method based on Zernike inverse transformation, relates to the technical field of image processing and artificial intelligence, and comprises the following steps: reconstructing an image based on a Zernike inverse transformation mechanism; simulating a pseudo-defect image by disturbing the amplitude, phase or dimension of a Zernike moment; combining an original image and the pseudo-defect image to construct a training set, jointly training a deep learning image detection model, and constructing a total loss function through a standard cross-entropy loss and a Zernike structure preservation loss; and in a scenario where real defect data is missing, weakly supervising image defect detection training through a Zernike reconstruction error or the degree of outlying in a Zernike space. Therefore, the deep learning image defect detection method based on the Zernike inverse transformation is suitable for a situation where defect samples are lacking and can enhance the robustness of an image defect detection model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing and artificial intelligence, in particular to a deep learning image defect detection method based on Zernike inverse transformation. BACKGROUND

[0002] In the existing industrial visual inspection system, the deep learning method has become the mainstream, but it is severely dependent on the supervised training of a large number of real defect samples. However, in the actual industrial scene, the defect types are various, the distribution is uneven, the data acquisition cost is high, and the lack of defect samples leads to the difficulty of traditional binary classification supervised learning training, the weak generalization ability of the model, and the difficulty in maintaining high detection accuracy in the actual complex scene. In view of this problem, the method based on reconstruction error or One-Class classification attempts to solve the problem, but this method has shortcomings in structure perception ability and abnormal model ability, especially in the scene lacking real NG images, the training effectiveness is limited.

[0003] In addition, the existing data enhancement means (such as rotation, clipping, noise disturbance) cannot generate structured defects, lack of realism and physical consistency, and cannot effectively simulate edge wear, deformation, cracks and other defect patterns with physical characteristics, which limits the improvement of model performance.

[0004] In view of the above problems, the present application uses Zernike polynomials to map normal images to a feature space, and through controllable disturbance of specific Zernike moments, inverse transformation reconstruction is performed to generate simulated defect images, and a deep learning model is combined for joint training to realize enhanced learning and robust detection of defect images. SUMMARY

[0005] The purpose of the present application is to provide a deep learning image defect detection method based on Zernike inverse transformation, to solve the problem that the prior art cannot effectively simulate edge wear, deformation, cracks and other defect patterns with physical characteristics, so that the model can construct an effective supervision signal in the case of lack of defect samples or complete lack of defect samples, and enhance the robustness of the image defect detection model.

[0006] To achieve the above purpose, the present application provides a deep learning image defect detection method based on Zernike inverse transformation, comprising the following steps:

[0007] S1, define the input image in the unit circle region, map the image to a feature space through Zernike polynomials, and then reconstruct the image through the Zernike inverse transformation mechanism;

[0008] S2, set a disturbance strategy, disturb the amplitude, phase or dimension of the Zernike moment, simulate a pseudo-defect image, and combine the input image and the pseudo-defect image to construct a training set;

[0009] S3, dividing the pseudo-defect images into different pseudo-defect levels according to the degree of disturbance and constructing positive-negative pairs, and training a structure-aware encoder based on an improved contrastive loss for extracting structure feature embeddings of the images;

[0010] The pseudo-defect levels include mild defects, moderate defects, and severe defects.

[0011] S4, extracting structure features of a to-be-tested image based on the trained structure-aware encoder, and evaluating abnormality of the to-be-tested image using normal images or mild defect images in the training set, and identifying a defect image according to a set threshold.

[0012] Further, in S1, the image reconstruction includes retaining a limited number of Zernike moment coefficients as a basic expression form of the simulated defect image.

[0013] Further, in S2, the disturbance strategy includes:

[0014] high-order moment deletion, reconstructing images with blurred boundaries and contour wear to simulate mild wear or blur defects;

[0015] low-order moment enhancement, reconstructing contour deformation images to simulate geometric abnormal defects;

[0016] odd-order moment injection, breaking the balance of image structure to simulate crack and fault defects;

[0017] composite disturbance, combining high-order moment deletion, low-order moment enhancement, and odd-order moment injection strategies to simulate composite defects.

[0018] Further, S3 includes taking the Zernike space distance between the original normal image and the disturbed pseudo-defect image as a defect level measurement index, as follows:

[0019] ;

[0020] In the formula, is the original normal image, is the disturbed pseudo-defect image, denotes the Zernike moment coefficient vector.

[0021] Further, in S3, constructing positive-negative pairs includes: taking the original normal image and the mild defect as a positive pair; taking the original normal image and the moderate defect and the severe defect as a negative pair.

[0022] Further, in S3, constructing positive-negative pairs also includes: taking the original normal image and the randomly enhanced image as an auxiliary positive pair.

[0023] Further, in S3, the improved contrastive loss is as follows:

[0024] ;

[0025] wherein, is the original normal image, is the feature of the slightly disturbed or enhanced image, is the feature of the moderately disturbed / severely disturbed image, is the cosine similarity function, is the temperature scaling coefficient.

[0026] Further, in S4, the abnormality degree of the to-be-tested image is evaluated, including: comparing the structural feature of the to-be-tested image with the mean value of the structural features of the normal samples in the training data set as a first evaluation index; or comparing the structural feature of the to-be-tested image with the reference feature center of the slightly defective sample as a second evaluation index.

[0027] Further, in S4, the evaluation index of the abnormality degree for selection includes the L2 norm between the structural feature of the to-be-tested image and the mean value of the structural features of the normal samples, or the cosine similarity between the structural feature of the to-be-tested image and the reference feature center of the slightly defective sample.

[0028] Further, S4 further includes weighting and fusing the first evaluation index and the second evaluation index to comprehensively evaluate the abnormality degree of the to-be-tested image.

[0029] Therefore, the image defect detection method based on the Zernike inverse transform deep learning adopted by the present application has the following technical effects:

[0030] (1) The present application proposes an image reconstruction method based on Zernike polynomial inverse transform, which is used to controllably generate multi-type defect simulation images, and realizes physical simulation of different types of defects (such as blur, geometric deformation, and asymmetric structure) by disturbing specific orders and components of Zernike matrix, and has controllability and physical interpretability;

[0031] (2) The present application inputs the simulation defect image and the original image into the deep neural network, and introduces a hierarchical contrast mechanism, which improves the robustness and discrimination ability of the model to complex defects, is easy to integrate into existing industrial detection systems, and makes the model have training and reasoning ability under the condition of no defect sample or lack of defect sample.

[0032] The technical solutions of the present application will be further described in detail through the following embodiments. DETAILED DESCRIPTION

[0033] The present application can be explained in more detail through the following embodiments, and the purpose of the present application is to protect all changes and improvements within the scope of the present application, and the present application is not limited to the following embodiments.

[0034] The application provides a deep learning image defect detection method based on Zernike inverse transformation, comprising the following steps:

[0035] S1, Zernike polynomial is a family of orthogonal complex number functions defined on a unit circle, has good image reconstruction and rotation invariance, and is widely used for image shape description. Based on this feature, the input image is first defined in the unit circle region, the image is mapped to the feature space through the Zernike polynomial, and the image is reconstructed through the Zernike inverse transformation mechanism. Specifically, given the input image , the Zernike orthogonal expansion in the unit circle is represented as:

[0036] ;

[0037] In the formula, , are the radial order and the angular order of the polynomial respectively; is the highest order of the Zernike moment; represents the Zernike moment, which is the inner product of the input image and the Zernike basis function ; represents the Zernike orthogonal basis function, which contains a radial polynomial and an angle term, and is represented as , is the Zernike radial polynomial, , .

[0038] By selecting the first several low-order and middle-order coefficients for image reconstruction, the main structural information is retained and the detailed texture is removed, thereby providing a basis for subsequent defect simulation.

[0039] S2, by purposefully disturbing the amplitude, phase or dimension of the Zernike moment, different types of structural defects are introduced during image reconstruction. Specifically, the disturbance strategies include:

[0040] (1) high-order moment deletion: deleting high-order terms, simulating light wear or blur type defects, and reconstructing a blurred image.

[0041] (2) low-order moment enhancement: applying gain , is a gain factor, introduces structural deformation, simulates geometric abnormal defects such as expansion and distortion, and reconstructs a profile deformation image.

[0042] (3) odd-order moment injection: injecting odd-order terms into a symmetric image , is an odd number, breaks the balance of the image structure, and simulates defects such as cracks and faults.

[0043] (4) Compound disturbance: Synthesize the disturbance strategies of (1)-(3) above to construct multi-type, hierarchical structure pseudo-defect images with physical rationality and structural continuity, thereby obtaining a more realistic compound defect map.

[0044] Through the above mechanism, structure-consistent and physically reasonable pseudo-defect images are generated without relying on real defect samples, which can simulate real defect features, significantly improve the model training effect under small sample conditions, and have higher sensitivity to contour deformation, subtle damage and atypical defects, and realize the generation of visual structure real, defect level controllable training data set without non-normal samples (NG samples).

[0045] S3, based on the Zernike disturbance degree, construct the pseudo-defect level, and introduce the training strategy of the structure contrast learning mechanism, build an explicit discriminant structure space based on only positive samples, as follows:

[0046] First, according to the disturbance intensity, the pseudo-defect image is divided into mild defect (high-order deletion), moderate defect (low-order enhancement) and severe defect (odd injection), and the Zernike space distance between the original normal image and the pseudo-defect image after disturbance is defined as the defect level measurement index:

[0047] ;

[0048] In the formula, is the original normal image, is the pseudo-defect image after disturbance, represents the Zernike moment coefficient vector.

[0049] In one embodiment, the size of the Zernike space distance is used to judge the pseudo-defect level. Specifically, two threshold parameters and are set, and the defect level is determined by statistical analysis, as follows:

[0050] When , it is judged as a mild defect; when , it is judged as a moderate defect; when , it is judged as a severe defect.

[0051] Then, construct positive and negative pairs according to the pseudo-defect level, as follows:

[0052] The original normal image and the mild defect ( ): positive pair.

[0053] The original normal image With moderate defect / severe defect (M / S) / ): negative pairs.

[0054] Original normal image With other random augmented images (traditional random augmentation method obtains): auxiliary positive pairs.

[0055] Here, , , are pseudo-defect images after different perturbations of the same image

[0056] In addition, an improved contrastive loss is introduced to train a structure-aware encoder, and the expression is as follows:

[0057] ;

[0058] In the formula, is the improved contrastive loss; is the original normal image; is the feature of the lightly perturbed or augmented image; is the feature of the moderately perturbed / severely perturbed image, and the subscript is the label of the moderate perturbation / severe perturbation; is the cosine similarity function; is the temperature scaling coefficient.

[0059] In this embodiment, the improved contrastive loss is not a general "semantic contrast", but a structural level contrast. It makes use of the pseudo-defect levels generated by Zernike perturbation, so that the model explicitly learns the structural differences between "normal light moderate / severe". The advantages of the improved contrastive loss include: (1) learning structural sensitivity, effectively detecting subtle defects; (2) reducing the false negative rate, especially for light abnormalities; (3) having hierarchical discrimination ability, not just binary classification; (4) not relying on real defect samples for training; (5) more interpretable feature space, which can directly quantify defect levels.

[0060] Based on the above, this embodiment uses CNN or Transformer as the backbone network, cooperates with the projection head to learn the structure space embedding, uses the constructed positive and negative pairs to train a structure-aware encoder, and uses the trained structure-aware encoder to extract the structure feature embedding of the image, and then identifies the abnormal image. This hierarchical contrast mechanism enables the model to have strong structure discrimination ability, which is superior to the traditional One-Class method, and is practical, simple to deploy, and easy to integrate into existing industrial detection systems.

[0061] ​S4, in the test phase, the inference of the model does not need to input the defect image, and only one image to be detected can complete defect recognition, as follows:

[0062] (1) input image: single image to be detected .

[0063] (2) Zernike expansion: calculate the Zernike moment coefficient vector of the image to be detected . .

[0064] (3) feature extraction: use the trained structure perception encoder to extract the deep feature representation of the image to be detected . .

[0065] (4) based on the extracted deep feature representation , the image to be detected is evaluated for abnormality, and the larger the value is, the more abnormal it is. According to the set threshold, whether the image to be detected is abnormal can be determined. .

[0066] In one of the implementable embodiments, threshold values , and , are also set to distinguish between normal and defects, and are used to subdivide defects into different levels of mild / moderate / severe.

[0067] In another implementable embodiment, the evaluation method of abnormality includes two methods: one is to compare with the structural feature mean value of the normal samples in the training data set, which is suitable for the case of compact normal data distribution; the other is to compare with the reference feature center of the mild defect samples, which is more robust when there is redundancy and complex background. The specific steps of comparing with the feature mean value of the normal samples in the training data set are as follows: first, calculate the structural feature mean center of the original normal images in the training set , then calculate the abnormal score of the image to be detected as the evaluation index of abnormality. Since the training set mean value sometimes contains a large amount of redundant structure (such as repeated background, complex texture), resulting in an inaccurate structure center. Therefore, a method of comparing with the reference mild sample is designed, and the image of the mild defect in the training data set is used in this embodiment, which is synthesized by deleting the Zernike high-order moment coefficient from the original normal image, and has a slight degraded structure feature. The specific steps are as follows: set the reference feature center of the mild pseudo-defect image , For the number of light pseudo-defect images, the cosine similarity score of the to-be-tested image and the reference feature center of the light pseudo-defect image is calculated as an evaluation index of abnormality.

[0068] In other embodiments, the two abnormality evaluation methods described above can be weightedly fused, or one of them can be selected alone according to the characteristics of the data.

[0069] Therefore, the deep learning image defect detection method based on Zernike inverse transformation is adopted, the defect is simulated based on the image structure, the model has strong structure discrimination ability, and the robustness of the model detection is improved. At the same time, it does not depend on the collection of real defect images, but only depends on normal samples to complete the training, the specific implementation is simple and easy to operate, is compatible with mainstream deep models, is easy to integrate into existing industrial detection systems.

[0070] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application but not to limit it, although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can still be modified or replaced by equivalents, and these modifications or equivalent replacements cannot make the modified technical solutions deviate from the spirit and scope of the technical solutions of the present application.

Claims

1. A deep learning image defect detection method based on Zernike inverse transformation, characterized in that, The method comprises the following steps: S1, defining an input image in a unit circle region, mapping the image into a feature space through a Zernike polynomial, and reconstructing the image through a Zernike inverse transformation mechanism; S2, setting a perturbation strategy, perturbing the amplitude, phase or dimension of the Zernike moment, simulating a pseudo-defect image, combining the input image with the pseudo-defect image to construct a training set; S3, according to the degree of perturbation, dividing the pseudo-defect image into different pseudo-defect levels and constructing positive-negative pairs, and then training a structure-aware encoder based on an improved contrastive loss, for extracting the structure feature embedding of the image; Wherein, the pseudo-defect levels include mild defects, moderate defects and severe defects; In S3, the improved contrastive loss is as follows: ； wherein is the original normal image, is a feature of a mildly perturbed or enhanced image, is a feature of a moderately perturbed or heavily perturbed image, is a cosine similarity function, is a temperature scaling coefficient; S4, based on the trained structure-aware encoder, extracting the structure features of the to-be-tested image, and using normal images or mild defect images in the training set to evaluate the abnormality of the to-be-tested image, and then identifying the defect image according to the set threshold. 2.The Zernike inverse transform based deep learning image defect detection method of claim 1, wherein, In S1, the image reconstruction includes retaining a limited number of Zernike moment coefficients as the basic expression form of the simulated defect image. 3.The Zernike inverse transform based deep learning image defect detection method of claim 1, wherein, In S2, the perturbation strategy includes: High-order moment deletion, reconstructing the image with blurred boundaries and contour wear to simulate mild wear or blur defects; Low-order moment enhancement, reconstructing the contour deformation image to simulate geometric abnormal defects; Odd-order moment injection, breaking the balance of the image structure to simulate crack and fault defects; Composite perturbation, combining high-order moment deletion, low-order moment enhancement and odd-order moment injection strategies to simulate composite defects. 4.The Zernike inverse transform based deep learning image defect detection method of claim 1, wherein, S3 includes taking the Zernike space distance between the original normal image and the perturbed pseudo-defect image as a defect level measurement index, as follows: ； wherein is the original normal image, is the perturbed pseudo-defect image, denotes the Zernike moment coefficient vector. 5.The Zernike inverse transform based deep learning image defect detection method of claim 1, wherein, In S3, the positive-negative pairs are constructed, including: taking the original normal image and the mild defect as the positive pair; taking the original normal image and the moderate defect and the severe defect as the negative pair. 6.The Zernike inverse transform based deep learning image defect detection method of claim 1, wherein, In S3, the positive-negative pairs are also constructed, including: taking the original normal image and the randomly enhanced image as an auxiliary positive pair. 7.The Zernike inverse transform based deep learning image defect detection method of claim 1, wherein, In S4, the abnormality of the to-be-tested image is evaluated, including: comparing the structure feature mean of the normal sample in the training data set as the first evaluation index; or comparing the reference feature center of the mild defect sample as the second evaluation index.

8. The Zernike inverse transform based deep learning image defect detection method of claim 7, wherein, In S4, the evaluation index of the abnormality for selection includes the L2 norm between the structure feature of the to-be-tested image and the structure feature mean of the normal sample, or the cosine similarity between the structure feature of the to-be-tested image and the reference feature center of the mild defect sample. 9.The Zernike inverse transform based deep learning image defect detection method of claim 7, wherein, S4 also includes weighting and fusing the first evaluation index and the second evaluation index to comprehensively evaluate the abnormality of the to-be-tested image.

Images