Workpiece defect detection method based on self-supervised twin comparison network

Abstract

The application provides a workpiece defect detection method based on a self-supervised twin comparison network, first, subgraphs are generated by dividing defect-free workpiece images through a sliding window, and a self-supervised similarity matrix label is calculated based on the spatial overlap of the subgraphs and the original graph; meanwhile, the subgraphs and the original graph are respectively subjected to enhancement processing; the enhanced subgraphs and the original graph are input into a twin feature extraction network, first, iterative optimization is performed through an affine adaptive architecture containing a FastMatch layer and an STNs layer: the FastMatch layer matches the features of the subgraphs and the original graph and outputs the highest similarity feature pair, and the STNs layer generates an affine matrix based on a full connection network to calibrate the perspective of the subgraphs; then, mutual convolution operation is performed on the optimized feature vectors to generate an initial similarity matrix; subsequently, a spatial similarity distribution graph is generated through a convolution network, and network model training is completed; during detection, the image to be detected and the reference image are input into the trained model after being aligned in perspective, and the existence of defects is determined based on the bidirectional similarity matching result and a threshold.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of non-destructive testing based on machine vision, and particularly relates to a workpiece defect detection method based on a self-supervised twin comparison network. BACKGROUND

[0002] The importance of industrial product quality detection is increasingly highlighted with the development of manufacturing refinement, especially in the fields of aerospace, automobile manufacturing and other safety-critical areas, where even minor defects can trigger a chain of failures. Traditional visual detection methods have gradually shown limitations in addressing such needs: early reliance on manual visual inspection is not only inefficient, but also affected by subjective factors such as the experience and fatigue of the inspector, making consistency difficult to guarantee; subsequent development of automated methods based on machine vision, such as template matching, edge detection, and morphological operations, have been applied in simple scenarios, but they essentially rely on manually designed features (such as gray gradient, texture histogram), and when there are reflections, stains on the workpiece surface, or slight appearance changes due to production batch differences, these methods are prone to false positives or missed detections. More critically, these methods are usually customized for specific workpieces, and when the detection object is changed, the feature extraction logic and judgment rules need to be redesigned, making it difficult to meet the rapid switching needs of flexible production lines.

[0003] The intervention of deep learning technology provides a new way to solve these pain points. Convolutional Neural Networks (CNN) have shown superior performance in image classification and object detection tasks due to their powerful feature learning capabilities. In the field of industrial defect detection, CNN-based methods can automatically capture complex features on the workpiece surface (such as texture distribution, structural integrity) through end-to-end learning, significantly improving tolerance to light changes and slight occlusions. Among them, twin networks are widely explored for defect detection because they focus on image similarity measurement. By comparing the feature differences between the image to be detected and the standard defect-free image, they can locate abnormal areas. This approach avoids the limitations of traditional methods that enumerate defect types, and in theory, it can detect unknown defects, making it more versatile.

[0004] However, the landing of twin networks in industrial scenarios still faces many realistic challenges. On the one hand, in industrial production, qualified workpieces account for the majority, defect samples are scarce and have various categories (such as scratches, depressions, bubbles, etc.), and it is difficult to construct a large-scale training dataset with annotations, while existing twin networks rely on supervised learning, and the performance decreases sharply when there is no annotation data. On the other hand, in industrial detection, the shooting angle of the workpiece often has uncontrollable changes, such as position deviation caused by conveyor belt vibration, slight adjustment of camera installation angle, etc., which will cause significant differences in image features of the same workpiece. The feature extraction module of the traditional twin network is not robust enough to such transformations, and it is easy to misjudge the normal angle difference as a defect. In addition, during network training, how to balance the "sensitivity to small defects" and "tolerance to normal variations" is also a difficulty. Overfitting to standard samples will lead to poor generalization ability, while relaxing the judgment threshold may introduce too many false positives, which all limit the further application of twin networks in industrial defect detection. SUMMARY

[0005] In view of the defects and deficiencies of the prior art, the present application provides a workpiece defect detection method based on a self-supervised twin comparison network, aiming to solve the problems of strong dependence on defect samples, high manual annotation cost, insufficient robustness to angle changes, and weak detection ability for small defects in traditional defect detection.

[0006] The method generates subgraphs by sliding window segmentation of non-defect workpiece images, calculates a self-supervised similarity matrix label based on the spatial overlap of subgraphs and original images, and does not rely on defect samples or manual annotation to realize the supervision signal of network training; at the same time, the subgraphs and original images are subjected to differential enhancement processing, the subgraph side focuses on channel compression, spatial exchange and other operations, and the original image is enhanced through brightness, contrast adjustment and other ways to improve the adaptability of the network to complex scenes. In the model training stage, the enhanced subgraph and original image are input into the twin feature extraction network, first through the affine adaptation architecture containing the FastMatch layer and the STNs layer, the FastMatch layer matches the subgraph and original image features and outputs the highest similarity feature pair, and the STNs layer generates an affine matrix to calibrate the subgraph angle, and strengthens the robustness of the network to angle changes; then the optimized feature vectors are subjected to mutual convolution operation to generate a similarity matrix, which is processed by a convolution network to generate a spatial similarity distribution graph, and the model training is completed by combining the information entropy weighted mean square error loss function, which strengthens the learning of high complexity areas through information entropy weight, and improves the small defect detection precision.

[0007] In the detection stage, the to-be-detected image and the reference image are input into the trained model after being matched with ORB and SIFT feature points to realize the perspective alignment, the comprehensive similarity is calculated based on the bidirectional maximum matching similarity and the spatial information entropy, and the low-confidence defect area is positioned through threshold screening. In addition, the method also evaluates the network performance through the comprehensive matching index, quantifies the model training quality through the matching accuracy, the average similarity and the category label, and ensures the stable and reliable detection effect.

[0008] The method of the application does not require defect samples and manual labeling, is suitable for complex workpiece scenes with few samples and many defect categories, significantly reduces the training difficulty, effectively improves the robustness to perspective changes and the detection ability of small defects by virtue of the affine adaptive architecture and the fine feature matching strategy, and has high practical application value.

[0009] The technical scheme specifically adopted by the application to solve the technical problem is:

[0010] A workpiece defect detection method based on a self-supervised twin comparison network, first divides a non-defect workpiece image into subgraphs through a sliding window, calculates a self-supervised similarity matrix label based on the spatial overlap of the subgraph and the original graph, and the generation of the label does not require defect sample labeling; meanwhile, the subgraph and the original graph are enhanced; the enhanced subgraph and the original graph are input into a twin feature extraction network, first iteratively optimized through an affine adaptive architecture containing a FastMatch layer (fast matching) and an STNs layer (spatial transformation network): the FastMatch layer matches the features of the subgraph and the original graph and outputs the highest similarity feature pair, and the STNs layer generates an affine matrix based on a full connection network to calibrate the perspective of the subgraph; then, mutual convolution operation is performed on the optimized feature vector to generate an initial similarity matrix; then, a spatial similarity distribution graph is generated through a convolution network to complete network model training; in the detection stage, the to-be-detected image and the reference image are input into the trained model after being perspective-aligned, and the defect existence is determined based on the bidirectional similarity matching result and the threshold.

[0011] Further, the process of calculating the self-supervised similarity matrix label based on the spatial overlap of the subgraph and the original graph comprises:

[0012] The position of the subgraph in the original graph is determined by the modulus operation and the division operation of the subgraph index and the number of subgraphs in the width direction of the original graph; the overlapping area of the subgraph and the original graph at a certain coordinate is the product of the effective overlap in the height direction and the effective overlap in the width direction, the effective overlap is the product of the absolute value of the difference between the coordinate and the height position of the subgraph and the step in the height / width direction, and the effective overlap is taken as a non-negative value; the similarity matrix label value is the overlapping area divided by the difference between (2*subgraph area) and the overlapping area.

[0013] Further, the enhancement processing comprises:

[0014] The enhancement processing of the sub-graph is channel compression, spatial exchange, random flip and adding Gaussian noise; the enhancement processing of the original graph is random change of brightness, contrast, saturation and hue, random gray scale change, random Gaussian blur and random transmission change.

[0015] Further, the matching process of the FastMatch layer includes:

[0016] The original graph feature vector is segmented by a sliding window with the same size as the sub-graph feature vector and a window size step, the similarity matrix of the sub-graph feature vector and the segmented original graph feature vector is calculated by cosine similarity or dot product operation of the feature vector, the original graph segmented feature vector with the highest similarity is selected and spliced with the sub-graph feature vector along the channel dimension to generate a comprehensive vector.

[0017] Further, the STNs layer includes two fully connected layers, and a Dropout layer is added to each layer, outputting an affine matrix tensor with 2 rows and 3 columns; the affine adaptive architecture repeats the joint processing of the FastMatch layer and the STNs layer at least 3 times to strengthen the affine adaptability.

[0018] Further, the network model training process uses an information entropy weighted mean square error loss function, which is calculated as follows: first, calculate the mean square error of the spatial similarity distribution map and the self-supervised similarity matrix label, the mean square error is the sum of the square errors of each position divided by the product of the height and width of the original graph and the number of sub-graphs, then the mean square error is weighted and summed with the information entropy matrix of the sub-image and divided by the total number of sub-images.

[0019] Further, the perspective alignment of the to-be-detected image and the reference image is realized by ORB and SIFT feature point matching; the bidirectional similarity matching result is the bidirectional maximum matching similarity of the to-be-detected image sub-graph and the corresponding sub-region of the reference image, including the maximum similarity of the to-be-detected image sub-graph matching the reference image, and the maximum similarity of the reference image sub-graph matching the to-be-detected image; the comprehensive similarity is obtained by weighting and fusing the indicators of the bidirectional maximum matching similarity after being interpolated and expanded to the size of the detected image and the spatial information entropy of the corresponding region, which is used to distinguish normal feature difference and defect signal; low confidence defect regions are screened by a pre-set threshold.

[0020] Further, it further includes the step of evaluating the performance of the network model by the comprehensive matching index, which is calculated based on the matching accuracy, the average similarity and the category label, wherein the matching accuracy is the ratio of the number of sub-graphs with consistent maximum similarity index and similarity exceeding the threshold to the total number of sub-graphs, and the average similarity is the sum of the bidirectional maximum similarity divided by the square of (2x number of sub-graphs).

[0021] And a workpiece defect detection system based on a self-supervised twin comparison network comprises:

[0022] An image preprocessing module is configured to generate subgraphs by sliding window segmentation on a non-defective workpiece image, calculate a self-supervised similarity matrix label based on the spatial overlap of the subgraphs and the original graph, and perform enhancement processing on the subgraphs and the original graph, respectively.

[0023] A twin feature extraction network module is configured to receive the enhanced subgraphs and the original graph, and perform iterative optimization through an affine adaptive architecture comprising a FastMatch layer and an STNs layer.

[0024] A similarity calculation module is configured to perform mutual convolution operation on the optimized feature vectors to generate an initial similarity matrix, and generate a spatial similarity distribution graph through a convolution network to complete network model training.

[0025] A defect detection module is configured to input a to-be-detected image and a reference image after perspective alignment into the trained model during detection, and determine the existence of defects based on the bidirectional similarity matching result and a threshold.

[0026] A computer device comprises a memory, a processor, and a computer program stored in the memory, and the processor implements the method described above when executing the computer program.

[0027] A non-transitory computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method described above.

[0028] Compared with the prior art, the present application and the preferred scheme thereof generate a self-supervised similarity matrix label based on the spatial overlap of subgraphs and original graphs, without relying on defect samples or manual labeling, thereby fundamentally solving the problems of difficult training data acquisition and high labeling cost in the prior art, and being particularly suitable for complex workpiece scenarios with few samples and many defect categories. Meanwhile, the affine adaptive architecture comprising the FastMatch layer and the STNs layer performs iterative optimization through feature matching and perspective calibration, greatly improves the robustness of the network to changes in illumination and perspective, and effectively avoids misjudgment caused by perspective differences in traditional methods.

[0029] The differential image enhancement strategy designs the enhancement means according to the characteristics of the subgraph and the original graph, further improves the generalization ability of the network; the information entropy weighted mean square error loss function strengthens the learning weight of the complex area, and enhances the detection sensitivity of the micro defect; the detection process based on the bidirectional similarity matching and the evaluation method of the comprehensive matching index jointly guarantee the accuracy and stability of the detection result, reduce the training difficulty, and ensure the reliability of the defect detection of the complex workpiece. BRIEF DESCRIPTION OF DRAWINGS

[0030] The application will be further described in detail below with reference to the drawings and specific embodiments:

[0031] Figure 1 is a network training data preprocessing step flowchart of an embodiment of the application.

[0032] Figure 2 is an image preprocessing flowchart of an embodiment of the application.

[0033] Figure 3 is an image label processing flowchart of an embodiment of the application.

[0034] Figure 4 is a feature extraction architecture diagram of a twin network proposed by an embodiment of the application.

[0035] Figure 5 is a CMI calculation flowchart of a twin network proposed by an embodiment of the application.

[0036] Figure 6 is an instance verification diagram of an embodiment of the application, in which (a), (b), (c) and (d) correspond to the detection results of four samples. DETAILED DESCRIPTION

[0037] To make the features and advantages of the application more obvious and easy to understand, the following embodiments are specifically described as follows:

[0038] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used in the specification have the same meaning as generally understood by those skilled in the art to which the present application belongs.

[0039] It should be noted that the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form, and in addition, it should be understood that when the terms "comprise" and / or "include" are used in the specification, they indicate the presence of a feature, step, operation, device, component and / or combination thereof.

[0040] The application provides a workpiece defect comparison scheme of a self-supervised twin comparison network, aiming to compare two pictures to obtain a spatial similarity matrix with translational invariance and affine adaptability.The scheme comprises a novel self-supervised twin comparison network and an evaluation index thereof, which are used to obtain a spatial similarity distribution map of the two pictures, and the network has the advantages of mutual convolution similarity calculation and affine adaptive architecture, and greatly improves the robustness to view angle changes while retaining the generalization performance of the mutual convolution network.In the actual application of workpiece defect detection, the self-supervised learning method shows superior generalization performance, and since it does not need to collect defect images or manually label labels, the acquisition of training data is simpler, the training difficulty is significantly reduced, and the network detection accuracy is high, and has high practical application value.

[0041] The novel self-supervised twin comparison network architecture provided by the application is mainly suitable for complex workpiece defect detection, and the network has the advantages of mutual convolution similarity calculation and affine adaptive architecture, and does not need to collect defect image samples or manually label labels, and is suitable for the case of small sample quantity, multiple defect categories and small defect, and significantly reduces the training difficulty while ensuring the network detection accuracy.

[0042] As a significant difference from the prior art, the application scheme adopts self-supervised learning with mutual convolution similarity calculation and affine adaptive architecture, does not need to collect abnormal samples or manually label labels, and is suitable for the case of small sample quantity, multiple defect categories and high training difficulty, has the advantages of significantly reducing the training cost, improving the detection efficiency and small defect detection capability.

[0043] In order to achieve the above-mentioned purpose, the implementation process of the embodiment of the application mainly includes three stages of model training stage, model verification and evaluation and complex workpiece defect detection method based on the network model.

[0044] In the model training stage, the actions performed include the following steps:

[0045] First, the picture input to the model training is segmented.As a preferred, the specific segmentation operation is to segment the picture according to a given step length based on a sliding window of a certain size, and the number of sub-pictures generated is related to the sliding window as follows:

[0046] ,

[0047] ;

[0048] Among them, And Respectively, the number of sub-pictures of the original image along the Direction and Direction, And The height and width of the original image. and The height and width of the sub-image, and Sliding windows respectively direction and The step size in the direction. As a preferred option, in actual testing, it can be taken as follows: , .

[0049] Based on the above segmentation results, corresponding sub-image labels are generated, such as... Figure 3 As shown, the label is a 3D tensor with a size of ( , , The label generation is based on the numerical value of spatial overlap (the label reflects the similarity relationship between the sub-image and the original image), and the label size is determined by the ratio of the sub-image to the original image. The numerical values ​​determine the positional relationship between the labels and sub-images:

[0050]

[0051]

[0052]

[0053] in, For the index of the sub-image, Original image The number of subgraphs in the direction segmentation. and These are the height and width coordinates of the label matrix, respectively. and Sliding windows respectively direction and The step size in the direction.

[0054] In the above formula, P W and P H These parameters, representing the window width and height respectively, control the spatial sliding range between the sub-image feature vector and the original image feature vector during cross-convolution. They directly affect the granularity and accuracy of similarity calculation and are core parameters controlling the spatial range of cross-convolution operations; their values ​​directly influence the granularity of feature matching and detection accuracy. Through collaboration with sliding window segmentation and affine transformations in STNs layers, these two parameters achieve a balance between global semantics and local details in workpiece defect detection.

[0055] The essence of this label is a "prior similarity" based on the spatial overlap between the subgraph and the original graph. It is used to guide the network to learn the spatial similarity relationship between different positions of the subgraph and the original graph. The goal of subsequent network training is to make the output similarity matrix close to the label. Therefore, the label can be understood as a "supervisory signal" for self-supervised learning.

[0056] Examples of the above process are shown below. Figure 2 As shown.

[0057] Simultaneously, image enhancement processing is performed on the segmented sub-images generated from the above segmentation results. Image enhancement techniques may include channel compression, spatial swapping, random flipping, and adding Gaussian noise. In channel compression processing, the two sub-images are compared according to a scaling factor. The fusion and spatial swapping operation randomly exchanges portions of two sub-images. While enhancing the sub-images, the same operation is performed on the corresponding labels of the sub-images.

[0058] The image information entropy is calculated for the sub-images after image enhancement. The formula for calculating the information entropy is:

[0059]

[0060] in, For gray levels, grayscale The probability of occurrence can be taken in the optimal solution. =100.

[0061] At the same time, the original image also needs to be enhanced. Image enhancement methods include at least: random changes in brightness, contrast, saturation, and hue; random grayscale changes; random Gaussian blur; and random transmission changes.

[0062] The above process serves as preprocessing, aiming to obtain the network's input data, target labels, and weights assigned in the loss function. The process is as follows: Figure 1 As shown.

[0063] Then, the enhanced sub-image and the enhanced original image are used as two inputs to the twin feature extraction network. The network obtains the feature vectors of the segmented sub-image and the original image, and the feature vectors of the original image are expanded to the same dimension as the sub-image.

[0064] In this invention, an FmSTNs architecture combining Fast Match and Affine Adaptive STNs is proposed. The purpose of this layer is to adjust the feature map using trained affine matrix parameters while the network extracts image features, enabling the network to acquire affine adaptability. The structure and processing flow of the Siamese feature extraction network are as follows: Figure 4 As shown:

[0065] After the first layer of CNN network, the primary feature vector is extracted from the segmentation subgraph and the original image.

[0066] The enhanced subgraph and the feature vector corresponding to the enhanced original image are input into the Fast Match layer. The layer divides the original image feature vector by a sliding window with the same size as the subgraph feature vector according to the window size step. The subgraph feature vector and the feature vector of the segmented original image are matched by dot product to obtain a similarity matrix as the input of the STNs network, estimate the affine transformation matrix, and splice the feature vector segmented from the original image with the highest similarity and the subgraph feature vector to obtain a comprehensive vector.

[0067] The similarity in this step is a feature vector level matching (calculated in the high-level feature space extracted by CNN), which is a feature alignment operation within the network. The goal is to capture semantic similarity and is used for feature enhancement (by splicing the highest similarity feature, the context information of the subgraph and the original image is fused, and the affine adaptability is improved).

[0068] The comprehensive vector is input into the STNs layer, which is composed of two fully connected layers. The two layers additionally add Dropout layers to prevent the network from failing to converge due to overfitting. This layer outputs Affine matrix tensor.

[0069] The affine matrix generated by the STNs layer can be dynamically adjusted P W and P H the spatial range, such as by rotating or scaling operations, to adapt the window to different views of the workpiece image. Its essence is to align the subgraph and the original image to make the model have affine adaptability, serving the case where the to-be-detected image and the template image are subject to view transformation during later verification.

[0070] The generated affine matrix tensor is used to adjust the view of the segmented subgraph, and all segmented subgraphs are subjected to the first view alignment.

[0071] The feature vectors extracted from the generated calibrated subgraph and the original image are input into the next twin network architecture, based on the joint architecture in the network, the feature vectors are extracted and calibrated repeatedly three times, so that the network has affine adaptability.

[0072] The subgraph image feature vector generated by the network is convolved with the original image feature vector to obtain the final similarity matrix, which is used to approximate the similarity of the label.

[0073] As a preferred solution, the cross convolution operation is a cross convolution operation of the subgraph feature vector and the original graph feature vector, to generate an initial similarity matrix; the convolutional network used includes 3-5 convolutional layers and pooling layers, which are used to extract high-level features from the optimized similarity matrix, to generate a spatial similarity distribution map with the same label size as the input similarity matrix.

[0074] In the cross convolution operation, the window width P W is consistent with the subgraph width, and the window height P H is consistent with the subgraph height, to realize global feature matching.

[0075] The similarity matrix is input into the CNN layer to obtain a high-level similarity matrix, and a similarity matrix tensor with the same size as the label size is output.

[0076] The network loss is calculated by the similarity matrix obtained above, the label formed in the foregoing step, and the image information entropy, and the loss function is:

[0077]

[0078]

[0079] Wherein, N is the total number of segmented subgraphs, is the network output tensor, is the generated label, is the information entropy matrix. Mean_Square refers to the mean square error, which is part of the network loss function calculation, and is used to measure the difference between the similarity matrix tensor output by the network and the self-supervised similarity matrix label.

[0080] After the model training is completed, the scheme further provides a verification and evaluation index of network training quality, and the CMI can be calculated through the process to effectively evaluate the training quality of the network, wherein when the CMI value is greater than 60, the network has good performance. The execution process is as shown in Figure 5

[0081] Two images are input, . The two images are of the same class, and label=1 is recorded, otherwise label=0. The two images are respectively cut into images by sliding windows with the same size, and the output , .

[0082] The images , and , are respectively input into the network, to obtain the similarity matrix of image 1 matching image 2 ​Similarity matrix of image 1 matching image 2 .

[0083] For the similarity matrix , sort it in dimension 1 (image dimension) to obtain the maximum similarity index The maximum similarity matrix corresponding thereto . For the similarity matrix , after sorting in dimensions 2 and 3 (spatial dimensions), adjust the size to The same, obtain the maximum similarity index thereof The maximum similarity matrix corresponding thereto .

[0084] The comprehensive matching index (CMI) is used as the network training quality evaluation index, when the CMI value is greater than 60, the network has good performance, and the CMI calculation formula is:

[0085]

[0086]

[0087]

[0088] Wherein, num is the total amount of test pictures, is the total number of segmentation subgraphs, is a matching threshold, and are mutual matching maximum similarity indexes, and are mutual matching maximum similarities, is an average similarity, is a matching precision, is a label, when two pictures are of the same class , and when they are of different classes .

[0089] The overall implementation process of the complex workpiece defect detection method provided by the embodiment of the application includes the following steps:

[0090] Step S1: 50-100 pictures of the same version of complex workpieces are shot based on an optical platform, and a picture of the same version without defects is selected as a reference image.

[0091] Step S2: select a detection target frame to intercept the picture, adjust the picture to the same size, and divide the picture into two groups of training pictures and verification pictures, and train the network according to the above network training model.

[0092] Step S3: Take the picture to be detected, align the image view angle using orb and sift feature point matching, and calculate the spatial information entropy of the detection sub-image and the detection reference image.

[0093] Step S4: Take the detection sub-image and the detection reference image as network input, and obtain the maximum mutual matching similarity of the detection image and the reference image based on the above network verification method 、 , and interpolate and enlarge them to the same size as the detection image. The similarity of the detection sub-image and the reference image is used to judge defects (low similarity area is a defect).

[0094] Step S5: Calculate the comprehensive similarity by the spatial information entropy obtained in step S3 and the maximum matching similarity obtained in step S4, and perform threshold screening to obtain the image spatial similarity matrix and its low confidence position.

[0095] As a preferred scheme of the embodiment, the image preprocessing means and matrix label in step S1 are used to start the segmentation operation at the initial position of the input image with a 96x96 sliding segmentation window. Each segmentation window generates a corresponding similarity matrix label. The segmented window and the similarity matrix label are spliced to form a corresponding sub-image and label.

[0096] As a preferred scheme of the embodiment, the enhanced sub-image and the enhanced original image in step S1 are used for image enhancement processing of the sub-image, including channel compression, spatial exchange, random flip, and adding Gaussian noise. The image enhancement processing of the original image includes random changes in brightness, contrast, saturation, hue, random gray scale, random Gaussian blur, and random transmission changes.

[0097] As a preferred scheme of the embodiment, the process in step S4 calculates the comprehensive matching index (CMI) evaluation to assess the network training quality. Two similar images (image 1 and image 2) are input into the network. For image 1 sub-image, the similarity matrix S 12 is calculated with image 2, and it is sorted in dimension 1 (image dimension). For image two sub-image, the similarity matrix S 21 is calculated with image 1, and it is sorted in dimensions 2 and 3 (spatial dimensions), so that there is no need for abnormal samples in the verification stage.

[0098] Based on the above embodiment scheme, in the four detection images as shown in Figure 6 , the left image is the reference image, the middle image is the image to be detected, and the right image is the heat map detection result. The closer to deep blue represents that the similarity between the left and middle images in the area is lower.

[0099] Based on the same inventive concept, the present application further provides a computer device, comprising: one or more processors, and a memory for storing one or more computer programs; the program comprises program instructions, and the processor is configured to execute the program instructions stored in the memory. The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, and are configured to implement one or more instructions, and are specifically configured to load and execute one or more instructions in the computer storage medium to implement the above method.

[0100] It needs to be further explained that, based on the same inventive concept, the present application further provides a computer storage medium, which stores a computer program, and the computer program is executed by the processor to perform the above method. The storage medium can adopt any combination of one or more computer readable media. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium may, for example, be but is not limited to an electrical, magnetic, optical, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples (non-exhaustive list) of the computer readable storage medium include: an electrical connection with one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or apparatus.

[0101] It should be noted that, unless otherwise defined, technical terms or scientific terms used in the present application shall have the common meaning understood by one of ordinary skill in the art to which the present application pertains. The terms "first", "second", and similar terms used in the present application do not denote any order, quantity, or importance, but are used to distinguish different components. The terms "comprise", "include", and similar terms mean that the elements or objects before the terms encompass the elements or objects listed after the terms and their equivalents, and do not exclude other elements or objects. The terms "connected" or "connected" and similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms "upper", "lower", "left", "right", and the like are used only to represent relative positional relationships, and when the absolute positions of the described objects are changed, the relative positional relationships can also be changed accordingly.

[0102] The above description is only the preferred embodiments of the present application, and is not intended to limit the present application in other forms. Any person skilled in the art can modify or change the above-mentioned disclosed technology into equivalent embodiments with equivalent changes. However, any simple modification, equivalent change, and modification of the above-mentioned embodiments without departing from the technical solution of the present application, and in accordance with the technical essence of the present application, still belongs to the protection scope of the present application.

[0103] The present application is not limited to the above-mentioned preferred embodiments, and anyone can derive other various forms of workpiece defect detection methods based on self-supervised twin comparison network under the inspiration of the present application. Any equivalent change and modification made in accordance with the scope of the present application shall be within the scope of the present application.

Claims

1. A workpiece defect detection method based on a self-supervised twin-comparison network, characterized by, First, the image of the non-defective workpiece is segmented by a sliding window to generate a subgraph, and a self-supervised similarity matrix label is calculated based on the spatial overlap of the subgraph and the original graph, which does not require defect sample labeling for label generation; meanwhile, the subgraph and the original graph are enhanced respectively; The enhanced subgraph and the original graph are input into a twin feature extraction network, which first iteratively optimizes through an affine adaptation architecture containing a FastMatch layer and an STNs layer: the FastMatch layer matches the subgraph and the original graph features and outputs the highest similarity feature pair, and the STNs layer generates an affine matrix based on a fully connected network to calibrate the subgraph perspective; then, the optimized feature vectors are subjected to mutual convolution operation to generate an initial similarity matrix; subsequently, a spatial similarity distribution map is generated through a convolution network to complete network model training; during detection, the image to be detected and the reference image are input into the trained model after perspective alignment, and the existence of defects is determined based on the bidirectional similarity matching result and the threshold value; The matching process of the FastMatch layer includes: The original graph feature vector is segmented by a sliding window with the same size as the subgraph feature vector and a step size equal to the window size, and the similarity matrix of the subgraph feature vector and the segmented original graph feature vector is calculated through cosine similarity or dot product operation of the feature vectors, and the original graph segmentation feature vector with the highest similarity is selected and concatenated with the subgraph feature vector along the channel dimension to generate a comprehensive vector; The STNs layer includes two fully connected layers, both of which are added with Dropout layers, and outputs an affine matrix tensor with 2 rows and 3 columns; the affine adaptation architecture repeats the joint processing of the FastMatch layer and the STNs layer at least 3 times to strengthen the affine adaptability.

2. The workpiece defect detection method based on a self-supervised twin comparison network according to claim 1, wherein: The process of calculating the self-supervised similarity matrix label based on the spatial overlap of the subgraph and the original graph includes: The position of the subgraph in the original graph is determined by the modulus operation and the division operation of the subgraph index and the number of subgraphs in the width direction of the original graph; the overlapping area of the subgraph and the original graph at a certain coordinate is the product of the effective overlap in the height direction and the effective overlap in the width direction, the effective overlap is the product of the absolute value of the difference between the coordinate and the height position of the subgraph and the step size in the height / width direction, taking the non-negative value; the similarity matrix label value is the overlapping area divided by the difference between (2×subgraph area) and the overlapping area.

3. The workpiece defect detection method based on a self-supervised twin comparison network according to claim 1, wherein: The enhancement processing includes: The enhancement processing of the subgraph includes channel compression, spatial exchange, random flipping and adding Gaussian noise; the enhancement processing of the original graph includes random changes in brightness, contrast, saturation, hue, random grayscale changes, random Gaussian blur and random transmission changes.

4. The self-supervised twin-comparison network-based workpiece defect detection method of claim 1, wherein: The network model adopts an information entropy weighted mean square error loss function in the network model training process, and the calculation method is as follows: first, the mean square error of the spatial similarity distribution map and the self-supervised similarity matrix label is calculated, the mean square error is the sum of the square errors of each position divided by the product of the height and width of the original image and the number of subgraphs, and then the mean square error and the information entropy matrix of the subgraph are weighted and summed and divided by the total number of subgraphs.

5. The self-supervised twin-comparison network-based workpiece defect detection method of claim 1, wherein: The view angle alignment of the to-be-detected image and the reference image is realized through ORB and SIFT feature point matching; the bidirectional similarity matching result is the bidirectional maximum matching similarity of the to-be-detected image subgraph and the corresponding sub-region of the reference image, including the maximum similarity of the to-be-detected image subgraph matching the reference image, and the maximum similarity of the reference image subgraph matching the to-be-detected image; the comprehensive similarity is obtained by weighting and fusing the bidirectional maximum matching similarity after being interpolated and expanded to the size of the detection image and the spatial information entropy of the corresponding region, which is used to distinguish normal feature difference and defect signal; the low confidence defect region is screened through a preset threshold.

6. The self-supervised twin-comparison network-based workpiece defect detection method of claim 1, wherein: Further comprising the step of evaluating the network model performance through the comprehensive matching index, the comprehensive matching index is calculated based on the matching accuracy, the average similarity and the category label, wherein the matching accuracy is the ratio of the number of subgraphs with consistent maximum similarity index and similarity exceeding the threshold to the total number of subgraphs, and the average similarity is the sum of the bidirectional maximum similarity divided by (2*number of subgraphs) squared.

7. A workpiece defect detection system based on a self-supervised twin-comparison network for implementing the method of claim 1, characterized by It comprises: An image preprocessing module for generating subgraphs by sliding window segmentation on the non-defective workpiece image, calculating a self-supervised similarity matrix label based on the spatial overlap of the subgraph and the original image, and enhancing the subgraph and the original image respectively, wherein the label generation does not require defect sample labeling; A twin feature extraction network module for receiving the enhanced subgraph and the original image, and iteratively optimizing through an affine adaptive architecture containing a FastMatch layer and an STNs layer; the FastMatch layer matches the subgraph and the original image features and outputs the highest similarity feature pair, and the STNs layer generates an affine matrix based on a fully connected network to calibrate the subgraph view angle; A similarity calculation module for performing mutual convolution operation on the optimized feature vectors to generate an initial similarity matrix, and generating a spatial similarity distribution map through a convolution network to complete the network model training; A defect detection module for inputting the to-be-detected image and the reference image after view angle alignment into the trained model to determine the existence of defects based on the bidirectional similarity matching result and the threshold. 8.A computer device, comprising a memory and a processor, and characterized in that, The memory stores a computer program, and the processor executes the computer program to realize the workpiece defect detection method based on the self-supervised twin comparison network in any one of claims 1-6.

Images