Component Defect Detection Method and System Based on Digital Speckle Spatiotemporal Features and IU-Net

Abstract

The present invention provides a method and system for detecting component defects based on the spatio-temporal characteristics of digital speckle and IU-Net. The method includes: building a digital shear speckle interferometry device, and then collecting shear speckle interferograms of component defects; extracting phase fringe patterns from the shear speckle interferograms, and filtering the noise of the phase fringe patterns by using a sine-cosine filtering method; taking the shear speckle interferograms of component defects as the time-domain characteristics of component defects, and taking the two-dimensional defect maps and three-dimensional wrapped phase maps after noise filtering as the spatial-domain characteristics of component defects; constructing a defect detection model based on IU-Net, inputting the spatial-domain characteristics of component defects into the trained defect detection model based on IU-Net to obtain an enhanced component defect map, and using the component defect map to realize the qualitative detection of component defects; inputting the time-domain characteristics of component defects and the spatial-domain characteristics of component defects into the trained defect detection model based on IU-Net for testing, identifying different types of defects, and realizing the quantitative detection of component defects.

Description

Technical Field

[0001] The present invention relates to the technical fields of laser non-destructive testing, optical image processing, and U-Net neural network, and particularly relates to a component defect detection method and system based on digital speckle spatio-temporal features and IU-Net. Background Art

[0002] In recent years, with the development of aerospace, various composite materials have been widely used. However, during the processing, manufacturing, and service of these composite materials, the performance of components is significantly reduced due to defects such as deformation and displacement, which has a great impact on the quality and working life of products. Therefore, non-destructive testing of key components of products is of great significance, especially in the fields of aerospace and precision manufacturing.

[0003] Shearing speckle interferometry technology has been widely used in component defect detection due to its advantages such as full-field, non-contact, high-precision, and high-sensitivity. However, in traditional component defect recognition based on shearing speckle interferometry, it is difficult to detect tiny defects, and the extraction of numerical parameters is hindered by inherent factors such as the experience judgment, subjectivity, and fatigue of technicians, resulting in too long detection time and reduced accuracy of results. Summary of the Invention

[0004] In view of the above drawbacks of the traditional component defect method based on shearing speckle interferometry, the present invention provides a component defect detection method and system based on digital speckle spatio-temporal features and IU-Net.

[0005] On the one hand, the present invention provides a component defect detection method based on digital speckle spatio-temporal features and IU-Net, including:

[0006] Step 1: Build a digital shearing speckle interferometry device, and then collect the shearing speckle interferogram of component defects;

[0007] Step 2: Extract the phase fringe pattern from the shearing speckle interferogram, and filter the noise of the phase fringe pattern by using the sine-cosine filtering method;

[0008] Step 3: Take the shearing speckle interferogram of component defects as the time-domain feature of component defects, and take the two-dimensional defect map and three-dimensional wrapped phase map after noise filtering as the spatial-domain features of component defects;

[0009] Step 4: Build a defect detection model based on IU-Net, specifically including: embedding a convolutional attention module in the original feature extraction stage of the U-Net neural network, introducing a residual learning module, and then performing multi-scale feature fusion;

[0010] Step 5: Input the component defect spatial domain features into the trained defect detection model based on IU-Net to obtain an enhanced component defect map, and use the component defect map to achieve qualitative detection of component defects;

[0011] Step 6: Input the component defect time domain features and the component defect spatial domain features into the trained defect detection model based on IU-Net for testing, identify different types of defects, and achieve quantitative detection of component defects.

[0012] Furthermore, the digital shear speckle interferometry device adopts a Michelson-type digital shear speckle interferometry system based on a 4f optical system.

[0013] Furthermore, in Step 2, the extraction of the phase fringe pattern from the shear speckle interferogram specifically includes: using a spiral scanning unwrapping method based on path tracking to unwrap the shear speckle interferogram to obtain a phase fringe pattern.

[0014] Furthermore, it also includes: extracting phase information using the spatial carrier method, specifically including:

[0015] First, set the carrier frequency introduced in the x direction, then the intensity I(x, y) of the shear speckle interferogram is expressed as:

[0016]

[0017] where f represents the linear spatial carrier frequency introduced in the x direction;

[0018] Then, expand formula (1) to obtain formula (2):

[0019]

[0020] Finally, perform a Fourier transform on formula (2) to obtain formula (3):

[0021] I(f x , f y ) = A(f x , f y ) + C(f x + f, f y ) + C(f x - f, f y ) (3)

[0022] where

[0023] Furthermore, in Step 2, the filtering of noise from the phase fringe pattern using the sine-cosine filtering method specifically includes:

[0024] Perform sine operation and cosine operation on the phase fringe pattern Δ(x, y) respectively to obtain two new phase fringe patterns s(x, y) and c(x, y):

[0025] s(x, y) = sin[Δ(x, y)]

[0026] c(x, y) = cos[Δ(x, y)]

[0027] Perform mean filtering with a filtering window of m×n on s(x, y) and c(x, y) to obtain the images s′(x, y) and c′(x, y) after mean filtering:

[0028]

[0029]

[0030] Divide the sine and cosine phase diagrams obtained by mean filtering and then perform arctangent transformation to obtain the phase diagram Δ′(x, y) after noise filtering:

[0031]

[0032] Furthermore, in step 4, the structure of the defect detection model based on IU-Net specifically includes: an encoder and a decoder; the encoder includes 9 network layers, the first 8 layers are cross-connected by convolutional layers and max-pooling layers in turn, and the last layer is a convolutional layer; the decoder includes 4 convolutional layers;

[0033] A convolutional attention module is embedded between the convolutional layers at the corresponding positions of the encoder and the decoder; residual connections are made between any two adjacent convolutional layers in the entire network.

[0034] On the other hand, the present invention provides a component defect detection system based on digital speckle spatio-temporal features and IU-Net, including:

[0035] A data acquisition module for collecting shear speckle interference patterns of component defects based on the established digital shear speckle interferometry device;

[0036] An image preprocessing module for extracting phase fringe patterns from the shear speckle interference patterns and filtering the noise of the phase fringe patterns by using the sine-cosine filtering method;

[0037] A model training module for training the constructed defect detection model based on IU-Net;

[0038] The model testing module is used to input the extracted component defect spatial domain features into the trained defect detection model based on IU-Net to obtain an enhanced component defect map, and use the component defect map to achieve qualitative detection of component defects; and input the component defect time domain features and component defect spatial domain features into the trained defect detection model based on IU-Net for testing, identify different types of defects, and achieve quantitative detection of component defects; wherein, the shear speckle interferogram of the component defect is used as the component defect time domain feature, and the two-dimensional defect map and three-dimensional wrapped phase map after noise filtering are used as the component defect spatial domain features.

[0039] Advantages of the present invention:

[0040] The component defect detection method based on digital speckle spatio-temporal features and IU-Net provided by the embodiment of the present invention has the ability to solve the problems of small proportion of the defect area in the entire working area and low defect recognition accuracy compared with the traditional component defect detection method, and the designed defect detection system has low performance requirements for computer equipment. Description of the drawings

[0041] Figure 1 It is a schematic flowchart of the component defect detection method based on digital speckle spatio-temporal features and IU-Net provided by the embodiment of the present invention;

[0042] Figure 2 It is a Michelson-type digital shear speckle interferometer system based on a 4f optical system provided by the embodiment of the present invention;

[0043] Figure 3 It is a schematic structural diagram of the component defect detection model based on IU-Net provided by the embodiment of the present invention;

[0044] Figure 4 It is a schematic diagram of the feature map processing process of the convolutional attention module provided by the embodiment of the present invention;

[0045] Figure 5 It is a schematic structural diagram of the residual learning module provided by the embodiment of the present invention. Detailed implementation manners

[0046] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0047] Such as Figure 1As shown in the figure, an embodiment of the present invention provides a component defect detection method based on digital speckle spatio-temporal characteristics and IU-Net, including the following steps:

[0048] S101: Set up a digital shear speckle interferometry device, and then collect the shear speckle interferogram of the component defect;

[0049] As an implementable manner, the embodiment of the present invention uses a Michelson-type digital shear speckle interferometry system based on a 4f optical system as the digital shear speckle interferometry device, and its structure is as Figure 2 shown. Among them, the focal length of the imaging lens is 8 mm, the digital camera uses STC-CL152A of Sentech Company, the shear device of the Michelson structure is composed of a beam splitter prism with a side length of 25 mm and two plane mirrors with a diameter of 25 mm, and a piezoelectric ceramic tube is attached behind one of the plane mirrors as a phase shifter; the 4f optical system is composed of two plano-convex lenses with a diameter and focal length of 40 mm.

[0050] S102: Extract the phase fringe pattern from the shear speckle interferogram, and use the sine-cosine filtering method to filter the noise of the phase fringe pattern;

[0051] As an implementable manner, a spiral scanning unwrapping method based on path tracking is used to unwrap the shear speckle interferogram to obtain a phase fringe pattern; specifically including: starting from the center point of the wrapped phase map, unwrapping along the pixel points in a spiral shape outward until a phase fringe pattern with good filtering effect is obtained. This method has a fast unwrapping speed and good effect, and can handle most forms of phase fringe patterns.

[0052] Furthermore, the spatial carrier method is used to extract the phase information, specifically including: First, set the introduction of the carrier frequency in the x direction, then the intensity I(x, y) of the shear speckle interferogram is expressed as:

[0053]

[0054] where f represents the introduction of a linear spatial carrier frequency in the x direction;

[0055] Then, expand formula (1) to obtain formula (2):

[0056]

[0057] Finally, perform a Fourier transform on formula (2) to obtain formula (3):

[0058] I(f x , f y ) = A(f x , f y ) + C(f x + f, fy ) + C(f x -f, f y ) (3)

[0059] Wherein,

[0060] Specifically, by introducing carrier frequency acquisition in space to introduce phase shift, the object light spectrum containing the deformation information of the object to be measured is separated from other background light spectra, and then the useful spectral region is extracted for Fourier transform to obtain the complex amplitude of the object light.

[0061] As an implementable manner, the method of using the sine-cosine filtering method to filter noise from the phase fringe pattern specifically includes:

[0062] Performing sine operation and cosine operation on the phase fringe pattern Δ(x, y) respectively to obtain two new phase fringe patterns s(x, y) and c(x, y):

[0063] s(x, y) = sin[Δ(x, y)]

[0064] c(x, y) = cos[Δ(x, y)]

[0065] Performing mean filtering with a filtering window of m×n (it should be noted that when the noise is relatively large, a relatively large filtering window needs to be selected) on s(x, y) and c(x, y) to obtain the images s′(x, y) and c′(x, y) after mean filtering:

[0066]

[0067]

[0068] Dividing the sine-cosine phase diagrams obtained by mean filtering and then performing arctangent transformation can obtain the phase diagram Δ′(x, y) after noise filtering:

[0069]

[0070] S103: Using the shear speckle interferogram of the component defect as the time-domain feature of the component defect, and using the two-dimensional defect map and three-dimensional wrapped phase map after noise filtering as the spatial-domain features of the component defect;

[0071] S104: Constructing a defect detection model based on IU-Net, specifically including: embedding a convolutional attention module in the feature extraction stage of the original U-Net neural network, introducing a residual learning module, and then performing multi-scale feature fusion;

[0072] Specifically, the defect detection model designed in the embodiments of the present invention, based on the improved U-Net (referred to as IU-Net), embeds a convolutional attention module in the feature extraction stage of the original U-Net neural network, introduces a residual learning module, and then performs multi-scale feature fusion.

[0073] As an implementable manner, as Figure 3 shown, the structure of the defect detection model based on IU-Net is a U-shaped symmetric network. The left half encoder corresponds to the image downsampling process, which is used to extract image feature information and reduce the spatial dimension; the right half decoder corresponds to the image upsampling process, which is used to gradually restore the detail information of the image. The encoder includes 9 network layers. The first 8 layers are sequentially cross-connected by convolutional layers and max-pooling layers, and the last layer is a convolutional layer; the decoder includes 4 convolutional layers;

[0074] A convolutional attention module is embedded between the convolutional layers at the corresponding positions of the encoder and the decoder. It can be simply and effectively integrated into the convolutional neural network model, improving the network's feature extraction ability without significantly increasing the network's parameter quantity and computational complexity; residual connections are made between any two adjacent convolutional layers in the entire network, which helps the deep network to extract complex features while preventing gradient disappearance and improving the overall performance.

[0075] Among them, the Convolutional Block Attention Model (CBAM) is jointly composed of two sub-modules: a channel attention module and a spatial attention module. As Figure 4 shown, represents the module for pixel-level multiplication; F C represents the channel-wise feature map, A C represents the channel attention module; F represents the input feature map; F R represents the output feature map, A S represents the spatial attention module. The processing process of the CBAM module for the input feature map is not the improvement point of the present invention and will not be elaborated here. The residual learning module is jointly composed of a residual path and an identity connection path, and the structure is as Figure 5 shown. Among them, the residual path includes two groups of convolutional layers and a Relu activation function. The output result of the residual learning module is jointly obtained by adding the output results of the residual path and the identity connection path.

[0076] Furthermore, for the different sizes of defect regions, to obtain multi-scale images of the shear speckle interferogram, first perform three consecutive average pooling operations on the shear speckle interferogram, and then combine the multi-scale shear speckle interferogram obtained through the three average pooling operations with the original shear speckle interferogram Figure 1Start as the input of the network, and inject shear speckle interference images of different sizes into four convolutional modules respectively at the encoder end of the network, and use them as the inputs of the four convolutional modules to increase the multi-scale features of the network model.

[0077] Among them, the purpose of the average pooling operation is to downsample the image. The specific process is as follows: take the average value within the sampling window area of the shear speckle interference image, so that the resolution of the output image is reduced and the size of the image is reduced. In this way, each average pooling operation will halve the size of the image.

[0078] S105: Input the component defect spatial domain features into the trained defect detection model based on IU-Net to obtain an enhanced component defect map, and use the component defect map to realize the qualitative detection of component defects;

[0079] S106: Input the component defect time domain features and the component defect spatial domain features into the trained defect detection model based on IU-Net to identify different types of defects and realize the quantitative detection of component defects.

[0080] Embodiment 2

[0081] Corresponding to the above method embodiments, the embodiment of the present invention also provides a component defect detection system based on digital speckle spatio-temporal features and IU-Net, including: a data acquisition module, an image preprocessing module, a model training module, and a model testing module.

[0082] The data acquisition module is used to collect shear speckle interference images of component defects based on the built digital shear speckle interferometry device. The image preprocessing module is used to extract the phase fringe map from the shear speckle interference image and filter the noise of the phase fringe map by using the sine-cosine filtering method. The model training module is used to train the built defect detection model based on IU-Net. The model testing module is used to input the extracted component defect spatial domain features into the trained defect detection model based on IU-Net to obtain an enhanced component defect map, and use the component defect map to realize the qualitative detection of component defects; and input the component defect time domain features and the component defect spatial domain features into the trained defect detection model based on IU-Net for testing, identify different types of defects, and realize the quantitative detection of component defects; among them, the shear speckle interference image of the component defect is used as the component defect time domain feature, and the two-dimensional defect map and the three-dimensional wrapped phase map after noise filtering are used as the component defect spatial domain features.

[0083] Specifically, the model training module sets the hyperparameters (including parameters such as learning rate, batch size, number of training times, parameter optimizer, etc.) required for training the defect detection model based on IU-Net, then uses the training data set to train the network model, and finally saves the trained network model.

[0084] It should be noted that the component defect detection system provided by the embodiments of the present invention is to implement the above method embodiments. For its functions, please refer to the above method embodiments specifically, and details are not described herein again.

[0085] The present invention can identify the characteristics and analyze the status of different defect types, provide a pre - plan for component maintenance, and ensure the safe operation of components.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A component defect detection method based on digital speckle spatio-temporal characteristics and IU-Net, characterized in that Including: Step 1: Set up a digital shear speckle interferometry device, and then collect the shear speckle interferogram of the component defect; Step 2: Extract the phase fringe pattern from the shear speckle interferogram, and use the sine-cosine filtering method to filter the noise of the phase fringe pattern; Step 3: Take the shear speckle interferogram of the component defect as the time-domain feature of the component defect, and take the two-dimensional defect map and three-dimensional wrapped phase map after noise filtering as the spatial-domain features of the component defect; Step 4: Construct a defect detection model based on IU-Net, specifically including: embedding a convolutional attention module in the feature extraction stage of the original U-Net neural network, introducing a residual learning module, and then performing multi-scale feature fusion; Step 5: Input the spatial-domain features of the component defect into the trained defect detection model based on IU-Net to obtain an enhanced component defect map, and use the component defect map to realize the qualitative detection of the component defect; Step 6: Input the time-domain features and spatial-domain features of the component defect into the trained defect detection model based on IU-Net for testing, identify different types of defects, and realize the quantitative detection of the component defect.

2. The component defect detection method based on digital speckle spatio-temporal features and IU-Net according to claim 1, wherein The digital shear speckle interferometry device adopts a Michelson-type digital shear speckle interferometry system based on a 4f optical system.

3. The component defect detection method based on digital speckle spatio-temporal features and IU-Net according to claim 1, wherein In Step 2, the extraction of the phase fringe pattern from the shear speckle interferogram specifically includes: using a spiral scanning unwrapping method based on path tracking to unwrap the shear speckle interferogram to obtain a phase fringe pattern.

4. The component defect detection method based on digital speckle spatio-temporal characteristics and IU-Net according to claim 3, wherein Also including: Using the spatial carrier method to extract phase information, specifically including: First, set the carrier frequency introduced in the x direction, then the intensity I(x, y) of the shear speckle interferogram is expressed as: where f represents the linear spatial carrier frequency introduced in the x direction; Then, expand formula (1) to obtain formula (2): Finally, perform a Fourier transform on formula (2) to obtain formula (3): I(f x ,f y ) = A(f x ,f y ) + C(f x + f, f y ) + C(f x - f, f y ) (3) Among them, 5. The component defect detection method based on digital speckle spatio-temporal features and IU-Net according to claim 1, characterized in that In Step 2, the use of the sine-cosine filtering method to filter the noise of the phase fringe pattern specifically includes: Performing sine operation and cosine operation on the phase fringe pattern Δ(x, y) respectively to obtain two new phase fringe patterns s(x, y) and c(x, y): s(x, y) = sin[Δ(x, y)] c(x, y) = cos[Δ(x, y)] Performing mean filtering with a filtering window of m×n on s(x, y) and c(x, y) to obtain the images s′(x, y) and c′(x, y) after mean filtering; Dividing the sine-cosine phase maps obtained by mean filtering and then performing an arctangent transform can obtain the phase map Δ′(x, y) after noise filtering; 6. The component defect detection method based on digital speckle spatio-temporal features and IU-Net according to claim 1, wherein In Step 4, the structure of the defect detection model based on IU-Net specifically includes: an encoder and a decoder; the encoder includes 9 network layers, the first 8 layers are cross-connected by convolutional layers and max-pooling layers in turn, and the last layer is a convolutional layer; the decoder includes 4 convolutional layers; A convolutional attention module is embedded between the convolutional layers at the corresponding positions of the encoder and the decoder; residual connections are made between any two adjacent convolutional layers in the whole network.

7. A component defect detection system based on digital speckle spatio-temporal features and IU-Net, characterized in that Including: A data acquisition module, configured to acquire a shear speckle interferogram of a component defect based on the established digital shear speckle interferometry device; An image preprocessing module, configured to extract a phase fringe pattern from the shear speckle interferogram and filter out noise from the phase fringe pattern by using a sine-cosine filtering method; A model training module, configured to train the constructed defect detection model based on IU-Net; A model testing module, configured to input the extracted spatial domain features of the component defect into the trained defect detection model based on IU-Net to obtain an enhanced component defect map, and use the component defect map to achieve qualitative detection of the component defect; And input the temporal domain features and spatial domain features of the component defect into the trained defect detection model based on IU-Net for testing, identify different types of defects, and achieve quantitative detection of the component defect; wherein, the shear speckle interferogram of the component defect is used as the temporal domain feature of the component defect, and the two-dimensional defect map and three-dimensional wrapped phase map after noise filtering are used as the spatial domain features of the component defect.

Images