A defect detection method, device and equipment based on infrared image

Abstract

The present application provides a defect detection method, device and equipment based on infrared images, which relates to the field of defect detection technology, including: obtaining an infrared temperature time series of an object to be tested; filtering the infrared temperature time series data to obtain denoised infrared temperature time series data; using principal component analysis to perform dimensionality reduction feature extraction on the denoised infrared temperature time series data to obtain an initial PCA infrared image; removing the background of the initial PCA infrared image to obtain a background-removed infrared image; inputting the background-removed infrared image into a pre-trained defect detection model based on a generative adversarial neural network to obtain a defect detection result of the object to be tested; the present application can realize non-destructive detection of surface defects / near-surface defects of different objects, especially non-destructive detection of surface defects / near-surface defects of honeycomb sandwich composite materials, and effectively improves the detection efficiency and detection accuracy.

Description

Technical Field

[0001] The present application relates to the field of defect detection technology, and in particular, to a defect detection method, device and equipment based on infrared images. Background Art

[0002] The preparation process of honeycomb sandwich composite materials is relatively complex, and the service environment is relatively harsh. It is easy to produce defects such as debonding and honeycomb core wrinkles, which may cause the composite structure to fail. In order to minimize the risk of structural failure, advanced non-destructive testing technology must be used to detect potential defects in a timely manner. Infrared thermal imaging technology is widely used in non-destructive testing of composite materials due to its high efficiency, rapidity, non-contact and easy detection.

[0003] However, due to the uneven thermal conductivity of honeycomb sandwich composites and their structural characteristics (such as multi-layer interlayers), the acquisition quality of infrared thermal imaging is affected, such as low resolution, poor uniformity, and weak contrast. Therefore, in the field of infrared defect recognition based on image processing, the traditional threshold segmentation method often cannot accurately segment defects because the segmentation result is highly dependent on the contrast of the image. In images with low contrast, that is, when the grayscale difference between the target and the background is not obvious, the selection of the threshold becomes difficult and easily leads to segmentation errors. In addition, the infrared image pixels of honeycomb sandwich composites are unevenly distributed, and the edges of defects are mixed with the background, which makes it easy to miss edges or introduce non-target information when using conventional clustering methods for defect extraction. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a defect detection method, device, and equipment based on infrared images, so as to solve the above-mentioned problems existing in the prior art, realize non-destructive detection of surface defects / near-surface defects of different objects, and improve the efficiency and accuracy of detection.

[0005] In a first aspect, a defect detection method based on infrared images is provided, which may include:

[0006] Obtain infrared temperature data of the object to be measured at each moment within a preset time period to obtain an infrared temperature time series;

[0007] Performing filtering processing on the infrared temperature time series data to obtain infrared temperature time series data after noise reduction;

[0008] The principal component analysis method is used to extract dimension reduction features from the infrared temperature time series data after noise reduction to obtain an initial PCA infrared image;

[0009] Removing the background of the initial PCA infrared image to obtain a background-removed infrared image;

[0010] The background-removed infrared image is input into a pre-trained defect detection model based on a generative adversarial neural network to obtain a defect detection result of the object to be tested; wherein the defect detection model based on a generative adversarial neural network is obtained by training a defect detection model based on a generative adversarial neural network based on a pre-constructed defect data set.

[0011] In an optional implementation, the infrared temperature time series is composed of a temperature matrix and corresponding time;

[0012] Obtain infrared temperature data of the object to be measured at each moment within a preset time period, including:

[0013] Using an infrared detection device to collect infrared images of the object to be detected at each moment within a preset time period;

[0014] For an infrared image at any moment, a temperature matrix at the moment is constructed based on the temperature values ​​of each pixel point of the infrared image;

[0015] Based on the temperature matrix constructed at each moment, the infrared temperature data of the object to be measured at each moment in a preset time period is obtained;

[0016] The infrared temperature time series data is filtered to obtain the infrared temperature time series data after noise reduction, including:

[0017] The infrared temperature time series data is subjected to bilateral filtering to obtain the infrared temperature time series data after noise reduction.

[0018] In an optional implementation, the temperature matrix includes at least one row of elements;

[0019] The principal component analysis method is used to extract dimension reduction features from the infrared temperature time series data after noise reduction to obtain an initial PCA infrared image, including:

[0020] For any temperature matrix, if the temperature matrix contains multiple rows of elements, then the row elements other than the first row elements are merged into the first row elements in order from small to large row numbers to obtain a new temperature matrix;

[0021] Sorting the obtained new temperature matrices according to the time sequence corresponding to the new temperature matrices to obtain a plurality of sorted new temperature matrices;

[0022] Based on the sorted multiple new temperature matrices, a two-dimensional temperature matrix is ​​constructed;

[0023] Decentralizing the two-dimensional temperature matrix to obtain a decentralized two-dimensional temperature matrix;

[0024] The principal component analysis method is used to perform dimension reduction feature extraction on the two-dimensional temperature matrix after decentering to obtain an initial PCA infrared image.

[0025] In an optional implementation, the two-dimensional temperature matrix is ​​decentralized to obtain a decentralized two-dimensional temperature matrix, including:

[0026] Calculating the average value of the elements in each column of the two-dimensional temperature matrix to obtain the average value of each column of the two-dimensional temperature matrix;

[0027] For each element of the two-dimensional temperature matrix, subtract the average value of the column where the element is located from the value of the element to obtain a decentralized value of the element;

[0028] The value of each element of the two-dimensional temperature matrix is ​​replaced by the decentralized value of the corresponding element to obtain a decentralized two-dimensional temperature matrix.

[0029] In an optional implementation, principal component analysis is used to perform dimension reduction feature extraction on the decentralized two-dimensional temperature matrix to obtain an initial PCA infrared image, including:

[0030] Decomposing the decentralized two-dimensional temperature matrix according to a plurality of preset eigenvalues ​​to obtain a plurality of eigenvectors corresponding to the plurality of eigenvalues;

[0031] For any eigenvector, multiply the eigenvector by the decentralized two-dimensional temperature matrix to obtain a eigenimage corresponding to the eigenvector;

[0032] The obtained multiple feature images are fused to obtain the initial PCA infrared image.

[0033] In an optional implementation, the method further includes:

[0034] Acquire a standard infrared image; wherein the standard infrared image is obtained by extracting features from a standard infrared temperature time series after noise reduction using a principal component analysis method; the standard infrared temperature time series after noise reduction is obtained by noise reduction of a standard infrared temperature time series; the standard infrared temperature time series is obtained by collecting a standard object using the infrared detection device; the standard object is a defect-free sample having the same material and geometric dimensions as the object to be tested;

[0035] Removing the background of the initial PCA infrared image to obtain a background-removed infrared image includes:

[0036] The standard infrared image is subtracted from the initial PCA infrared image to obtain an infrared image after background removal.

[0037] In an optional implementation, the defect detection model based on generative adversarial neural network;

[0038] A generator, used for enhancing the infrared image after background removal to obtain a defect image of the object to be tested;

[0039] A discriminator, comprising: 3 convolution blocks, 2 residual blocks and 2 fully connected layers; used to generate a defect detection result of the object to be tested based on the defect image of the object to be tested;

[0040] The generator includes: an encoder and a decoder;

[0041] The encoder comprises a global feature attention module and a multi-scale hole convolution module; the encoder is used to extract target features in the infrared image after background removal;

[0042] The decoder comprises a multi-scale hole convolution module; the decoder is used to restore the target features to obtain a defect image of the object to be tested;

[0043] Any multi-scale dilated convolution module consists of a dilated convolution layer and a mapping layer.

[0044] In a second aspect, a defect detection device based on infrared images is provided, which may include:

[0045] The acquisition unit is used to obtain infrared temperature data of the object to be measured at each moment in a preset time period to obtain an infrared temperature time series;

[0046] A processing unit, used for filtering the infrared temperature time series data to obtain infrared temperature time series data after noise reduction;

[0047] An extraction unit, used for performing dimension reduction feature extraction on the infrared temperature time series data after noise reduction by using principal component analysis to obtain an initial PCA infrared image;

[0048] A removal unit, used for removing the background of the initial PCA infrared image to obtain an infrared image after background removal;

[0049] A detection unit is used to input the background-removed infrared image into a pre-trained defect detection model based on a generative adversarial neural network to obtain a defect detection result of the object to be tested; wherein the defect detection model based on a generative adversarial neural network is obtained by training a defect detection model based on a generative adversarial neural network based on a pre-constructed defect data set.

[0050] In a third aspect, an electronic device is provided, the electronic device comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0051] Memory, used to store computer programs;

[0052] The processor is used to implement any method step described in the first aspect when executing the program stored in the memory.

[0053] In a fourth aspect, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, any method step described in the first aspect is implemented.

[0054] This application realizes the accurate detection of surface defects and near-surface defects of objects made of honeycomb sandwich composite materials or other materials, and improves the efficiency and accuracy of detection. This application sets an array heat source in the infrared detection device to make the detected workpiece heated more evenly and reduce the infrared acquisition error; uses a generative adversarial neural network to build a defect detection model, automatically learns complex infrared image features, and accurately identifies defects in infrared images; at the same time, a generator based on a global feature attention module is used to significantly improve the resolution and clarity of infrared images, thereby improving the accuracy of defect recognition.

[0055] The present application can effectively improve the computing efficiency by converting a three-dimensional matrix into a two-dimensional matrix and then performing feature extraction; by selecting multiple principal components to obtain multiple feature images and then performing feature image fusion, an infrared image containing more feature details and more comprehensive features can be obtained; by removing the background of all infrared images, noise and interference can be effectively removed to obtain high-quality infrared images; defect detection based on high-quality infrared images can effectively improve the precision, accuracy and reliability of defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0057] Figure 1 A flow chart of a defect detection method based on infrared images provided in an embodiment of the present application;

[0058] Figure 2 A schematic diagram of the structure of an infrared acquisition device provided in an embodiment of the present application;

[0059] Figure 3 A schematic diagram of a defect detection model training and testing process based on a generative adversarial neural network provided in an embodiment of the present application;

[0060] Figure 4 A schematic diagram of the structure of a defect detection device based on infrared images provided in an embodiment of the present application;

[0061] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0062] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0063] The defect detection method based on infrared images provided in the embodiment of the present application can be applied in a server or in a terminal with strong computing power. The server can be a physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms. The terminal can be a user equipment (User Equipment, UE) such as a mobile phone, a smart phone, a laptop, a digital broadcast receiver, a personal digital assistant (PDA), a tablet computer (PAD), a handheld device, a vehicle-mounted device, a wearable device, a computing device or other processing equipment connected to a wireless modem, a mobile station (Mobile Station, MS), a mobile terminal (Mobile Terminal), etc. The terminal and the server can be directly or indirectly connected through a wired or wireless communication method, which is not limited in this application.

[0064] The preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application. In addition, the embodiments and features in the embodiments of the present application may be combined with each other if there is no conflict.

[0065] Figure 1The following is a flow chart of a defect detection method based on infrared images provided in an embodiment of the present application. Figure 1 As shown, the method may include:

[0066] Step S110, acquiring infrared temperature data of the object to be measured at each moment within a preset time period to obtain an infrared temperature time series; filtering the infrared temperature time series data to obtain infrared temperature time series data after noise reduction.

[0067] In the embodiment of the present application, the object to be measured may be an object made of a honeycomb sandwich composite material.

[0068] In the embodiment of the present application, the infrared temperature time series is obtained based on the continuous collection of the object to be measured by the infrared detection device within a preset time period; specifically, the infrared temperature time series is a three-dimensional sequence composed of a temperature matrix and corresponding time.

[0069] In an embodiment of the present application, the preset time period is the time period from the start of heating to the end of cooling of the object to be tested; the acquisition frequency can be 3 seconds, that is, the object to be tested is collected once every 3 seconds to obtain an infrared temperature time series.

[0070] In the embodiment of the present application, the infrared temperature time series is obtained by processing the infrared images collected by the infrared detection device at each moment; a temperature matrix corresponds to a frame of thermal image; each element in the temperature matrix is ​​the temperature value of each pixel point of the thermal image.

[0071] In the embodiment of the present application, infrared temperature data of the object to be measured at each moment in a preset time period is obtained to obtain an infrared temperature time series, including:

[0072] An infrared detection device is used to collect infrared images of the object to be tested at each moment within a preset time period (including the heating process and the cooling process); for the infrared image at any moment, a temperature matrix at that moment is constructed based on the temperature value of each pixel point of the infrared image; based on the constructed temperature matrix at each moment, the infrared temperature data of the object to be tested at each moment within the preset time period is obtained.

[0073] In the embodiment of the present application, the infrared detection device is specifically an infrared thermal imaging detection device; the infrared thermal imaging detection device is provided with a phased array pulse laser emission unit and an infrared camera; when working, a phased array pulse laser is used to heat the object to be tested, and the phased array pulse laser is evenly distributed on one side of the detection part, and by emitting high-energy pulse lasers, the surface of the honeycomb sandwich panel to be tested absorbs energy instantly, generates heat wave diffusion, and forms a temperature gradient. Before collecting the temperature time series of the object to be tested, an infrared camera is used to collect the surface temperature of a standard object (a defect-free sample with the same material and the same geometric size as the object to be tested) to determine the surface temperature of the object to be tested, and then targeted power adjustment is performed on each pulse laser emission unit to achieve uniform surface temperature of the object to be tested.

[0074] In practical applications, due to different internal structures, the surface field distribution is uneven, and defect information can be detected through peak temperature, peak time and single moment temperature information, so that defects can also be clearly observed in infrared thermal imaging. Therefore, this application identifies defects in materials by collecting the temperature change rate in different areas. In order to use this difference to identify defects, an infrared thermal image data set is used, and the pixel value sequence of each pixel point is expanded into a temperature-time curve in the time domain.

[0075] In the embodiment of the present application, the infrared detection device adopts the transmission type active infrared thermal imaging nondestructive detection technology; Figure 2 As shown, the infrared detection device includes: an infrared thermal camera, a matrix thermal excitation source, and a signal collector; the working process of the infrared detection device includes: using a heating source to heat the object to be tested, and then using a thermal imager to record the temperature field changes on the surface of the object to be tested, so as to characterize the defects to be tested.

[0076] In the embodiment of the present application, in order to make the object under test heated evenly, an array heat source is used to emit a uniform heat source, thereby improving the acquisition accuracy of the infrared camera.

[0077] In the embodiment of the present application, it is also necessary to obtain a standard infrared image; the standard infrared image is obtained by the following steps: pre-collecting a standard object using an infrared detection device to obtain a standard infrared temperature time series; then denoising the standard infrared temperature time series to obtain a denoised standard infrared temperature time series; then using the principal component analysis method to perform dimensionality reduction feature extraction processing on the denoised standard infrared temperature time series to obtain a standard infrared image; specifically, the standard object is a defect-free sample with the same material and geometric dimensions as the object to be measured.

[0078] In the embodiment of the present application, the infrared temperature time series data is filtered to obtain the infrared temperature time series data after noise reduction, including:

[0079] The infrared temperature time series data is processed by bilateral filtering to obtain the infrared temperature time series data after noise reduction.

[0080] Step S120, using principal component analysis to extract dimension reduction features from the infrared temperature time series data after noise reduction to obtain an initial PCA infrared image; removing the background of the initial PCA infrared image to obtain an infrared image after background removal.

[0081] In the embodiment of the present application, the principal component analysis method is used to extract the dimension reduction features of the infrared temperature time series data after noise reduction to obtain the initial PCA infrared image, including:

[0082] For any temperature matrix, if the temperature matrix contains multiple rows of elements, then the other row elements other than the first row elements are merged into the first row elements in order from small to large according to the row number to obtain a new temperature matrix; the obtained new temperature matrices are sorted according to the time sequence corresponding to each new temperature matrix to obtain multiple sorted new temperature matrices; based on the sorted multiple new temperature matrices, a two-dimensional temperature matrix is ​​constructed; the two-dimensional temperature matrix is ​​decentralized to obtain a decentralized two-dimensional temperature matrix; the principal component analysis method is used to reduce the dimension of the decentralized two-dimensional temperature matrix and extract features to obtain an initial PCA infrared image.

[0083] In the embodiment of the present application, the two-dimensional temperature matrix is ​​decentralized to obtain a decentralized two-dimensional temperature matrix, including:

[0084] Calculate the average value of the elements in each column of the two-dimensional temperature matrix to obtain the average value of each column of the two-dimensional temperature matrix; for each element of the two-dimensional temperature matrix, subtract the average value of the column where the element is located from the value of the element to obtain the decentralized value of the element; replace the value of each element of the two-dimensional temperature matrix with the decentralized value of the corresponding element to obtain the decentralized two-dimensional temperature matrix.

[0085] In the embodiment of the present application, the principal component analysis method is used to perform dimension reduction feature extraction on the decentralized two-dimensional temperature matrix to obtain an initial PCA infrared image, including:

[0086] According to the preset multiple eigenvalues, the decentralized two-dimensional temperature matrix is ​​decomposed to obtain multiple eigenvectors corresponding to the multiple eigenvalues; for any eigenvector, the eigenvector is multiplied by the decentralized two-dimensional temperature matrix to obtain the eigenimage corresponding to the eigenvector; the obtained multiple eigenimages are fused to obtain the initial PCA infrared image.

[0087] In the embodiment of the present application, the temperature matrix is ​​constructed based on the temperature values ​​of each pixel of the corresponding thermal image, so the temperature matrix contains at least one row of elements, and the size of the temperature matrix is ​​the size of the thermal image represented by pixels; therefore, the three-dimensional infrared temperature time series can be expressed as T×A×B, where T represents the number of thermal image frames, and A and B represent the length and width of each frame of the temperature matrix, respectively. Since the three-dimensional matrix requires a lot of resources for feature extraction and processing, it is time-consuming and the extraction effect is not good. Therefore, in order to improve the calculation efficiency, the present application converts the three-dimensional temperature time series into a two-dimensional temperature matrix for feature extraction.

[0088] For example, suppose there are 2 frames of thermal images, each of which contains 4 pixels in total. The temperature values ​​of each pixel in the first frame are 1, 2, 3, and 4, respectively, and the temperature values ​​of each pixel in the second frame are 5, 6, 7, and 8, respectively. The two temperature matrices are and ; First, merge the data of the second row of each temperature matrix into the first row to obtain the new temperature matrices [1,3,2,4] and [5,7,6,8] of the two frames of thermal images; secondly, sort the two new temperature matrices to obtain a two-dimensional temperature matrix ; Calculate the average values ​​of each column of the two-dimensional temperature matrix to be 3, 5, 4, and 6 respectively; Subtract the average value of the column from each element, that is, , the decentralized two-dimensional temperature matrix is ​​obtained as [ In the two-dimensional temperature matrix after decentering, the element value of each column represents the temperature change of the same position point in the thermal image of different frames. By decentering the temperature matrix, the contrast between defects and no defects can be improved, thereby improving the accuracy of defect detection.

[0089] In an embodiment of the present application, eigenvalue decomposition is performed on the decentralized two-dimensional temperature matrix to decompose the matrix into a combination of multiple characteristic eigenvectors and eigenvalues; based on the size of the eigenvalues ​​of each eigenvector, the weight or importance corresponding to each eigenvector can be determined; the eigenvector is multiplied by the decentralized two-dimensional temperature matrix to obtain the corresponding eigenvector matrix, and the eigenvector matrix is ​​transformed to obtain a eigenimage corresponding to the eigenvector; and based on the weight or importance corresponding to each eigenvector or a preset ratio, multiple eigenimages are fused to obtain an initial PCA infrared image.

[0090] In the embodiment of the present application, there can be multiple principal components, and the characteristic images obtained by principal component analysis based on different principal components are different; a single principal component can only reflect a certain feature and is not comprehensive; therefore, the present application determines multiple different eigenvectors through multiple different eigenvalues, and then obtains multiple different characteristic images, and then fuses them to obtain an initial PCA infrared image containing more features and more comprehensive, which helps to improve the precision and accuracy of detection. The process of selecting the principal component is the process of selecting the eigenvalue, sorting all the eigenvalues ​​obtained by decomposition according to size, selecting the eigenvectors corresponding to the first n eigenvalues, and thus obtaining the corresponding n characteristic images; n is a positive integer.

[0091] In one embodiment of the present application, n may be set to 3.

[0092] In another embodiment of the present application, it is also possible to directly obtain the corresponding temperature trend characteristic image, periodic fluctuation information characteristic image and local or subtle temperature change pattern characteristic image by presetting three characteristic values ​​of specific sizes; and fuse the temperature trend characteristic image, periodic fluctuation information characteristic image and local or subtle temperature change pattern characteristic image to obtain the initial PCA infrared image.

[0093] In an embodiment of the present application, the temperature trend characteristic image, the periodic fluctuation information characteristic image and the local or subtle temperature change pattern characteristic image fusion or other characteristic images can be fused by presetting the weight ratio of each characteristic image (for example, set to 5, 3, and 2 respectively); the size of the characteristic value can be used as the weight ratio for fusion; the size of the characteristic value after certain processing can also be used as the weight ratio for fusion.

[0094] In an embodiment of the present application, removing the background of the initial PCA infrared image to obtain the infrared image after background removal includes: subtracting the standard infrared image from the initial PCA infrared image to obtain the infrared image after background removal.

[0095] Step S130: input the infrared image after background removal into a pre-trained defect detection model based on a generative adversarial neural network to obtain a defect detection result of the object to be tested.

[0096] In an embodiment of the present application, the defect detection model based on the generative adversarial neural network is obtained by training the defect detection model based on the generative adversarial neural network based on a pre-constructed defect data set.

[0097] In an embodiment of the present application, a defect detection model based on a generative adversarial neural network includes:

[0098] A generator, used to enhance the infrared image after background removal using a fully convolutional neural network based on a multi-scale attention module to obtain a defect image of the object to be tested;

[0099] The discriminator is used to generate a defect detection result of the object to be tested based on the defect image of the object to be tested.

[0100] In an embodiment of the present application, the generator includes an encoder and a decoder; the encoder is used to extract target features in the infrared image after background removal, and the decoder is used to restore the encoded target features to match the enhanced image.

[0101] In an embodiment of the present application, the encoder includes a global feature attention module and a multi-scale dilated convolution module; the multi-scale dilated convolution module is composed of a dilated convolution layer and a mapping layer; wherein the global feature attention module better captures the long-distance dependency between pixels by integrating the global information in the feature image, thereby alleviating the problem of discontinuity of defect features in infrared images; the multi-scale dilated convolution module uses dilated convolutions with different expansion rates to learn the feature information of defects of different shapes to improve the accuracy of the model in identifying defect edges; the dilated convolution layer obtains multi-scale local feature information by cascading three different sizes of dilated convolutions, and the mapping layer generates a spatial weight map through 1×1 convolution, batch normalization and Sigmoid function to highlight key areas and local details.

[0102] In an embodiment of the present application, the decoder also includes a multi-scale dilated convolution module; any multi-scale dilated convolution module consists of a dilated convolution layer and a mapping layer; the dilated convolution layer obtains multi-scale local feature information by cascading three dilated convolutions of different sizes, and the mapping layer generates a spatial weight map through 1×1 convolution, batch normalization and Sigmoid function to highlight key areas and local details.

[0103] In one embodiment of the present application, the global attention module is in the last layer of the encoder, and a multi-scale hole convolution module is introduced in the residual connection.

[0104] In an embodiment of the present application, the present application improves the generator by adding a global feature attention module to the encoder and replacing all convolutional layers of the encoder and decoder with multi-scale dilated convolution modules; therefore, the present application does not limit the number of convolutional layers of the encoder and decoder, and the encoder may have only one global feature attention module and one multi-scale dilated convolution module, or may have multiple convolutional layers and multiple fully connected layers.

[0105] In another embodiment of the present application, a residual channel attention module is provided in the generator, and the residual channel attention module integrates the residual dense block (RCB) and the channel attention mechanism (CA) to optimize feature learning; the RCB module is responsible for the learning and retention of deep features, while the CA focuses on the extraction of key features by adjusting the weights of each channel, thereby enhancing the ability to restore infrared image details; the residual channel attention block in each residual block can adaptively learn the dependencies between image features. The channel attention mechanism first performs global average pooling and outputs a compressed channel. After extracting features through two layers of convolution, a sigmoid gating mechanism is used to limit the output to the range of [0, 1]. Finally, the obtained result is multiplied by the original feature, thereby giving different weights to each channel.

[0106] Compared with the traditional components of super-resolution generative adversarial networks, the residual channel attention mechanism provides more powerful feature extraction and image reconstruction capabilities, especially in dealing with texture and detail defects in infrared images. In addition, the residual channel attention module can effectively alleviate the gradient problem caused by the increase in network depth and significantly improve the resolution and clarity of infrared images.

[0107] In an embodiment of the present application, the discriminator is responsible for distinguishing the subtle differences between the generated image and the real image, and promoting the improvement of image quality; specifically, the discriminator adopts a residual U-net network, including: 3 convolution blocks, 2 residual blocks and 2 fully connected layers; each convolution block is composed of a two-dimensional convolution layer with a convolution kernel size of 3 × 3, a maximum pooling layer with a step size of 2 × 2, and a ReLU function; it is used to generate a defect detection result of the object to be tested based on the defect image of the object to be tested.

[0108] In the embodiment of the present application, the initial PCA infrared image is the input of the generative adversarial network generator, and the corresponding real defect image and the enhanced PCA infrared image are the input of the discriminator.

[0109] like Figure 3 As shown, in an embodiment of the present application, a defect detection model based on a generative adversarial neural network is trained using a pre-generated adversarial data set; through training, a nonlinear mapping relationship is achieved between the pixel points of each background-removed infrared defect image and the real defect image; and then the trained defect detection model based on a generative adversarial neural network is tested using an infrared image generated by the newly tested temperature time series data.

[0110] Specifically, the generation of adversarial datasets includes:

[0111] Firstly, a standard sample with artificial prefabricated cracks, holes and other defects is made and experimentally collected to obtain a training set of temperature time series for the generative adversarial neural network model; the collected samples can be 1000 groups;

[0112] Secondly, the principal component analysis method is used to process the temperature time series training set to obtain the processed adversarial temperature series set;

[0113] Finally, the processed adversarial temperature sequence set is expanded on the time axis by pixels to obtain the temperature time series, and the defect image is split into one-dimensional sequences by rows to obtain the adversarial dataset.

[0114] Corresponding to the above method, the embodiment of the present application also provides a defect detection device based on infrared images, such as Figure 4 As shown, the defect detection device based on infrared image includes:

[0115] The acquisition unit 410 is used to acquire infrared temperature data of the object to be measured at each moment in a preset time period to obtain an infrared temperature time series;

[0116] The processing unit 420 is used to filter the infrared temperature time series data to obtain the infrared temperature time series data after noise reduction;

[0117] An extraction unit 430 is used to extract dimension reduction features from the infrared temperature time series data after noise reduction by using a principal component analysis method to obtain an initial PCA infrared image;

[0118] A removal unit 440, used to remove the background of the initial PCA infrared image to obtain an infrared image after background removal;

[0119] The detection unit 450 is used to input the infrared image after background removal into a pre-trained defect detection model based on a generative adversarial neural network to obtain a defect detection result of the object to be tested; wherein the defect detection model based on the generative adversarial neural network is obtained by training the defect detection model based on the generative adversarial neural network based on a pre-constructed defect data set.

[0120] The functions of each functional unit of the infrared image-based defect detection device provided in the above-mentioned embodiments of the present application can be realized through the above-mentioned method steps. Therefore, the specific working process and beneficial effects of each unit in the infrared image-based defect detection device provided in the embodiments of the present application will not be repeated here.

[0121] The present application also provides an electronic device, such as Figure 5 As shown, it includes a processor 510 , a communication interface 520 , a memory 530 and a communication bus 540 , wherein the processor 510 , the communication interface 520 , and the memory 530 communicate with each other via the communication bus 540 .

[0122] Memory 530, for storing computer programs;

[0123] The processor 510 is used to execute the program stored in the memory 530 to implement the following steps:

[0124] Obtain infrared temperature data of the object to be measured at each moment within a preset time period to obtain an infrared temperature time series;

[0125] Perform bilateral filtering on the infrared temperature time series data to obtain the infrared temperature time series data after noise reduction;

[0126] The principal component analysis method is used to extract the dimension reduction features of the infrared temperature time series data after noise reduction to obtain the initial PCA infrared image.

[0127] The background of the initial PCA infrared image is removed to obtain an infrared image after background removal;

[0128] The infrared image after background removal is input into a pre-trained defect detection model based on a generative adversarial neural network to obtain a defect detection result of the object to be tested; wherein, the defect detection model based on the generative adversarial neural network is obtained by training the defect detection model based on the generative adversarial neural network based on a pre-constructed defect data set.

[0129] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0130] The communication interface is used for communication between the above electronic device and other devices.

[0131] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0132] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0133] The implementation methods and beneficial effects of the components of the electronic device in the above embodiments to solve the problems can be seen in Figure 1 The various steps in the illustrated embodiment are implemented, therefore, the specific working process and beneficial effects of the electronic device provided by the embodiment of the present application are not repeated here.

[0134] In another embodiment provided in the present application, a computer-readable storage medium is provided, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer executes any of the infrared image-based defect detection methods in the above embodiments.

[0135] In another embodiment provided in the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any of the infrared image-based defect detection methods in the above embodiments.

[0136] Those skilled in the art will appreciate that the embodiments in the present application can be provided as methods, systems, or computer program products. Therefore, the embodiments in the present application can be in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments in the present application can be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0137] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0138] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0139] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0140] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the embodiments of the present application.

[0141] Obviously, those skilled in the art can make various changes and modifications to the embodiments in the present application without departing from the spirit and scope of the embodiments in the present application. Thus, if these modifications and variations of the embodiments in the present application fall within the scope of the claims and their equivalents in the embodiments of the present application, the embodiments of the present application are also intended to include these modifications and variations.

Claims

1. A defect detection method based on infrared images, characterized in that: The method comprises: Obtain infrared temperature data of the object to be measured at each moment within a preset time period to obtain an infrared temperature time series; the preset time period is the time period of the heating process and the time period of the cooling process involved in the object to be measured from the beginning of heating to the end of cooling; Performing filtering processing on the infrared temperature time series data to obtain infrared temperature time series data after noise reduction; The principal component analysis method is used to extract dimension reduction features from the infrared temperature time series data after noise reduction to obtain an initial PCA infrared image; Removing the background of the initial PCA infrared image to obtain a background-removed infrared image; Inputting the background-removed infrared image into a pre-trained defect detection model based on a generative adversarial neural network to obtain a defect detection result of the object to be tested; wherein the defect detection model based on a generative adversarial neural network is obtained by training a defect detection model based on a generative adversarial neural network based on a pre-constructed defect data set; Wherein, the infrared temperature time series is a three-dimensional sequence consisting of a temperature matrix and corresponding time; Obtain infrared temperature data of the object to be measured at each moment within a preset time period, including: An infrared detection device is used to collect infrared images of the object to be detected at each moment within a preset time period; the infrared detection device includes: an infrared thermal imager, a matrix thermal excitation source and a signal collector; For an infrared image at any moment, a temperature matrix at the moment is constructed based on the temperature values ​​of each pixel point of the infrared image; Based on the temperature matrix constructed at each moment, the infrared temperature data of the object to be measured at each moment in a preset time period is obtained; Wherein, the temperature matrix contains at least one row of elements; The principal component analysis method is used to extract dimension reduction features from the infrared temperature time series data after noise reduction to obtain an initial PCA infrared image, including: For any temperature matrix, if the temperature matrix contains multiple rows of elements, then the row elements other than the first row elements are merged into the first row elements in order from small to large row numbers to obtain a new temperature matrix; Sorting the obtained new temperature matrices according to the time sequence corresponding to the new temperature matrices to obtain a plurality of sorted new temperature matrices; Based on the sorted multiple new temperature matrices, a two-dimensional temperature matrix is ​​constructed; Decentralizing the two-dimensional temperature matrix to obtain a decentralized two-dimensional temperature matrix; The principal component analysis method is used to perform dimension reduction feature extraction on the two-dimensional temperature matrix after the center is removed to obtain an initial PCA infrared image; Furthermore, the defect detection model based on generative adversarial neural network; A generator is used to enhance the infrared image after background removal by using a fully convolutional neural network based on a multi-scale attention module to obtain a defect image of the object to be tested; the generator includes: an encoder and a decoder; the encoder is used to extract target features in the infrared image after background removal, and the decoder is used to restore the encoded target features to match the enhanced image; wherein the encoder includes a global feature attention module and a multi-scale hole convolution module; the multi-scale hole convolution module is composed of a hole convolution layer and a mapping layer; wherein the global feature attention module better captures the long-distance dependency between pixels by integrating the global information in the feature image, thereby alleviating the problem of discontinuity of defect features in the infrared image; the multi-scale hole convolution module utilizes The feature information of defects of different shapes is learned by using hole convolutions with different expansion rates to improve the accuracy of the model in identifying defect edges. The hole convolution layer obtains multi-scale local feature information by cascading three hole convolutions of different sizes. The mapping layer generates a spatial weight map through 1×1 convolution, batch normalization and Sigmoid function to highlight key areas and local details. The decoder also contains a multi-scale hole convolution module. Any multi-scale hole convolution module consists of a hole convolution layer and a mapping layer. The hole convolution layer obtains multi-scale local feature information by cascading three hole convolutions of different sizes. The mapping layer generates a spatial weight map through 1×1 convolution, batch normalization and Sigmoid function to highlight key areas and local details. A discriminator is used to generate a defect detection result of the object to be tested based on the defect image of the object to be tested; the discriminator includes: 3 convolution blocks, 2 residual blocks and 2 fully connected layers; each convolution block consists of a two-dimensional convolution layer with a convolution kernel size of 3×3, a maximum pooling layer with a step size of 2×2, and a ReLU function.

2. The method according to claim 1, characterized in that The infrared temperature time series data is filtered to obtain the infrared temperature time series data after noise reduction, including: The infrared temperature time series data is subjected to bilateral filtering to obtain the infrared temperature time series data after noise reduction.

3. The method according to claim 1, characterized in that The two-dimensional temperature matrix is ​​decentralized to obtain a decentralized two-dimensional temperature matrix, including: Calculating the average value of the elements in each column of the two-dimensional temperature matrix to obtain the average value of each column of the two-dimensional temperature matrix; For each element of the two-dimensional temperature matrix, subtract the average value of the column where the element is located from the value of the element to obtain a decentralized value of the element; The value of each element of the two-dimensional temperature matrix is ​​replaced by the decentralized value of the corresponding element to obtain a decentralized two-dimensional temperature matrix.

4. The method according to claim 1, characterized in that The principal component analysis method is used to perform dimension reduction feature extraction on the two-dimensional temperature matrix after decentering to obtain an initial PCA infrared image, including: Decomposing the decentralized two-dimensional temperature matrix according to a plurality of preset eigenvalues ​​to obtain a plurality of eigenvectors corresponding to the plurality of eigenvalues; For any eigenvector, multiply the eigenvector by the decentralized two-dimensional temperature matrix to obtain a eigenimage corresponding to the eigenvector; The obtained multiple feature images are fused to obtain the initial PCA infrared image.

5. The method according to claim 1, characterized in that The method further comprises: Acquire a standard infrared image; wherein the standard infrared image is obtained by extracting features from a standard infrared temperature time series after noise reduction using a principal component analysis method; the standard infrared temperature time series after noise reduction is obtained by noise reduction of a standard infrared temperature time series; the standard infrared temperature time series is obtained by collecting a standard object using the infrared detection device; the standard object is a defect-free sample having the same material and geometric dimensions as the object to be tested; Removing the background of the initial PCA infrared image to obtain a background-removed infrared image includes: The standard infrared image is subtracted from the initial PCA infrared image to obtain an infrared image after background removal.

6. A defect detection device based on infrared images, characterized in that: The device comprises: The acquisition unit is used to obtain infrared temperature data of the object to be measured at each moment within a preset time period to obtain an infrared temperature time series; the preset time period is the time period of the heating process and the time period of the cooling process involved in the object to be measured from the beginning of heating to the end of cooling; A processing unit, used for filtering the infrared temperature time series data to obtain infrared temperature time series data after noise reduction; An extraction unit, used for performing dimension reduction feature extraction on the infrared temperature time series data after noise reduction by using principal component analysis to obtain an initial PCA infrared image; A removal unit, used for removing the background of the initial PCA infrared image to obtain an infrared image after background removal; A detection unit, used for inputting the background-removed infrared image into a pre-trained defect detection model based on a generative adversarial neural network to obtain a defect detection result of the object to be detected; wherein the defect detection model based on a generative adversarial neural network is obtained by training a defect detection model based on a generative adversarial neural network based on a pre-constructed defect data set; Wherein, the infrared temperature time series is a three-dimensional sequence composed of a temperature matrix and corresponding time; the acquisition unit is specifically used to: use an infrared detection device to collect infrared images of the object to be measured at each moment in a preset time period; the infrared detection device includes: an infrared thermal camera, a matrix thermal excitation source and a signal collector; for the infrared image at any moment, based on the temperature value of each pixel point of the infrared image, a temperature matrix at the moment is constructed; based on the constructed temperature matrix at each moment, the infrared temperature data of the object to be measured at each moment in the preset time period is obtained; Wherein, the temperature matrix contains at least one row of elements; the extraction unit is specifically used for: for any temperature matrix, if the temperature matrix contains multiple rows of elements, then other row elements other than the first row elements are merged into the first row elements in order from small to large row numbers to obtain a new temperature matrix; sorting the obtained new temperature matrices according to the time sequence corresponding to each new temperature matrix to obtain a plurality of sorted new temperature matrices; constructing a two-dimensional temperature matrix based on the sorted plurality of new temperature matrices; de-centering the two-dimensional temperature matrix to obtain a decentralized two-dimensional temperature matrix; using principal component analysis to perform dimensionality reduction feature extraction on the decentralized two-dimensional temperature matrix to obtain an initial PCA infrared image; Furthermore, the defect detection model based on the generative adversarial neural network; the generator is used to enhance the infrared image after background removal by using a fully convolutional neural network based on a multi-scale attention module to obtain a defect image of the object to be tested; the generator includes: an encoder and a decoder; the encoder is used to extract the target features in the infrared image after background removal, and the decoder is used to restore the encoded target features to match the enhanced image; wherein the encoder includes a global feature attention module and a multi-scale hole convolution module; the multi-scale hole convolution module is composed of a hole convolution layer and a mapping layer; wherein the global feature attention module better captures the long-distance dependency between pixels by integrating the global information in the feature image, thereby alleviating the problem of discontinuity of defect features in the infrared image The multi-scale hole convolution module uses hole convolutions with different expansion rates to learn the feature information of defects of different shapes to improve the accuracy of the model in identifying defect edges. The hole convolution layer obtains multi-scale local feature information by cascading three hole convolutions of different sizes. The mapping layer generates a spatial weight map through 1×1 convolution, batch normalization and Sigmoid function to highlight key areas and local details. The decoder also contains a multi-scale hole convolution module. Any multi-scale hole convolution module consists of a hole convolution layer and a mapping layer. The hole convolution layer obtains multi-scale local feature information by cascading three hole convolutions of different sizes. The mapping layer generates a spatial weight map through 1×1 convolution, batch normalization and Sigmoid function to highlight key areas and local details. A discriminator is used to generate a defect detection result of the object to be tested based on the defect image of the object to be tested; the discriminator includes: 3 convolution blocks, 2 residual blocks and 2 fully connected layers; each convolution block consists of a two-dimensional convolution layer with a convolution kernel size of 3×3, a maximum pooling layer with a step size of 2×2, and a ReLU function.

7. An electronic device, characterized in that: The electronic device comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-5 when executing a program stored in a memory.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images