A method for detecting surface defects of building materials using infrared polarization imaging

Abstract

The present invention provides a method for detecting surface defects of building materials by infrared polarization imaging, which belongs to the field of image processing. A building material surface defect detection model is constructed through input, image preprocessing, a feature extraction module, a feature fusion module, a building material polarization enhanced image module, a building material surface defect detection module and an output, and polarization image, infrared image and depth image information are fused to provide a more comprehensive feature representation. The model has a stronger recognition ability for subtle defects, and a self-attention mechanism further optimizes the focusing effect of the features, avoids the interference of background noise, and ensures that the defective area is fully identified. The present invention has better adaptability and performance in the field of defect detection, and can cope with more complex and changeable building material detection tasks, especially showing stronger advantages in detecting tiny defects and complex surfaces.

Description

Technical Field

[0001] The invention belongs to the field of image processing, and in particular relates to a method for detecting surface defects of building materials by infrared polarization imaging. Background Art

[0002] With the rapid development of the construction industry, the quality inspection of building materials has become an important means to ensure the safety and durability of building structures. Traditional methods for detecting surface defects of building materials mainly rely on manual visual inspection, ultrasonic detection, X-ray imaging and other technologies. Although these methods can detect surface defects such as cracks and corrosion to a certain extent, they have some inherent disadvantages, such as low efficiency of manual inspection, radiation risk of X-rays, and high requirements on material thickness for ultrasonic detection. Therefore, how to improve the efficiency, accuracy and safety of surface defect detection of building materials has always been a hot topic of research in industry and academia.

[0003] In recent years, with the development of imaging technology, infrared imaging technology has gradually become an effective means widely used in material defect detection due to its non-destructive and real-time characteristics. By capturing the infrared radiation on the surface of the object, it can provide a temperature distribution map of the material surface, thereby revealing abnormalities on the surface and inside of the object; polarization imaging technology, as an emerging imaging method, has received widespread attention in recent years. Polarization imaging can effectively provide additional information on surface details by capturing the polarization characteristics of light reflected from the surface of the object, especially on surfaces with high reflected light intensity or less texture. Polarization imaging can reveal defect characteristics that ordinary infrared imaging cannot capture. By analyzing data such as polarization angle and polarization degree, polarization imaging technology can reveal the geometric characteristics of the object surface, such as the location, shape and depth of defects such as cracks, corrosion and peeling.

[0004] However, existing polarization imaging technologies are mostly used in the visible light band and are often used separately from traditional infrared imaging technologies. Even in the field of infrared polarization imaging, traditional infrared polarization imaging methods mainly rely on the separate processing of polarization information or depth information, and cannot fully utilize the advantages of both for joint analysis. As a result, in the detection of surface defects in building materials, there are many challenges such as insufficient information utilization, insufficient detection capabilities in complex scenes, and lack of effective multimodal fusion strategies. Summary of the invention

[0005] The present invention provides a method for detecting surface defects of building materials by infrared polarization imaging, aiming to integrate polarization features, infrared features and depth features of building material images to enhance the surface detail information of the building material polarization images and improve the model's ability to identify building material defects, and comprises the following steps.

[0006] S1. Production of a dataset of building material surface defects. Use a polarization industrial camera with a depth sensor to shoot building materials with surface defects at different polarization angles to obtain polarization images, infrared images, and depth images of building materials, and annotate them. After annotation, a dataset of building material surface defects is obtained.

[0007] S2. Image preprocessing: introducing adaptive standard deviation and improving Gaussian filtering to preprocess the polarization image of building materials and generate polarization preprocessed images of building materials.

[0008] S3, feature extraction module, uses different convolution layers to extract features from the building material polarization preprocessed image, the building material infrared image and the building material depth image to obtain the building material polarization features, the building material infrared features and the building material depth features.

[0009] S4, feature fusion module, adds the polarization features and infrared features of building materials, and then splices them with the depth features of building materials. Use 1×1 convolution to reduce the dimension of the spliced ​​features to obtain the fusion features of building materials.

[0010] S5, building material polarization enhanced image module, uses the self-attention mechanism to process the building material fusion features and the building material polarization features to obtain the first-order building material fusion features, and the channel honeycomb operation generates the second-order building material fusion features. The first-order building material fusion features and the second-order building material fusion features are added together and then the number of channels is reduced through 1×1 convolution to obtain the building material polarization enhanced image.

[0011] S6, building material surface defect detection module, extracts the spatial features of the polarization-enhanced image of building materials through three 3×3 convolutional layers, reduces the size of the feature map through the pooling layer, flattens the pooled feature map into a vector, uses a fully connected layer for linear transformation, and uses the Softmax activation function to classify the types of building material defects.

[0012] S7. Construct a building material surface defect detection model, including input, image preprocessing, feature extraction module, feature fusion module, building material polarization enhanced image module, building material surface defect detection module and output.

[0013] S8. Training and testing of building material surface defect detection model. Use building material surface defect dataset to train the model. After the training is completed, obtain the surface image of the building material to be tested and input it into the building material surface defect detection model to obtain the classification result.

[0014] Preferably, in S1, a polarization industrial camera with a depth sensor is used to photograph building materials with surface defects at different polarization angles to obtain a polarization image I of the building material. P ∈R H×W×4 、Infrared image of building materials I R ∈R H×W×1 and building material depth image I D ∈R H×W×1 , where H, W and 4 are the height, width and number of channels of the polarization image of building materials, H, W and 1 are the height, width and number of channels of the infrared image of building materials, and H, W and 4 are the height, width and number of channels of the depth image of building materials. They are annotated and the annotated defects are divided into five categories, including cracks, corrosion, peeling, holes and blistering. After the annotation is completed, the surface defect dataset of building materials is obtained.

[0015] Preferably, in S2, for the building material polarization image I P Preprocessing is performed, and adaptive standard deviation is introduced and applied to Gaussian filtering. The size and standard deviation of the filter kernel are adaptively adjusted according to the characteristics of the local area. For different channels in the polarization image of building materials, the standard deviation σ is adaptively adjusted according to the gradient information of each channel. adaptive , for each channel c, calculate the adaptive standard deviation:

[0016]

[0017] Where x and y represent the height and width of the polarization image of the building material on channel c, respectively, and α is an adjustment parameter that controls the upper limit of the standard deviation. is the gradient of the polarization image of building materials at position (x, y, c). The image is convolved with a Gaussian filter kernel that introduces an adaptive standard deviation to generate a polarization preprocessed image of building materials.

[0018]

[0019] Among them I P (x, y, c) is the pixel value of the polarized image of the building material at position (x, y, c), G(xi, yj, σ adaptive (x, y, c)) is a Gaussian kernel, resulting in a polarization preprocessed image of building materials Where H, W and 4 are the height, width and number of channels of the polarization preprocessed image of building materials.

[0020] Preferably, in S2, an adaptive standard deviation is introduced to dynamically adjust the standard deviation according to the gradient information of the image, which can ensure that a smaller standard deviation is used in areas with larger gradients to avoid blurred edges. Using a larger standard deviation in smooth areas can effectively remove noise, making the denoising process more intelligent and precise, while retaining key texture and edge information during denoising, which is particularly suitable for scenarios that need to retain details, such as surface defect detection of building materials.

[0021] Preferably, in S3, the process of extracting features from the polarization preprocessed image of building materials, the infrared image of building materials and the depth image of building materials is respectively:

[0022] S31. Use 3×3 convolution to extract features from the polarization preprocessed image of building materials to obtain polarization features of building materials. Among them, MaxPool represents the maximum pooling, ReLU is the ReLU activation function, and Conv 3 Represents 3×3 convolution, the polarization characteristics of building materials obtained in and C represent the height, width, and number of channels of the polarization characteristics of the building materials, respectively;

[0023] S32, use 5×5 convolution to extract features from the infrared image of building materials, and obtain the infrared features F of building materials R =MaxPool(ReLU(Conv 5 (I R ))), where MaxPool represents the maximum pooling, ReLU is the ReLU activation function, Conv 5 Represents 5×5 convolution, and the infrared features of building materials obtained in and C represent the height, width, and number of channels of infrared features of building materials, respectively;

[0024] S33, use 5×5 convolution to extract features from the building material depth image to obtain the building material depth feature F D =MaxPool(ReLU(Conv 5 (I D ))), where MaxPool represents the maximum pooling, ReLU is the ReLU activation function, Conv 5 Represents 5×5 convolution, the deep features of building materials obtained in and C represent the height, width, and number of channels of the depth feature of the building material, respectively.

[0025] Preferably, in S3, the polarization preprocessed image of building materials shows tiny geometric changes on the surface. Using 3x3 convolution to extract features from the polarization preprocessed image of building materials can more accurately capture details in local areas, avoid over-smoothing or information loss in areas with rich details, and maintain high resolution of features. The infrared image of building materials and the depth image of building materials contain a wider range of scene information, involving larger-scale surface changes. The infrared image of building materials shows large-scale thermal changes on the surface through temperature distribution, while the depth image of building materials reflects the overall structure of the surface. Using 5x5 convolution to capture a wide range of features is helpful to detect larger-scale defects such as cracks and corrosion. At the same time, it can cover a larger receptive field, helping the network to understand the global structure and temperature distribution changes, and then model the overall morphology of surface defects of building materials.

[0026] Preferably, in S4, the polarization feature of the building material and the infrared feature of the building material are added, and then spliced ​​with the depth feature of the building material, and the dimensionality of the spliced ​​feature is reduced using a 1×1 convolution to obtain the building material fusion feature F fused =Conv 1 (Concat((F P +F R ), F D )), where Conv 1 represents 1×1 convolution, Concat represents concatenation operation, and the generated building material fusion features in and C represent the height, width, and number of channels of the building material fusion feature, respectively.

[0027] Preferably, in S4, the information provided by the polarization characteristics of building materials and the infrared characteristics of building materials are highly complementary. The polarization characteristics of building materials help to reveal the geometric and reflective properties of the surface of building materials, while the infrared characteristics of building materials provide information on the surface temperature and thermal changes of building materials. The fusion of the two can provide richer surface information of building materials, which helps to better capture defects such as cracks and corrosion on the surfaces of complex building materials; the polarization characteristics of building materials and the infrared characteristics of building materials focus on providing the surface characteristics of building materials, and the depth characteristics of building materials focus on providing the geometric structure information of building materials. Splicing the added polarization characteristics of building materials and infrared characteristics of building materials into the depth characteristics of building materials can ensure that the characteristics of each mode are effectively retained, enhance the representation ability of the model, and avoid unnecessary feature redundancy.

[0028] Preferably, in S5, the building material polarization enhanced image module step includes:

[0029] S51, polarization characteristics of building materials F PFusion characteristics with building materials F fused Add them together, and then perform three different linear transformations through the fully connected layer to convert them into query vectors, key vectors, and value vectors. The specific process is as follows:

[0030] [q 1 , k 1 , v 1 ]=(F P +F fused )×[W q , W k , W v ],

[0031] where q 1 , k 1 , v 1 represents the query vector, key vector and value vector of the added building material polarization features and building material fusion features, respectively, W q , W k , W v They are used to generate q 1 , k 1 , v 1 The learning weight matrix of

[0032] S52, first-order building material fusion feature generation, query vector q 1 and key vector k 1 Concatenate and input to the fully connected layer for linear transformation. Use the tanh activation function to limit the output between -1 and 1, increase nonlinearity, and then add the value vector v 1 Multiply each channel by channel, and finally obtain the first-order building material fusion features through a fully connected layer. The specific process is as follows:

[0033] F′ fused =FC(tanh(FC(Concat(q 1 , k 1 )))⊙v 1 ),

[0034] Where FC represents the fully connected layer, Concat represents the concatenation operation, tanh represents the tanh activation function, and ⊙ represents the channel-by-channel multiplication operation;

[0035] S53, generate second-order building material fusion features through channel honeycomb operation, first add the building material polarization features and the building material fusion features, then perform linear transformation through a fully connected layer, and then extract the weight list through the global average pool, multiply the two sets of weights in the weight list with the building material polarization features and the building material fusion features respectively, and then add them channel by channel, and then multiply them with the value vectors of the added building material polarization features and building material fusion features channel by channel, and then pass through a fully connected layer to obtain the second-order building material fusion features. The specific process is:

[0036]

[0037] Where FC represents the fully connected layer, H and W represent the height and width of the feature map after linear transformation through the fully connected layer, and s p ,s fused are the weights corresponding to the polarization features of building materials and the fusion features of building materials in the weight list, and ⊙ represents the channel-by-channel multiplication operation;

[0038] S54, after adding the first-order building material fusion features and the second-order building material fusion features channel by channel, further processed by a 1×1 convolution to reduce the number of channels and map the feature map into a building material polarization enhanced image, I P_enhanced =Conv 1 (F′ fused +F″ fused ), where Conv 1 Represents the polarization image of building materials after 1×1 convolution and enhancement I P_enhanced ∈R H ×W×4 , where H, W, and 4 represent the height, width, and number of channels of the polarization-enhanced image of building materials, respectively.

[0039] Preferably, in S5, the polarization features of building materials and the fusion features of building materials are spliced, and then the spliced ​​feature map is converted into queries, keys and values. The importance of different regions is dynamically adjusted through the subsequent attention mechanism, so that the network can strengthen its attention to the defect location according to the importance of the region; the tanh activation function is used to perform nonlinear transformation on the feature map, which increases the separability of the data and the model's learning ability for nonlinear features. The model can better handle the complex features after the fusion of polarization images with infrared images and depth images, and further enhance the significance of defective areas of building materials; through the self-attention mechanism, the model is helped to reduce background interference and focus on defective areas. The convolution operation is used to finally enhance the features, making the defect information more prominent, especially in the detection of tiny defects such as cracks and corrosion on the material surface. The enhanced polarization image can provide clearer surface details, and the generated building material polarization-enhanced image can be used as the input of the subsequent defect detection model to ensure the high accuracy and robustness of the detection results.

[0040] Preferably, in S6, the building material surface defect detection module process includes:

[0041] S61, extract the spatial features of the polarization enhanced image of building materials through three 3×3 convolutional layers. The specific process is: F 0 =I P_enhanced , F l+1 =ReLU(BN(Conv 3 (F l ))), l = 0, 1, 2,

[0042] Where ReLU is the ReLU activation function, BN stands for batch normalization, Conv 3 represents 3×3 convolution, F l represents the input of the l+1th convolutional layer, F l+1 Represents the output of the l+1th convolutional layer, and the value range of l is 0, 1, 2;

[0043] S62, reduce the size of the feature map through the pooling layer while retaining the most important spatial information, then flatten the pooled feature map into a vector, and use the fully connected layer for linear transformation. The specific process is:

[0044] F 4 =FC(Flatten(MaxPool(F 3 ))),

[0045] Among them, MaxPool represents the maximum pooling operation, Flatten represents the flattening operation, and FC is the fully connected layer;

[0046] S63. Use the Softmax activation function to classify the types of building material defects and output the probability of each defect category. in represents the predicted probability of the type of building material defects, and softmax represents the softmax activation function;

[0047] S64. Use the cross entropy loss function to measure the error between the category prediction and the true label. The cross entropy loss function formula is: where y i Indicates the actual building material defect type label, represents the probability of the building material defect category predicted by the model, and log represents the log function.

[0048] Preferably, in S6, convolution, ReLU and batch normalization can help extract features of the surface of building materials and enhance sensitivity to defective areas of building materials; pooling and flattening operations reduce the size of feature maps while retaining the most significant features for subsequent processing; this method can efficiently extract and fuse multimodal information, thereby improving the accuracy of building material surface defect detection.

[0049] Preferably, in S7, for the building material surface defect detection model, the building material polarization image, the building material infrared image, and the building material depth image are input into the model, and the building material polarization image is first preprocessed, and then the image enters the feature extraction module to extract features, and the feature fusion module to complete feature fusion, and then the image enters the building material polarization enhancement image module to generate the building material polarization enhancement image, and finally enters the building material surface defect detection module for processing to obtain the detection output result.

[0050] Compared with the prior art, the present invention has the following technical effects: the polarization image, infrared image and depth image information are integrated to provide a more comprehensive feature representation, the model has a stronger recognition ability for subtle defects, the self-attention mechanism further optimizes the focusing effect of the features, and enables the model to dynamically adjust the attention level of each area, avoiding the interference of background noise, and ensuring that the defective area is fully identified. The present invention has better adaptability and performance in the field of defect detection, and can cope with more complex and changeable building material detection tasks, especially showing stronger advantages when detecting tiny defects and complex surfaces. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a flow chart of a method for detecting surface defects of building materials using infrared polarization imaging provided by the present invention.

[0052] Figure 2 This is a diagram of the image processing process of building materials using infrared polarization imaging provided by the present invention.

[0053] Figure 3 It is a structural diagram of the polarization-enhanced image module of building materials provided by the present invention.

[0054] Figure 4 It is a structural diagram of a building material surface defect detection module provided by the present invention.

[0055] Figure 5 A diagram showing the effect of detecting surface defects of building materials in an embodiment provided by the present invention. DETAILED DESCRIPTION

[0056] The following will be combined with the accompanying drawings of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] See also Figures 1 to 5 The present invention provides a method for detecting surface defects of building materials by infrared polarization imaging, which aims to integrate polarization features, infrared features and depth features of building material images to enhance the surface detail information of the building material polarization image and improve the model's ability to identify building material defects, and includes the following steps.

[0058] S1. Use a polarization industrial camera with a depth sensor to take photos of building materials with surface defects at four polarization angles: 0°, 45°, 90°, and 135°. A total of 3,000 images are obtained to obtain polarization images of building materials I. P ∈R 640×640×4 、Infrared image of building materials I R ∈R 640×640×1 and building material depth image I D ∈R 640×640×1 , where 640, 640 and 4 are the height, width and number of channels of the polarization image of building materials, 640, 640 and 1 are the height, width and number of channels of the infrared image of building materials, and 640, 640 and 4 are the height, width and number of channels of the depth image of building materials. They are annotated and the annotated defects are divided into five categories, including cracks, corrosion, peeling, holes and blistering. After the annotation is completed, the surface defect dataset of building materials is obtained, and the training set and test set are divided into 7:3.

[0059] S2. Image preprocessing: introducing adaptive standard deviation and improving Gaussian filtering to preprocess the polarization image of building materials and generate polarization preprocessed images of building materials.

[0060] Furthermore, in S2, for the building material polarization image I PPreprocessing is performed, and adaptive standard deviation is introduced and applied to Gaussian filtering. The size and standard deviation of the filter kernel are adaptively adjusted according to the characteristics of the local area. For different channels in the polarization image of building materials, the standard deviation σ is adaptively adjusted according to the gradient information of each channel. adaptive , for each channel c, calculate the adaptive standard deviation:

[0061]

[0062] Where x and y represent the height and width of the polarized image of building materials on channel c, respectively. α is an adjustment parameter that controls the upper limit of the standard deviation. Its value range is (0.5, 2). A smaller value is used to maintain finer image features, and a larger value is used for smoothing and denoising. In this embodiment, the value is 0.7. is the gradient of the polarization image of building materials at position (x, y, c). The image is convolved with a Gaussian filter kernel that introduces an adaptive standard deviation to generate a polarization preprocessed image of building materials.

[0063] Among them I P (x, y, c) is the pixel value of the polarized image of the building material at position (x, y, c), G(xi, yj, σ adaptive (x, y, c)) is a Gaussian kernel, resulting in a polarization preprocessed image of building materials Where 640, 640 and 4 are the height, width and number of channels of the polarization preprocessed image of building materials.

[0064] S3, feature extraction module, uses different convolution layers to extract features from the building material polarization preprocessed image, the building material infrared image and the building material depth image to obtain the building material polarization features, the building material infrared features and the building material depth features.

[0065] Furthermore, in S3, the process of extracting features from the polarization preprocessed image of building materials, the infrared image of building materials, and the depth image of building materials is as follows:

[0066] S31. Use 3×3 convolution to extract features from the polarization preprocessed image of building materials to obtain polarization features of building materials. Among them, MaxPool represents the maximum pooling, ReLU is the ReLU activation function, and Conv 3 Represents 3×3 convolution, and the polarization characteristics of building materials obtained are F P ∈R 320×320×64 , where 320, 320, and 64 represent the height, width, and number of channels of the polarization characteristics of the building material, respectively;

[0067] S32, use 5×5 convolution to extract features from the infrared image of building materials, and obtain the infrared features F of building materials R =MaxPool(ReLU(Conv 5 (I R ))), where MaxPool represents the maximum pooling, ReLU is the ReLU activation function, Conv 5 Represents 5×5 convolution, and the infrared feature F of the building material obtained R ∈R 320×320×64 , where 320, 320 and 64 represent the height, width and number of channels of infrared characteristics of building materials, respectively;

[0068] S33, use 5×5 convolution to extract features from the building material depth image to obtain the building material depth feature F D =MaxPool(ReLU(Conv 5 (I D ))), where MaxPool represents the maximum pooling, ReLU is the ReLU activation function, Conv 5 Represents a 5×5 convolution, and the obtained building material depth feature F D ∈R 320×320×64 , where 320, 320, and 64 represent the height, width, and number of channels of the building material depth feature, respectively.

[0069] S4, feature fusion module, adds the polarization features and infrared features of building materials, and then splices them with the depth features of building materials. Use 1×1 convolution to reduce the dimension of the spliced ​​features to obtain the fusion features of building materials.

[0070] Furthermore, in S4, the polarization features of building materials and the infrared features of building materials are added, and then spliced ​​with the depth features of building materials. The 1×1 convolution is used to reduce the dimension of the spliced ​​features to obtain the building material fusion feature F fused =Conv 1 (Concat((F P +F R ), F D )), where Conv 1 represents 1×1 convolution, Concat represents concatenation operation, and the generated building material fusion feature F fused ∈R 320×320×64 , where 320, 320, and 64 represent the height, width, and number of channels of the building material fusion feature, respectively.

[0071] S5, building material polarization enhanced image module, uses the self-attention mechanism to process the building material fusion features and the building material polarization features to obtain the first-order building material fusion features, and the channel honeycomb operation generates the second-order building material fusion features. The first-order building material fusion features and the second-order building material fusion features are added together and then the number of channels is reduced through 1×1 convolution to obtain the building material polarization enhanced image.

[0072] Further, in S5, the building material polarization enhancement image module step includes:

[0073] S51, polarization characteristics of building materials F P Fusion characteristics with building materials F fused Add them together, and then perform three different linear transformations through the fully connected layer to convert them into query vectors, key vectors, and value vectors. The specific process is as follows:

[0074] [q 1 , k 1 , v 1 ]=(F P +F fused )×[W q , W k , W v ],

[0075] where q 1 , k 1 , v 1 represents the query vector, key vector and value vector of the added building material polarization features and building material fusion features, respectively, W q , W k , W v They are used to generate q 1 , k 1 , v 1 The learning weight matrix, W q , W k , W v It is learned adaptively during the training process and initialized using Xavier initialization.

[0076] Where N() represents Gaussian distribution, n in and n out Respectively represent the dimensions of input and output, and are both set to 64 in this embodiment;

[0077] S52, first-order building material fusion feature generation, query vector q 1 and key vector k 1 Concatenate and input to the fully connected layer for linear transformation. Use the tanh activation function to limit the output between -1 and 1, increase nonlinearity, and then add the value vector v 1Multiply each channel by channel, and finally obtain the first-order building material fusion features through a fully connected layer. The specific process is as follows:

[0078] F′ fused =FC(tanh(FC(Concat(q 1 , k 1 )))⊙v 1 ),

[0079] Where FC represents the fully connected layer, Concat represents the concatenation operation, tanh represents the tanh activation function, and ⊙ represents the channel-by-channel multiplication operation;

[0080] S53, generate second-order building material fusion features through channel honeycomb operation, first add the building material polarization features and the building material fusion features, then perform linear transformation through a fully connected layer, and then extract the weight list through the global average pool, multiply the two sets of weights in the weight list with the building material polarization features and the building material fusion features respectively, and then add them channel by channel, and then multiply them with the value vectors of the added building material polarization features and building material fusion features channel by channel, and then pass through a fully connected layer to obtain the second-order building material fusion features. The specific process is:

[0081]

[0082] Where FC represents the fully connected layer, H and W represent the height and width of the feature map after linear transformation through the fully connected layer, and s p ,s fused are the weights corresponding to the polarization features of building materials and the fusion features of building materials in the weight list, and ⊙ represents the channel-by-channel multiplication operation;

[0083] S54, after adding the first-order building material fusion features and the second-order building material fusion features channel by channel, further processed by a 1×1 convolution to reduce the number of channels and map the feature map into a building material polarization enhanced image, I P_enhanced =Conv 1 (F′ fused +F″ fused ), where Conv 1 Represents the polarization image of building materials after 1×1 convolution and enhancement I P_enhanced ∈R 640×640×4 , where 640, 640, and 4 represent the height, width, and number of channels of the polarization-enhanced image of building materials, respectively.

[0084] S6, building material surface defect detection module, three 3×3 convolutional layers extract the spatial features of the polarization enhanced image of building materials, reduce the size of the feature map through the pooling layer, and then flatten the pooled feature map into a vector, use the fully connected layer for linear transformation, and use the Softmax activation function to classify the types of building material defects.

[0085] Further, in S6, the building material surface defect detection module process includes:

[0086] S61, extract the spatial features of the polarization enhanced image of building materials through three 3×3 convolutional layers. The specific process is: F 0 =I P_enhanced , F l+1 =ReLU(BN(Conv 3 (F l ))), l = 0, 1, 2,

[0087] Where ReLU is the ReLU activation function, BN stands for batch normalization, Conv 3 represents 3×3 convolution, F l represents the input of the l+1th convolutional layer, F l+1 Represents the output of the l+1th convolutional layer, and the value range of l is 0, 1, 2;

[0088] S62, reduce the size of the feature map through the pooling layer while retaining the most important spatial information, then flatten the pooled feature map into a vector, and use the fully connected layer for linear transformation. The specific process is:

[0089] F 4 =FC(Flatten(MaxPool(F 3 ))),

[0090] Among them, MaxPool represents the maximum pooling operation, Flatten represents the flattening operation, and FC is the fully connected layer;

[0091] S63. Use the Softmax activation function to classify the types of building material defects and output the probability of each defect category. in represents the predicted probability of the type of building material defects, and softmax represents the softmax activation function;

[0092] S64. Use the cross entropy loss function to measure the error between the category prediction and the true label. The cross entropy loss function formula is: where y i Indicates the actual building material defect type label, represents the probability of the building material defect category predicted by the model, and log represents the log function.

[0093] S7. Construct a building material surface defect detection model, including input, image preprocessing, feature extraction module, feature fusion module, building material polarization enhanced image module, building material surface defect detection module and output.

[0094] Furthermore, in S7, for the building material surface defect detection model, the building material polarization image, the building material infrared image, and the building material depth image are input into the model, and the building material polarization image is first preprocessed, and then enters the feature extraction module to extract features, and the feature fusion module to complete feature fusion, and then enters the building material polarization enhancement image module to generate the building material polarization enhancement image, and finally enters the building material surface defect detection module for processing to obtain the detection output result.

[0095] S8. Training and testing of building material surface defect detection model. Use building material surface defect dataset to train the model. After the training is completed, obtain the surface image of the building material to be tested and input it into the building material surface defect detection model to obtain the classification result.

[0096] Furthermore, the building material surface defect detection model was coded using the Pycharm application and Python language, using the Pytorch framework. The model input polarization images of building materials with a resolution of 640×640×4, infrared images of building materials with a resolution of 640×640×1, and depth images of building materials with a resolution of 640×640×1 were trained. The model was trained for 300 epochs, of which 50 epochs were frozen.

[0097] The above are only preferred embodiments of the present invention. It should be pointed out that a person skilled in the art can make several modifications and improvements without departing from the inventive concept of the present invention, and these all fall within the protection scope of the present invention.

Claims

1. A method for detecting surface defects of building materials by infrared polarization imaging, characterized in that: The following steps are involved: S1, collect images of surface defects of building materials and construct a data set; S2, introduce adaptive standard deviation and improve Gaussian filtering to pre-process polarization images of building materials; S3, feature extraction module, uses convolutional layer to extract features; S4, feature fusion module, adds the polarization features of building materials and the infrared features of building materials, and then splices them with the depth features of building materials. After splicing, the convolution layer is used for dimensionality reduction; S5, building material polarization enhanced image module, uses self-attention mechanism to generate first-order building material fusion features, channel honeycomb operation to generate second-order building material fusion features, adds the first-order building material fusion features and the second-order building material fusion features, and then reduces the number of channels; S6, building material surface defect detection module, extracts the spatial features of the building material polarization enhanced image through the convolution layer, reduces the size and flattens it through the pooling layer, and obtains the type of building material defects after linear transformation using the fully connected layer; S7, constructing a building material surface defect detection model, including input, image preprocessing, feature extraction module, feature fusion module, building material polarization enhanced image module, building material surface defect detection module and output; S8. Training and testing of building material surface defect detection models.

2. The method for detecting surface defects of building materials by infrared polarization imaging according to claim 1, further characterized in that: In S2, for the building material polarization image I P Preprocessing is performed, and adaptive standard deviation is introduced and applied to Gaussian filtering. The size and standard deviation of the filter kernel are adaptively adjusted according to the characteristics of the local area. For different channels in the polarized image of building materials, according to the gradient information of each channel, for each channel c, the adaptive standard deviation calculation formula is: Where x and y represent the height and width of the polarization image of the building material on channel c, respectively, and α is an adjustment parameter that controls the upper limit of the standard deviation. is the gradient of the polarization image of the building material at the position (x, y, c). The image is convolved using a Gaussian filter kernel with an adaptive standard deviation. The generation process of the polarization preprocessed image of the building material is: Among them I P (x, y, c) is the pixel value of the polarized image of the building material at the position (x, y, c), G(xi, yj, σa daptive (x, y, c)) is a Gaussian kernel, resulting in a polarization preprocessed image of building materials Where H, W and 4 are the height, width and number of channels of the polarization preprocessed image of building materials.

3. The method for detecting surface defects of building materials by infrared polarization imaging according to claim 2 is further characterized in that: In S3, the process of extracting features from the polarization preprocessed image of building materials, the infrared image of building materials and the depth image of building materials are respectively: S31. Use 3×3 convolution to extract features from the polarization preprocessed image of building materials to obtain polarization features of building materials. Where MaxPool represents the maximum pooling, ReLU is the ReLU activation function, and Conv3 represents 3×3 convolution. The polarization characteristics of building materials are obtained. in and C represent the height, width, and number of channels of the polarization characteristics of the building materials, respectively; S32, use 5×5 convolution to extract features from the infrared image of building materials, and obtain the infrared features F of building materials R =MaxPool(ReLU(Conv5(I R ))), where I R Represents the infrared image of building materials, MaxPool represents the maximum pooling, ReLU is the ReLU activation function, Conv5 represents 5×5 convolution, and the infrared features of building materials are obtained in and C represent the height, width, and number of channels of infrared features of building materials, respectively; S33, use 5×5 convolution to extract features from the building material depth image to obtain the building material depth feature F D =MaxPool(ReLU(Conv5(I D ))), where I D Represents the depth image of building materials, MaxPool represents the maximum pooling, ReLU is the ReLU activation function, Conv5 represents 5×5 convolution, and the obtained building material depth features in and C represent the height, width, and number of channels of the depth feature of the building material, respectively.

4. The method for detecting surface defects of building materials by infrared polarization imaging according to claim 3 is further characterized in that: In S4, the polarization feature of the building material and the infrared feature of the building material are added, and then spliced ​​with the depth feature of the building material, and the dimensionality of the spliced ​​features is reduced using 1×1 convolution to obtain the fusion feature F of the building material. fused =Conv1(Concat((F P +F R ), F D )), where Conv1 represents 1×1 convolution, Concat represents concatenation operation, and the generated building material fusion features in and C represent the height, width, and number of channels of the building material fusion feature, respectively.

5. The method for detecting surface defects of building materials by infrared polarization imaging according to claim 4 is further characterized in that: In said S5, the step of said building material polarization enhancement image module comprises: S51, polarization characteristics of building materials F P Fusion characteristics with building materials F fused Add them together, and then perform three different linear transformations through the fully connected layer to convert them into query vectors, key vectors, and value vectors. The specific process is as follows: [q1,k1,v1]=(F P +F fused )×[W q ,W k ,W v ], Where q1, k1, and v1 represent the query vector, key vector, and value vector of the added building material polarization features and building material fusion features, respectively. q , W k , W v Respectively represent the learning weight matrices used to generate q1, k1, and v1; S52, first-order building material fusion feature generation, concatenate the query vector q1 and the key vector k1, input them into the fully connected layer for linear transformation, use the tanh activation function to limit the output between -1 and 1, and then multiply it with the value vector v1 channel by channel, and finally pass it through a fully connected layer to obtain the first-order building material fusion feature. The specific process is as follows: F′ fused =FC(tanh(FC(Concat(q1,k1)))⊙v1), Among them, FC represents the fully connected layer, Concat represents the concatenation operation, tanh represents the tanh activation function, and ⊙ represents the channel-by-channel multiplication operation; S53, generate second-order building material fusion features through channel honeycomb operation, first add the building material polarization features and the building material fusion features, then perform linear transformation through a fully connected layer, and then extract the weight list through the global average pool, multiply the two sets of weights in the weight list with the building material polarization features and the building material fusion features respectively, and then add them channel by channel, and then multiply them with the value vectors of the added building material polarization features and building material fusion features channel by channel, and then pass through a fully connected layer to obtain the second-order building material fusion features. The specific process is: Where FC represents the fully connected layer, H and W represent the height and width of the feature map after linear transformation through the fully connected layer, and s p ,s fused are the weights corresponding to the polarization features of building materials and the fusion features of building materials in the weight list, and ⊙ represents the channel-by-channel multiplication operation; S54, after adding the first-order building material fusion features and the second-order building material fusion features channel by channel, further processed by a 1×1 convolution to reduce the number of channels and map the feature map into a building material polarization enhanced image, I P_enhanced =Conv1(F′ fused +F″ fused ), where Conv1 represents a 1×1 convolution, and the enhanced polarization image of building materials I P_enhanced ∈R H×W×4 , where H, W, and 4 represent the height, width, and number of channels of the polarization-enhanced image of building materials, respectively.

6. The method for detecting surface defects of building materials by infrared polarization imaging according to claim 5, further characterized in that: In S6, the building material surface defect detection module comprises: S61. Extract the spatial features of the polarization-enhanced image of building materials through three 3×3 convolutional layers. The specific process is as follows: F0=I P_enhanced ,F l+1 =ReLU(BN(Conv3(F l ))),l=0,1,2, Where ReLU is the ReLU activation function, BN represents batch normalization, Conv3 represents 3×3 convolution, and F l represents the input of the l+1th convolutional layer, F l+1 Represents the output of the l+1th convolutional layer, and the value range of l is 0, 1, 2; S62, reduce the size of the feature map through the pooling layer while retaining the most important spatial information, then flatten the pooled feature map into a vector, and use the fully connected layer for linear transformation. The specific process is: F4=FC(Flatten(MaxPool(F3))), Among them, MaxPool represents the maximum pooling operation, Flatten represents the flattening operation, and FC is the fully connected layer; S63. Use the Softmax activation function to classify the types of building material defects and output the probability of each defect category. in represents the predicted probability of the type of building material defects, and softmax represents the softmax activation function; S64. Use the cross entropy loss function to measure the error between the category prediction and the true label. The cross entropy loss function formula is: where y i Indicates the actual building material defect type label, represents the probability of the building material defect category predicted by the model, and log represents the log function.

7. The method for detecting surface defects of building materials by infrared polarization imaging according to claim 6 is further characterized in that: In S7, for the building material surface defect detection model, the building material polarization image, the building material infrared image, and the building material depth image are input into the model, and the building material polarization image is first preprocessed, and then the image enters the feature extraction module to extract features, and the feature fusion module to complete feature fusion, and then the image enters the building material polarization enhancement image module to generate the building material polarization enhancement image, and finally enters the building material surface defect detection module for processing to obtain the detection output result.

Images