A polarization-based significant defect detection method

Abstract

This invention provides a polarization-based saliency defect detection method, comprising the following steps: (1) constructing a polarization image dataset based on polarization images; (2) building a polarization-based saliency defect detection network structure; and (3) inputting data for network training. This invention designs a novel network structure based on an optical model to extract the physical features of polarization images, utilizing the differences in polarization characteristics of different defects for saliency detection, thus providing a new solution for defect detection in industrial scenarios.

Description

Technical Field

[0001] This invention relates to the field of polarization imaging technology, and more specifically to a method for detecting significant defects based on polarization. Background Technology

[0002] Polarization imaging, by analyzing the evolution of the polarization state distribution of light waves, can effectively distinguish the differences in polarization characteristics between reflected and scattered light from a target, thereby enhancing features such as target edges and texture information. Deep learning, as a data-driven feature mining tool, exhibits unique advantages in complex feature association modeling and nonlinear mapping learning. Combining the multidimensional optical information of polarization imaging with the adaptive feature extraction capabilities of deep learning provides a new path for salient target detection in complex defect detection environments. Summary of the Invention

[0003] To overcome the shortcomings of the existing technology, this invention provides a polarization-based saliency defect detection method, which constructs a polarization cross-modal enhancement network. The encoder extracts polarization features from the polarization image, performs feature concatenation, and then uses a cross-modal attention fusion strategy to finally obtain a predicted map for saliency defect detection.

[0004] To achieve the above objectives, the present invention adopts the following technical solution: a polarization-based method for detecting significant defects, comprising the following steps:

[0005] Step 1: Construct a saliency defect detection dataset for polarized images and preprocess the dataset. Group the images according to different defects in the images, then label the images in the dataset, marking the location and category of the target objects in the images. Divide the training, validation and test sets according to an 8:1:1 ratio to construct the original data suitable for training the saliency detection model.

[0006] Step 2: Combining the physical model of the bidirectional polarization distribution function and Fresnel formula, the distribution law of defect polarization information is derived, and the mapping relationship between defects and polarization distribution is obtained; based on the construction of a convolutional neural network, a network layer for extracting polarization information is constructed according to the expression.

[0007] Step 3: The polarization component images (0°, 45°, 90° linear polarization) and intensity component S0 are used as inputs to the model. The dual-branch network extracts polarization information features separately, and then the features are fused through the polarization defect feature fusion module.

[0008] Step 4: Train the constructed network using the significant defect detection dataset described in Step 1 to obtain a weight file for polarization saliency detection. Use this trained weight file to perform defect saliency detection on glass in an industrial environment. Use Mean Absolute Error (MAE), Fmeasure (F), and Smeasure (S) as evaluation metrics to measure the model performance.

[0009] Furthermore, the construction of the saliency detection dataset mentioned in step one involves placing glass with different defects into the environment as the target object, capturing images using a DoFp camera under active light conditions, obtaining images in four polarization directions, extracting polarization information from them, obtaining multidimensional polarization information in the images, removing data interfered with by noise, and expanding the dataset based on data filtering through data cropping, flipping, moving, mirroring, and other data augmentation techniques to improve the generalization ability of the model.

[0010] Furthermore, in step three, the polarization fusion module (PCEM) is designed to fuse the feature representations of the three polarization branches and fuse them with s0 as input to the next-level network.

[0011] Furthermore, RGB and polarization images are input into the network. RGB captures color features, and polarization extracts high-dimensional polarization complementary features. Feature extraction is performed on each branch through a multi-branch encoder (MN) network. The MN feature extraction layer structure includes a saliency detection network using a dual-branch encoder structure, and a ResNet-50 backbone network. Each convolutional module contains a 3x3 convolutional layer and a batch normalization layer, and uses the ReLU activation function to enhance nonlinear feature mining capabilities. Multi-branch features are then fused through a fusion network to achieve comprehensive representation. Subsequently, cross-modal feature fusion is performed, and finally, a saliency prediction map is generated through decoding.

[0012] Compared with the prior art, the present invention has the following beneficial effects:

[0013] In defect detection scenarios, polarization-based saliency defect detection algorithms can effectively focus on changes in polarization information in target defects. Guided by multimodal information, the model exhibits better robustness and generalization compared to traditional models. Attached Figure Description

[0014] Figure 1 This is a flowchart illustrating the overall implementation of the present invention;

[0015] Figure 2 This is a diagram of the experimental apparatus for the present invention;

[0016] Figure 3 A schematic diagram of the established polarization-based saliency defect detection network;

[0017] Figure 4 This is a diagram of the polarization fusion feature network.

[0018] Figure 5 This is a diagram showing the detection effect of the present invention;

[0019] Figure 6 This is a table showing the comparison results of the detection effects of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] This invention discloses a polarization-based method for detecting significant defects, such as... Figure 1 As shown, it includes the following steps:

[0022] Step 1: Construct a salient defect detection dataset for polarized images, preprocess the dataset by grouping images according to different defects, label the images in the dataset, mark the location and category of the target objects in the images, and construct the original data suitable for training the target detection model.

[0023] Step 2: Combining the polarization bidirectional reflection distribution function and Fresnel formula, the physical model is derived to obtain the distribution law of defect polarization information and the mapping relationship between defects and polarization distribution. Defects on the glass surface (such as scratches) are usually directional. When light shines on the edge of the defect, it will produce a strong depolarization or polarization state change. Images at various polarization angles can capture this optical anisotropy caused by the geometry of the defect. This invention utilizes three polarization channels of 0°, 45°, and 90° linear polarization obtained by a split-plane camera. Its physical significance lies in the complete reconstruction of the Stokes vector space describing the linear polarization state of the target surface. According to the theory of polarization bidirectional reflection distribution function (pBRDF), when light shines on a defect-free glass surface, it is controlled by Fresnel reflection, and its energy distribution among the three polarization channels follows a fixed physical ratio. However, when there are significant defects such as scratches, chipped edges, or impurities, the local abrupt change in geometric tilt angle or the change in material refractive index will cause the light to produce a depolarization effect or polarization azimuth angle deflection, thereby causing nonlinear fluctuations in the response intensity between 0°, 45°, and 90° linear polarization. The network structure designed in this invention enhances the detection of subtle defects by mining the differential features (i.e., polarization complementary features) between these three physical channels. Different defects in an image show significant differences in polarization information compared to flawless areas. Polarization imaging technology can store these differences in polarization images. Based on a convolutional neural network, a network layer for extracting polarization information is constructed according to an expression.

[0024] Step 3: The polarization component images (0°, 45°, 90° linear polarization) and intensity component S0 are used as inputs to the model. The dual-branch network extracts polarization information features separately, and then the features are fused through the polarization defect feature fusion module.

[0025] Step 4: Train the constructed network using the significant defect detection dataset described in Step 1 to obtain a weight file for polarization saliency detection. Use this trained weight file to perform defect saliency detection on glass in an industrial environment. Use Mean Absolute Error (MAE), Fmeasure (F), and Smeasure (S) as evaluation metrics to measure the model performance.

[0026] Furthermore, the method for constructing the polarization saliency detection dataset in step one is as follows: Figure 2 As shown: The target object surface 3 is illuminated by light source 2. Polarization component images of the target object at different angles and positions are captured by polarization camera 1 to obtain the original polarization image dataset. The required multidimensional polarization information in the image is obtained, noisy images are filtered out, and data enhancement operations such as flipping, mirroring and cropping are performed on the data. The captured dataset is saved to PC computer 4.

[0027] Figure 3The diagram illustrates the polarization-based saliency defect network. RGB and polarization images are input into the network. RGB captures color features, while polarization extracts high-dimensional polarization complementary features. Feature extraction is performed on each branch using a multi-branch encoder (MN) architecture. The MN feature extraction layer structure includes a saliency detection network employing a dual-branch encoder structure, and a ResNet-50 backbone network. Each convolutional module contains a 3x3 convolutional layer and a batch normalization layer, uniformly using the ReLU activation function to enhance nonlinear feature mining capabilities. Multi-branch features are then fused through a fusion network to achieve comprehensive representation. Subsequently, cross-modal feature fusion is performed, and finally, decoding generates a saliency prediction map.

[0028] Figure 4 This is the polarization fusion module. It is designed to fuse the feature representations of the three polarization branches and input them into the next-level network along with s0. The polarization fusion module (PCEM) first concatenates the feature maps of the three polarization branches (0°, 45°, and 90° linear polarization), then extracts the high-frequency polarization weights of the defect region through a spatial attention mechanism, and finally adds them element-wise with the feature map of the intensity component s0, achieving cross-modal fusion of light intensity information and polarization complementary information.

[0029] Figure 5 The example image used here is an example of predicting impurities and defects in glass during industrial inspection. The model uses pre-trained weights for material identification, outputs the defect location, performs saliency detection, and segments the target. It can be seen that the algorithm successfully separates the target defect from the background, providing a new solution for the field of defect detection.

[0030] Figure 6 This is a quantitative evaluation chart of the detection results. Mean Absolute Error (MAE), F-measure (F), and S-measure (S) are used to evaluate the model's performance. The experimental results clearly show that it has higher detection accuracy compared to traditional saliency detection models FCN and Unet algorithms. This invention helps in defect identification and segmentation. Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of this invention and are not intended to limit it.

[0031] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting significant defects based on polarization, characterized in that, Includes the following steps: Step 1: Construct a saliency defect detection dataset for polarized images. Place glass with different defects into the environment as targets. Under active light conditions, use a focal plane camera (Dofp) to capture images in four polarization directions (0°, 45°, 90° linear polarization and intensity component S0). Preprocess the saliency defect detection dataset by grouping the images according to different defects. Then, label the images in the dataset, marking the location and category of the target objects. Divide the training, validation, and test sets into an 8:1:1 ratio to construct the original data suitable for training the saliency detection model. Step 2: Combining the polarization bidirectional distribution function and Fresnel's formula, the physical model is derived to obtain the distribution law of defect polarization information and the mapping relationship between defects and polarization distribution. Different defects in the image have significant differences in polarization information compared to flawless areas. Polarization imaging technology can store this difference in polarization images. Based on the construction of a convolutional neural network, a network layer for extracting polarization information is built according to the expression. Step 3: Using the polarization component images (0°, 45°, 90° linear polarization) and intensity component S0 as inputs to the model, the dual-branch network extracts polarization information features respectively. By calculating the energy distribution differences of the 0°, 45°, and 90° polarization images, the depolarization effect and optical anisotropy generated by the defect region are quantified. The physical mapping relationship is then embedded into the initial feature extraction process of the neural network using a 1*1 convolutional layer, and finally fused through the polarization defect feature fusion module. Step 4: Train the constructed network using the significant defect detection dataset described in Step 1 to obtain a weight file for polarization saliency detection. Use this trained weight file to perform defect saliency detection on glass in an industrial environment. Use Mean Absolute Error (MAE), Fmeasure (F), and Smeasure (S) as evaluation metrics to measure the model performance.

2. The polarization-based significant defect detection method according to claim 1, characterized in that: The construction of the saliency detection dataset mentioned in step one: Glass with different defects is placed in the environment as the target object, and images are captured by a focal plane camera (Dofp) under active light conditions to obtain images in four polarization directions. The polarization information is extracted to obtain multidimensional polarization information in the images. At the same time, data interfered with by noise is removed. Based on the data screening, data augmentation techniques such as data cropping, flipping, moving, and mirroring are used to expand the dataset and improve the generalization ability of the model.

3. The polarization-based significant defect detection method according to claim 2, characterized in that: In step three, the polarization fusion module (PCEM) is designed to fuse the feature representations of the three polarization branches and fuse them with s0 as input to the next-level network.

4. The polarization-based significant defect detection method according to claim 3, characterized in that: RGB and polarization images are input into the network. RGB captures color features, and polarization extracts high-dimensional polarization complementary features. Feature extraction is performed on each branch through a multi-branch encoder (MN) network. The MN feature extraction layer structure includes a saliency detection network using a dual-branch encoder structure, and a ResNet-50 backbone network. Each convolutional module contains a 3x3 convolutional layer and a batch normalization layer, and uses the ReLU activation function to enhance nonlinear feature mining capabilities. Multi-branch features are then fused through a fusion network to achieve comprehensive representation. Subsequently, cross-modal feature fusion is performed, and finally, a saliency prediction map is generated through decoding.

Images