An underwater crack defect detection method based on polarized light field imaging

Abstract

The application discloses an underwater crack defect detection method based on a polarized light field imaging, first, a polarized light field collection device is built, aiming at the underwater crack defect data problem, a small-scale underwater crack defect dataset is established and an unsupervised network is used for detection; the collected polarized light field image is processed to obtain a visible light intensity, a polarization degree and a depth image, a selection attention module is designed to fuse deep features of the three images, a multi-scale network structure is used to take into account crack defects of different sizes, and the detection accuracy is improved. The application can improve the detection precision, and achieve the purpose of quickly and accurately identifying and detecting underwater facility crack defects.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of pattern recognition, and particularly relates to an underwater crack defect detection method. BACKGROUND

[0002] Dam, underwater oil and gas pipeline and other hydraulic structures play an important role in ensuring the safety of hydropower and natural gas energy. However, due to the complex underwater environment, various defects and damages will inevitably occur in the long-term use of hydraulic structures, which will affect the safe operation of the facilities and even cause major accidents in serious cases, threatening the safety of people's lives and property. Therefore, crack defect detection of hydraulic structures has important research significance.

[0003] The visual-based underwater crack defect detection method relies on clear image information of the scene, but factors such as weak light, strong absorption and scattering, and various suspended matters in the complex underwater environment often result in low image quality, which seriously affects the accuracy of defect detection. In addition, there is no public underwater crack defect dataset, and it is difficult to collect a large number of hydraulic structure crack defect images in real environment. In the face of these problems, a polarized light field imaging device is used to collect underwater crack image data, which uses polarized light field imaging to suppress water backscattering light and can obtain the depth information of the scene, thereby improving the underwater image clarity and expanding the image information dimension. However, there is no fixed method for using multi-dimensional information to detect underwater crack defects, so it is still necessary to study effective methods to improve the detection accuracy. SUMMARY

[0004] In order to overcome the shortcomings of the prior art, the application provides an underwater crack defect detection method based on polarized light field imaging. First, a polarized light field acquisition device is built, a small-scale underwater crack defect dataset is established for the underwater crack defect data problem, and an unsupervised network is used for detection. The collected polarized light field images are processed to obtain visible light intensity, polarization degree and depth images, a selection attention module is designed to fuse the deep features of the three images, a multi-scale network structure is used to consider crack defects of different sizes, and the detection accuracy is improved. The application can improve the detection accuracy and achieve the purpose of quickly and accurately identifying and detecting the crack defects of underwater facilities.

[0005] The technical solution adopted by the application to solve the technical problems comprises the following steps:

[0006] Step 1: Build a polarized light field acquisition device;

[0007] A 2x2 planar array camera system is composed of 4 visible light polarization cameras, and the 4 visible light polarization cameras have overlapping fields of view;

[0008] Camera synchronization control using hardware synchronization trigger device: the synchronization trigger outputs a pulse signal to the camera, and the camera performs a shooting action according to the pulse signal;

[0009] Data transmission using a gigabit switch, connecting the gigabit switch with each camera and PC; using a four-port gigabit network card on the PC, each port is responsible for powering a camera and transmitting the data collected by the camera;

[0010] Before using the polarized camera array acquisition device, it is calibrated in air and underwater environment respectively to obtain the internal and external parameters of the camera array;

[0011] Step 2: Establishing the underwater crack defect data set;

[0012] Using the polarized camera array to collect underwater target images from multiple perspectives, using a cement board to simulate underwater buildings, and by adding milk to the water to simulate different turbidity levels of underwater environment, and using a turbidity meter to measure the turbidity of the water after adding milk; The experimental scene is built in a glass water tank, and the glass water tank is filled with water. When collecting images, the cement board is fixed in the glass water tank, and the polarized camera array acquisition device is used to collect images; When collecting, the camera resolution is set to the maximum resolution, and the light source is a halogen photography lamp simulating natural light; Finally, the collected images are used to establish the underwater crack defect data set;

[0013] Step 3: Process the collected data;

[0014] Polarization demosaicing is performed on the images of the underwater crack defect data set to obtain four I0, I 45 , I 90 , I 135 images of different polarization directions, and further calculation to obtain the visible light intensity image, and then perform underwater polarization clarification processing;

[0015] According to the I0, I 45 , I 90 , I 135 images, the polarization degree image of the target scene is calculated;

[0016] Meanwhile, the depth image of the scene is calculated from the original images of the underwater crack defect data set;

[0017] Step 4: Designing a crack defect detection network and training it;

[0018] The crack defect detection network uses a teacher-student network framework, the teacher network is a network pre-trained on image classification tasks, using ResNet-18 feature extraction network, the student network uses the same architecture as the teacher network, and the same level features are fused using a selection attention module, as follows:

[0019] Step 4-1: first, the clarified visible light intensity image, the polarization degree image, the depth image are respectively input into the teacher-student network, and the respective feature maps {C k,l} are obtained through feature extraction of the network, wherein k={1, 2, 3} represents three inputs, and l={1, 2, 3} represents three feature levels;

[0020] Step 4-2: the feature maps from the same level are weighted and fused through the spatial weights {alpha 1,l , alpha 2,l , alpha 3,l} of the three branches of the selection attention module, to obtain the fused features F l = alpha 1,l C 1,l + alpha 2,l C 2,l + alpha 3,l C 3,l of different levels;

[0021] Step 4-3: the fused features F l of different levels obtained by the teacher network and the student network are calculated to obtain the loss loss;

[0022] Step 4-4: the loss loss calculated in step 4-3 is weighted and summed as the total loss, so as to train the student network, and the weight parameters of the teacher network are fixed and unchanged during the training process;

[0023] Step 4-5: in the test process, the distance between the feature maps output by the teacher network and the student network is calculated to obtain the anomaly maps of each level of the test image, and the anomaly maps of each level are up-sampled and multiplied element by element to obtain the final detection result.

[0024] Preferably, the size of the glass water tank is 60*40*40cm.

[0025] Preferably, the camera uses the maximum resolution of 2448*2224 when collecting images, and the focal length is 16mm lens.

[0026] The beneficial effects of the present application are as follows:

[0027] The present application uses the unique advantages of the polarization light field imaging method in the underwater environment to collect underwater target images, thereby providing multi-dimensional image information of the underwater target, designs an unsupervised detection network to fuse the input image features, improves the detection precision, and achieves rapid and accurate identification and detection of underwater facility crack defects, which is helpful for the safe operation of dams and underwater oil and gas pipelines and protects the safety of people's lives and property. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 is a physical diagram of the polarization camera array acquisition device of the present application.

[0029] Figure 2 is a physical diagram of the synchronous trigger of the present application.

[0030] Figure 3 is part of the underwater image data collected by the embodiment of the present application.

[0031] Figure 4 is the data processing result of the embodiment of the present application.

[0032] Figure 5 is the crack defect detection network architecture of the present application.

[0033] Figure 6 is the network detection result of the embodiment of the present application. DETAILED DESCRIPTION

[0034] The present application will be further described below in conjunction with the drawings and embodiments.

[0035] The purpose of the present application is to provide an underwater crack defect detection method based on polarization light field imaging to solve the problem of crack defect detection of hydraulic facilities.

[0036] The present application provides an underwater crack defect detection method based on polarization light field imaging, which is implemented according to the following steps.

[0037] 1. First, build a polarization light field acquisition device. In the present application, four visible light polarization cameras are assembled into a planar array. The polarization camera has high image resolution, high acquisition frame rate, and can use a hardware synchronous trigger device to ensure the synchronization of the acquired data. In order to ensure good imaging effect, a certain overlapping field of view is required between the cameras to obtain multi-view information. The center-to-center distance between the cameras in the built polarization camera array is kept fixed horizontally and vertically. The cameras are fixed on the steel plate with holes through screws, and finally the entire array device is supported by a tripod. The entire imaging device is as shown in Figure 1 . Finally, a 2x2 planar array camera system is formed.

[0038] When using a polarization camera array to collect images, each camera needs to collect at the same time, so the synchronization problem between the cameras needs to be considered. The polarization camera used in the present application is an industrial camera, so a hardware synchronous trigger device can be used for synchronization control, as shown in Figure 2 . The synchronous trigger outputs a pulse signal to the camera, and the camera takes a picture according to the external trigger signal. The synchronous trigger can effectively control the camera to collect synchronously, ensuring that the collected image data is of the same scene at the same time, providing a basis for subsequent image processing algorithms.

[0039] The synchronous trigger ensures that the camera array can synchronously collect data, however, synchronous collection also means that there will be a large amount of data to be transmitted, which brings great challenges to the synchronous data transmission module. A gigabit switch is used for data transmission, which is connected with each camera and the PC. In order to solve the problem of very low frame rate or even camera disconnection during synchronous collection, the network card on the PC is replaced with a four-port gigabit network card, each port is responsible for powering and transmitting the data collected by a camera, which ensures that there is enough bandwidth to transmit data, so that the camera can synchronously collect and have a higher collection frame rate.

[0040] Before using the polarization camera array collection device, the camera needs to be calibrated to obtain the internal and external parameters of the camera array for subsequent image algorithm processing. In the present application, different application scenarios are calibrated in air and underwater environment to obtain the corresponding camera parameters.

[0041] 2. Establishing an underwater crack defect data set. Since there is no public underwater crack defect data set at present, and it is difficult to collect crack defects existing in hydraulic facilities in real environment, the present application carries out simulation experiment in laboratory environment to establish an underwater crack defect data set, which provides data basis for subsequent detection algorithm. The present application uses a polarization camera array to collect underwater target images from multiple perspectives, uses a cement board to simulate underwater buildings, drops milk into water to simulate underwater environments with different turbidity, and uses a turbidity meter to measure the turbidity of water after adding milk. The experimental scene is built in a 60x40x40(cm) glass water tank, the glass water tank is filled with water, the cement board is fixed in the glass water tank during image collection, and the polarization camera array collection device is used for collection. The camera resolution is set to the maximum resolution 2448x2224 during collection, a 16mm focal length camera lens is used, and a halogen photographic lamp simulating natural light is used as light source. The collected underwater image data is shown in Figure 3 .

[0042] 3. Processing collected data. The collection device used in the present application is a polarization camera array, and the collected image data is four original mosaic images from different perspectives, so further processing is needed. First, the polarization demosaicing processing is performed on the image to obtain four I0, I 45 , I 90 , I 135The visible light intensity image is further calculated. In addition, the polarization light field imaging can effectively suppress the backscattering light of the water body and improve the underwater image definition, so that the underwater image is further processed for underwater polarization clarification. The array camera is used in the present application to collect target images from multiple viewing angles, and the depth information of the scene can be calculated according to the multi-view stereo geometry, so that the depth map of the scene can be calculated from the original image data. The final data processing result is shown in Figure 4 .

[0043] 4. Designing a crack defect detection network and training the network. As shown in Figure 5 , the network body adopts a teacher-student network framework, and the teacher network is a network pre-trained on an image classification task. In the present application, a ResNet-18 feature extraction network is used. At the same time, in order to reduce information loss, the student network adopts the same architecture as the teacher network, and in addition, a selection attention module is used for fusion for the same level feature. The forward working process of the network will be described below: Figure 5

[0044] (1) First, the clarified visible light intensity image, the polarization degree image and the depth image are respectively input into the teacher-student network, and the feature maps {C k,l} of each are obtained through feature extraction of the network, wherein k={1,2,3} represents three inputs, and l={1,2,3} represents three feature levels;

[0045] (2) The feature maps from the same level are fused by the spatial weights {α 1,l ,α 2,l ,α 3,l} of the three branches of the selection attention module, to obtain the fused features F l =α 1, l C 1,l +α 2,l C 2,l +α 3,l C 3,l , l={1,2,3};

[0046] (3) The fused features F l from the teacher network and the student network of different levels are calculated to obtain the loss loss;

[0047] (4) The loss loss calculated in (3) is weighted and summed as the total loss to train the student network. The weight parameters of the teacher network are fixed and unchanged during the training process;

[0048] ​(5) In the test process, the distance between the feature maps output by the teacher network and the student network is calculated to obtain the anomaly map of each level of the test image, and the anomaly maps of each level are up-sampled and multiplied element by element to obtain the final detection result. Embodiments

[0050] According to the foregoing steps, first, the polarization camera array acquisition device is built, and the calibration of the device is completed using the corresponding calibration algorithm. For the crack defects existing on the hydraulic facilities, the target image is collected using the polarization camera array acquisition device, and then the data processing is performed according to the foregoing description to obtain the clear visible light image, the polarization degree image and the depth image. The trained weight file is loaded into the network model, the three images are input into the network, and the network outputs the detection result after forward inference, as shown in Figure 6 .

Claims

1. A method for detecting underwater crack defects based on polarized light field imaging, characterized in that, The method comprises the following steps: Step 1: build a polarized light field acquisition device; Use four visible light polarization cameras to form a 2x2 planar array camera system, and the four visible light polarization cameras have an overlapping field of view; Use a hardware synchronization trigger device for camera synchronization control: the synchronization trigger outputs a pulse signal to the camera, and the camera performs a shooting action according to the pulse signal; Use a gigabit switch for data transmission, connect the gigabit switch with each camera and the PC, and use a four-port gigabit network card on the PC, with each port responsible for powering and transmitting data collected by a camera; Before using the polarization camera array acquisition device, calibrate it in air and underwater environments to obtain the internal and external parameters of the camera array; Step 2: Establish an underwater crack defect dataset; Use the polarization camera array to collect underwater target images from multiple perspectives, use a cement board to simulate an underwater building, drop milk into the water to simulate different turbidity levels of the underwater environment, and use a turbidity meter to measure the turbidity of the water after adding milk; The experimental scene is built in a glass water tank, which is filled with water. When collecting images, the cement board is fixed in the glass water tank, and the polarization camera array acquisition device is used for collection; When collecting, the camera resolution is set to the maximum resolution, and the light source is a halogen photography lamp simulating natural light; Finally, an underwater crack defect dataset is established using the collected images; Step 3: Process the collected data; The polarization demosaicking processing is performed on the images of the underwater crack defect data set to obtain I0, I 45 , I 90 , I 135 images of four different polarization directions, and the visible light intensity image is further calculated, and then the underwater polarization clarification processing is performed. According to I0, I 45 , I 90 , I 135 The image calculates the polarization degree image of the target scene; At the same time, the depth image of the scene is calculated from the original images of the underwater crack defect dataset; Step 4: Design a crack defect detection network and train it; The crack defect detection network uses a teacher-student network framework, the teacher network is a pre-trained network for image classification tasks, and the ResNet-18 feature extraction network is used, the student network uses the same architecture as the teacher network, and the selection attention module is used for fusion for the same level features, as follows: Step 4-1: Firstly, the clear visible light intensity image, the polarization degree image, and the depth image are input into the teacher-student network respectively, and the feature maps {C k,l} are obtained through feature extraction of the network, wherein k = {1, 2, 3} represents three inputs, and l = {1, 2, 3} represents three feature levels. Step 4-2: Feature maps from the same level are fused by weighting the feature maps of the three branches of the selection attention module through the spatial weights {a 1,l , a 2,l , a 3,l}, to obtain the fused feature of different levels F l = a 1,l C 1,l + a 2,l C 2,l + a 3,l C 3,l ; Step 4-3: The fused feature F obtained by the different levels of teacher network and student network l Calculate the loss loss; Step 4-4: The losses calculated in step 4-3 at different levels are weighted and summed as the total loss to train the student network, and the weight parameters of the teacher network are fixed during training; Step 4-5: In the test process, the distance between the feature maps output by the teacher network and the student network is calculated to obtain the abnormal images at each level of the test image, and the abnormal images at each level are upsampled and multiplied element by element to obtain the final detection result.

2. The underwater crack defect detection method based on polarization optical field imaging according to claim 1, characterized in that, The size of the glass water tank is 60x40x40cm.

3. The underwater crack defect detection method based on polarized light field imaging according to claim 1, characterized in that, The camera uses the maximum resolution of 2448x2224 when collecting images, and the focal length is 16mm.

Images