Underwater disturbance image restoration method based on three-stage reconstruction

Abstract

The invention provides an underwater disturbance image restoration method based on three-stage reconstruction, and belongs to the technical field of underwater imaging. According to the method, for the problems of image distortion and blurring caused by turbulence, thermal disturbance and other factors in an underwater imaging environment, periodic disturbance is removed by using a compressed sensing technology through feature point extraction and trajectory tracking; processing aperiodic disturbance by adopting an optical flow estimation algorithm of local polynomial expansion; fine disturbance elimination is carried out through a three-time non-rigid B-spline registration technology; finally, residual noise is removed through robust principal component analysis, and a high-quality restored image is obtained. According to the underwater disturbance image restoration method based on three-stage reconstruction, multiple technical advantages are integrated, the image quality and the measurement precision are effectively improved, and the method is especially suitable for scenes with high-precision imaging requirements such as nuclear fuel assembly detection and has good robustness and adaptability.

Description

Technical Field

[0001] The present invention relates to the technical field of underwater imaging, and in particular to a method for restoring an underwater disturbed image based on three-stage reconstruction. Background Art

[0002] In the field of underwater imaging, particularly in applications such as nuclear fuel assembly inspection, underwater archaeology, and ocean observation, images captured by imaging equipment often exhibit geometric distortion and blurring due to factors such as water turbulence, thermal disturbances, and variations in the medium's refractive index. These disturbances severely impact image quality and subsequent measurement accuracy, limiting the application scope and reliability of underwater visual inspection technology.

[0003] At present, research on underwater disturbed image restoration has made some progress. The main methods include: 1. Optical flow estimation algorithm: Optical flow estimation algorithms restore images by estimating the motion vectors of pixels in an image sequence. For example, Farneback's dense optical flow estimation algorithm based on polynomial expansion can estimate the displacement field of an image by comparing the motion between consecutive frames. However, this method generally assumes that the image perturbations are small and the brightness is constant, making it limited in its effectiveness when dealing with large or non-periodic perturbations.

[0004] 2. Compressed sensing technology: Compressed sensing theory uses sparse signal reconstruction to restore the original image using less sampled data. In underwater perturbed image restoration, compressed sensing can use the displacement trajectories of feature points to estimate global pixel motion, thereby restoring the image in its undisturbed state. However, this method is only applicable to periodic perturbations, and the restoration effect is significantly affected by the number and distribution of feature points.

[0005] 3. Image registration technology: Image registration eliminates geometric distortion by aligning a perturbed image with a reference image. Traditional image registration methods include rigid and non-rigid registration. Non-rigid registration techniques, such as B-spline interpolation, are better able to handle local geometric deformations. However, image registration effectiveness is highly dependent on the quality of the reference image and is computationally complex in practical applications.

[0006] 4. Deep Learning Methods: In recent years, deep learning has made significant progress in image restoration. For example, underwater image de-perturbation algorithms based on generative adversarial networks (GANs) can restore clear images by learning the image's distortion field. However, these methods typically require large amounts of training data and still have certain limitations when dealing with complex underwater environments.

[0007] Although the above methods can improve the quality of underwater disturbance images to a certain extent, they each have obvious limitations: The optical flow estimation algorithm is only applicable to scenes with small disturbances and is not very effective for large disturbances or non-periodic disturbances.

[0008] Compressed sensing technology can only process disturbances with periodic characteristics, and has high requirements on the number and distribution of feature points.

[0009] Image registration technology relies on high-quality reference images and is prone to falling into local optimality under complex perturbations.

[0010] Deep learning methods require a large amount of training data and have high requirements on hardware resources in practical applications, making it difficult to meet real-time requirements.

[0011] In addition, existing technologies often find it difficult to simultaneously meet the requirements of high precision, high robustness and real-time performance when dealing with complex underwater environments (such as the high radiation and high turbulence environments in nuclear fuel assembly inspection). Summary of the Invention

[0012] The purpose of the present invention is to provide an underwater disturbance image restoration method based on three-stage reconstruction, which integrates multiple technical advantages to effectively improve image quality and measurement accuracy. It is particularly suitable for scenarios with high-precision imaging requirements such as nuclear fuel component inspection, and has good robustness and adaptability.

[0013] To achieve the above object, the present invention provides a method for restoring an underwater disturbed image based on three-stage reconstruction, comprising the following steps: Step S1: extracting feature points from the underwater disturbance image sequence, performing trajectory tracking, reconstructing the image using compressed sensing technology, and removing periodic disturbances; Step S2: Using a local polynomial expansion optical flow estimation algorithm to calculate the optical flow between consecutive frames, and removing disturbances from the image through backward mapping; Step S3: Using the cubic non-rigid B-spline registration technique to perform refined disturbance elimination on the image processed by the first two stages; Step S4: Use the robust principal component analysis method to remove the sparse noise remaining in the image to obtain the final restored image.

[0014] Preferably, in step S1, the compressed sensing technology performs sparse decomposition on the displacement trajectory of the feature points through Fourier transform, and reconstructs the displacement trajectory of the global pixel points using the compressed sensing algorithm, and then obtains a preliminary restored image through reverse interpolation.

[0015] Preferably, in step S2, the optical flow estimation algorithm based on local polynomial expansion estimates the motion displacement of each pixel by calculating the optical flow vector between consecutive frames, and uses these displacement vectors to perform reverse mapping on the image to remove non-periodic disturbances.

[0016] Preferably, in step S3, the non-rigid B-spline registration technology corrects local geometric deformation in the image by controlling the optimization of grid points.

[0017] Preferably, in step S4, the robust principal component analysis method decomposes the image matrix to separate sparse noise and low-rank components, thereby removing the sparse noise remaining in the image.

[0018] Therefore, the present invention adopts the above-mentioned underwater disturbance image restoration method based on three-stage reconstruction, and the beneficial technical effects are as follows: (1) The present invention uses a phased approach to gradually remove periodic and non-periodic disturbances from images. First, compressed sensing technology is used to remove periodic disturbances. Then, an optical flow estimation algorithm based on local polynomial expansion is used to process non-periodic disturbances. Finally, a cubic non-rigid B-spline registration technique is used for fine-tuning. This combination of techniques significantly improves the accuracy of image restoration, effectively recovering image details and structural information, meeting the high-precision imaging requirements of nuclear fuel assembly inspection and other applications.

[0019] (2) This paper introduces advanced image processing techniques at multiple stages, such as sparse decomposition for compressed sensing, motion vector analysis for optical flow estimation, and local geometric deformation correction for non-rigid registration. This allows the method to adapt to complex underwater environments, including high turbulence, thermal disturbances, and high-radiation backgrounds. Experimental results show that the method exhibits good robustness on various datasets (such as TianSet, JamesSet, and OurSet), and can effectively restore high-quality images even when images have large blur and sparse texture features.

[0020] (3) The three-stage reconstruction strategy adopted by the present invention is not only applicable to the complex working conditions of nuclear fuel assembly inspection, but can also be flexibly adjusted according to different application scenarios. For example, when processing images with periodic perturbations, compressed sensing technology can quickly remove background noise; while when processing non-periodic perturbations, optical flow estimation and non-rigid registration techniques can further optimize image quality. This flexibility makes the present invention have broad application prospects in various underwater imaging tasks.

[0021] (4) Through refined image restoration, the present invention can significantly improve the accuracy of image-based measurements. In nuclear fuel assembly inspections, the restored images can more accurately reflect the diameter and gap size of the fuel rods, providing a more reliable basis for nuclear safety and fuel assembly reliability assessments.

[0022] (5) The present invention fully considers the complexity and practicality of the system in its design. By optimizing the algorithm process and reducing the dependence on hardware resources, this method can achieve efficient image restoration at a low computational cost. For example, in the non-rigid B-spline registration stage, by controlling the optimization of grid points, the computational workload is reduced while improving the registration accuracy. This design makes the present invention not only suitable for laboratory environments, but also can be quickly deployed and applied in actual industrial scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a flow chart of a method for restoring an underwater disturbed image based on three-stage reconstruction according to the present invention; Figure 2 Comparison of the restoration effects of four restoration schemes on TianSet; Figure 3 Comparison of the recovery effects of four recovery schemes on JamesSet; Figure 4 Comparison of the restoration effects of four restoration schemes on OurSet. DETAILED DESCRIPTION

[0024] The technical solution of the present invention is further described below with reference to the accompanying drawings and embodiments.

[0025] Unless otherwise defined, technical or scientific terms used in the present invention shall have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs.

[0026] Example 1 like Figure 1 As shown, a method for restoring an underwater disturbed image based on three-stage reconstruction includes the following steps: Step S1: extracting feature points from the underwater disturbance image sequence, performing trajectory tracking, reconstructing the image using compressed sensing technology, and removing periodic disturbances; Compressed sensing technology uses Fourier transform to sparsely decompose the displacement trajectory of feature points, and uses compressed sensing algorithm to reconstruct the displacement trajectory of global pixel points, and then obtains a preliminary restored image through reverse interpolation.

[0027] Step S2: Using a local polynomial expansion optical flow estimation algorithm to calculate the optical flow between consecutive frames, and removing disturbances from the image through backward mapping; The optical flow estimation algorithm based on local polynomial expansion estimates the motion displacement of each pixel by calculating the optical flow vector between consecutive frames, and uses these displacement vectors to reversely map the image to remove non-periodic disturbances.

[0028] Step S3: Using the cubic non-rigid B-spline registration technique to perform refined disturbance elimination on the image processed by the first two stages; Non-rigid B-spline registration technology corrects local geometric deformation in images by controlling the optimization of grid points.

[0029] Step S4: using a robust principal component analysis method to remove the sparse noise remaining in the image to obtain the final restored image; The robust principal component analysis method decomposes the image matrix to separate sparse noise and low-rank components, thereby removing the residual sparse noise in the image and obtaining a high-quality restored image.

[0030] The present invention will be further described below through specific examples.

[0031] In this embodiment, the following data sets are used for testing: TianSet: collected and compiled by Tian et al., characterized by a large degree of disturbance and a certain periodicity, and the image content is mainly text.

[0032] JamesSet and SyntheticSet: collected and compiled by James et al., they are characterized by low disturbance and obvious periodicity, with regular grids in the image background and obvious features.

[0033] OurSet: We collected and created our own dataset. The images are relatively simple, with some images being blurred to a large extent and with sparse texture features. The images are mainly from ImageNet2012, and some images are simulated stripe images and real objects (optical support rods). We collected 10 101=1010 images for traditional restoration, 3897 18 = 70146 images used for deep learning. Image size is 512 384.

[0034] Experimental content.

[0035] This example compares four restoration schemes (Oreifej, James, Zhang, and the scheme proposed in this example, collectively referred to as Ours). Twenty-five perturbed image sequences were used. Except for the image sequences in the public dataset, which have a fixed frame count (61 frames / 101 frames), all other image sequences used 101 frames. Experimental equipment: Intel i5-9400F, 16GB RAM, MATLAB R2020a.

[0036] (1) Comparison of restoration effects on TianSet.

[0037] The final effects of the four recovery schemes are as follows Figure 2 As shown in the top row of images, the other three algorithms all have some degree of perturbation or blur that is not completely eliminated. In the second row of images, the last line of text is magnified. Comparing the word "ponds", it can be seen that the proposed algorithm has slightly better recovery effect than the other three. Comparing the third row of images, it can be clearly seen that the recovery effect is better than the other three. In the last row of images, except for the Oreifej method, the other three methods have very similar recovery effects.

[0038] (2) Comparison of restoration effects on JamesSet.

[0039] from Figure 3 Comparing the restoration results, except for the Oreifej method using image registration, the other three methods all show excellent restoration effects, and the human eye can hardly tell the difference from the original image.

[0040] (3) Comparison of restoration effects on OurSet.

[0041] from Figure 4 Comparing the restoration effects, except that the James method has a certain degree of blur in the restoration results of the three images, the restoration results of the other three algorithms are all good.

[0042] It is worth noting that the contents not elaborated in detail in the present invention are all prior art and are well known to those skilled in the art.

[0043] Therefore, the present invention adopts the above-mentioned underwater disturbance image restoration method based on three-stage reconstruction, integrates multiple technical advantages, effectively improves image quality and measurement accuracy, and is particularly suitable for scenarios with high-precision imaging requirements such as nuclear fuel component inspection, and has good robustness and adaptability.

[0044] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solutions of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for restoring underwater disturbed images based on three-stage reconstruction, characterized in that: The following steps are involved: Step S1: extracting feature points from the underwater disturbance image sequence, performing trajectory tracking, reconstructing the image using compressed sensing technology, and removing periodic disturbances; Step S2: Using a local polynomial expansion optical flow estimation algorithm to calculate the optical flow between consecutive frames, and removing disturbances from the image through backward mapping; Step S3: Using the cubic non-rigid B-spline registration technique to perform refined disturbance elimination on the image processed by the first two stages; Step S4: Use the robust principal component analysis method to remove the sparse noise remaining in the image to obtain the final restored image.

2. The underwater disturbance image restoration method based on three-stage reconstruction according to claim 1 is characterized in that: In step S1, the compressed sensing technology performs sparse decomposition of the displacement trajectory of the feature points through Fourier transform, and reconstructs the displacement trajectory of the global pixel points using the compressed sensing algorithm, and then obtains a preliminary restored image through reverse interpolation.

3. The underwater disturbance image restoration method based on three-stage reconstruction according to claim 1 is characterized in that: In step S2, the optical flow estimation algorithm based on local polynomial expansion estimates the motion displacement of each pixel by calculating the optical flow vector between consecutive frames, and uses these displacement vectors to perform reverse mapping on the image to remove non-periodic disturbances.

4. The underwater disturbance image restoration method based on three-stage reconstruction according to claim 1 is characterized in that: In step S3, the non-rigid B-spline registration technology corrects the local geometric deformation in the image by controlling the optimization of the grid points.

5. The underwater disturbance image restoration method based on three-stage reconstruction according to claim 1 is characterized in that: In step S4, the robust principal component analysis method decomposes the image matrix to separate the sparse noise and low-rank components, thereby removing the sparse noise remaining in the image.

Images