An image restoration method based on Helmholtz-Hodge decomposition
Through Helmholtz-Hodge decomposition and image registration technology, geometric distortion and color shift problems in underwater vacuum image restoration are solved, and the rapid and accurate restoration of irregularly distorted images is achieved, which improves adaptability and processing speed. More visual effects and real-time.
Patent Information
- Application Number
- CN202111309872.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-11-07
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2041-11-07
AI Technical Summary
The existing underwater image restoration technology has problems of geometric distortion, color shift and low contrast. The image sequence-based method has a large amount of calculation and high time complexity, while the method based on water surface waveform estimation has high environmental requirements and relies on Modeling accuracy.
Using the image restoration method based on Helmholtz-Hodge decomposition, the adaptively adjustable structured light is projected to the fluid surface, and the two-dimensional twisted vector field is extracted using feature points of the twisted structured light image and the reference structured light image to perform Helmholtz-Hodge decomposition. Divergence potential, curvature potential and harmonization components are obtained, interpolation processing is performed and image registration is performed to achieve fast and accurate image restoration.
It realizes fast and accurate restoration of an irregularly distorted image, with stronger adaptability and real-time performance, ensuring the reliability and accuracy of distorted image restoration, with better visual effects and faster processing speed.
Smart Images

Figure CN114022386B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the field of optical imaging technology, in particular to an underwater aerial imaging image restoration technology, in particular to an image restoration method based on Helmholtz-Hodge decomposition. Background Art
[0002] As one of the important means for human beings to explore the ocean world, underwater imaging technology has been widely used in marine geological exploration, underwater target identification and marine military. However, the random fluctuations of the water surface and the complexity of the underwater environment will cause the observed image to have problems such as geometric distortion, color deviation and low contrast. Therefore, it is of great significance to carry out research on underwater imaging image restoration technology.
[0003] At present, the research on image restoration technology is mainly carried out from two aspects.
[0004] The first is an image restoration method based on image sequences. This scheme assumes that the real image without distortion is hidden in the image sequence of the distorted scene observed through the water surface many times. It assumes that the scene and the camera are stationary, and uses the image sequence of the same stationary scene to reconstruct the undistorted image. This type of algorithm requires a video stream of a certain length, generally requires multiple iterations, has a large amount of calculation, high time complexity, and cannot guarantee real-time performance.
[0005] The second is the restoration method based on water surface waveform estimation. This type of algorithm first estimates the water surface waveform, and then restores the distorted image synchronously based on the waveform. The restoration method based on water surface waveform estimation is suitable for the restoration of most degraded images caused by water surface ripples. It does not require image registration or prior knowledge, and has a certain effect on distortion and blur. However, this type of method has high requirements on the environment and is very dependent on the accuracy of modeling. Summary of the invention
[0006] In view of the shortcomings of the existing underwater air imaging image restoration algorithm, the present invention provides an image restoration method based on Helmholtz-Hodge decomposition. The method can quickly and accurately restore the image through only one irregularly distorted image, has good real-time performance, has stronger adaptability, ensures the reliability and accuracy of distorted image restoration, and has better visual effects and faster processing speed.
[0007] The technical solution adopted by the present invention to solve its technical problem is:
[0008] An image restoration method based on Helmholtz-Hodge decomposition comprises the following steps:
[0009] Step 1), projecting an adaptively adjustable reference structured light image onto the surface of the fluid, distorting the structured light due to the fluctuation of the fluid surface to obtain a distorted structured light image, and simultaneously obtaining a distorted target scene image through the same wavefront, extracting feature points from the distorted structured light image and the reference structured light image, and obtaining a two-dimensional distorted vector field of structured light feature points;
[0010] Step 2), perform Helmholtz-Hodge decomposition on the two-dimensional distortion vector field that causes image distortion to obtain the divergence potential D, the curl potential R and the harmonic component ;
[0011] Step 3), the divergence potential D, curl potential R and harmonic components obtained in step 2) Interpolation processing obtains a continuous and smooth divergence component , a continuous and smooth curl component and a continuous and smooth harmonic component , and the two-dimensional distortion vector field of the structured light image is estimated by the continuous and smooth divergence component, the continuous and smooth curl component and the continuous and smooth harmonic component;
[0012] Step 4), performing image registration on the distorted structured light image and outputting a restored structured light image;
[0013] Step 5), perform image registration on the distorted target scene image and output a restored target scene image.
[0014] Furthermore, in step 1), the two-dimensional distorted vector field The definition is as follows:
[0015]
[0016] Among them, (U ij ,V ij ) is the pixel coordinate of the distorted structured light image, (u ij ,v ij ) is the pixel coordinate of the reference structured light image, i represents the row value of the image, and j represents the column value of the image.
[0017] In step 2), the two-dimensional distorted vector field Perform Helmholtz-Hodge decomposition to distort the vector field in two dimensions Its divergence and curl And when the boundary conditions are known, any two-dimensional distorted vector field can be uniquely decomposed into the following three components:
[0018]
[0019] in, is the non-rotational component perpendicular to the boundary, is the incompressible component parallel to the boundary, To balance the weight, is the gradient operator, operator J is a two-dimensional vector rotated counterclockwise by π / 2. The divergence potential D and the curl potential R are obtained by solving the following Poisson equation, and the harmonic component is obtained by calculation ,
[0020]
[0021] In step 4), the two-dimensional distortion vector field of the structured light image estimated in step 3) is used to solve the spatial transformation model between the distorted structured light image and the reference structured light image:
[0022]
[0023] Among them, (U ij ,V ij ) is the pixel coordinate of the distorted structured light image, (u ij ,v ij ) is the pixel coordinate of the reference structured light image, i represents the row value of the image, and j represents the column value of the image; the restored structured light image is then obtained by image registration.
[0024] In step 5), the spatial transformation model of the distorted structured light image and the reference structured light image in step 4) is used to further solve the spatial transformation model of the distorted target scene image and the restored target scene image.
[0025]
[0026] in, is the pixel coordinate of the distorted target scene image, is the pixel coordinate of the restored target scene image, i represents the row value of the image, j represents the column value of the image, and λ is the physical quantity of the displacement proportional relationship between the reflected and refracted light; the restored target scene image is then obtained through image registration.
[0027] The present invention can be used for water, including river water, lake water and sea water, as well as fluids such as alcohol and oil.
[0028] This method actively projects adaptively adjustable structured light onto the fluid surface. The fluctuation of the fluid surface causes the distortion of the structured light. The displacement vector field of the distorted structured light image and the standard structured light image is decomposed by the Helmholtz-Hodge decomposition theory. Finally, the distorted original scene is restored synchronously through algorithmic deconvolution and projection reconstruction technology.
[0029] Advantages or beneficial effects of the present invention:
[0030] (1) A regular structured light image is introduced in advance, and then the fluid surface fluctuation information is obtained through Helmholtz-Hodge decomposition, which avoids the problems of large computational complexity and poor real-time performance of image matching restoration methods;
[0031] (2) The image can be restored quickly and accurately through an irregularly distorted image, which has low requirements on the environment and has stronger adaptability;
[0032] (3) It is supported by complete and mature mathematical theories, which can ensure the reliability and accuracy of distorted image restoration. Compared with traditional image restoration algorithms, it has better visual effects and faster processing speed. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] Figure 1 This is a flow chart of an image restoration method based on Helmholtz-Hodge decomposition in an embodiment;
[0034] Figure 2a-2i This is a specific process of the image restoration method based on Helmholtz-Hodge decomposition in the embodiment, wherein: Figure 2a It is a standard structured light image; Figure 2b To distort the structured light image; Figure 2c is the distorted target scene image; Figure 2d Helmholtz-Hodge decomposition for a two-dimensional distorted vector field; Figure 2e is the non-rotational component ; Figure 2f The incompressible component ; Figure 2g To reconcile the ; Figure 2h is the restored structured light image; Figure 2i is the restored target scene image. DETAILED DESCRIPTION
[0035] The contents of the present invention are described in detail below in conjunction with the drawings and embodiments. The embodiments described with reference to the drawings are exemplary. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments, and are intended to be used to explain the present invention, but cannot be understood as limiting the present invention.
[0036] Example:
[0037] The following describes an image restoration method based on Helmholtz-Hodge decomposition proposed in an embodiment of the present invention with reference to the accompanying drawings. Based on structured light active imaging, an adaptive and adjustable regular structured light is first projected onto the water surface. The random disturbance of the water surface causes the structured light to be distorted. The feature point position information of the distorted structured light and the standard structured light is used, and then the water surface fluctuation information is obtained by Helmholtz-Hodge decomposition. Finally, the original target scene image is synchronously restored by algorithmic deconvolution and projection reconstruction.
[0038] like Figure 1 As shown, an image restoration method based on Helmholtz-Hodge decomposition includes the following steps:
[0039] Step (1), projecting an adaptively adjustable reference structured light image onto the water surface, such as Figure 2a As shown in , the structured light is distorted due to the fluctuation of the water surface, and a distorted structured light image is obtained, such as Figure 2b As shown, the distorted target scene image through the same wave surface is obtained at the same time, such as Figure 2c As shown, feature points are extracted from the distorted structured light image and the reference structured light image to obtain a two-dimensional distorted vector field of the structured light feature points. like Figure 2d As shown, the two-dimensional distorted vector field The definition is as follows:
[0040]
[0041] Among them, (U ij ,V ij ) is the pixel coordinate of the distorted structured light image, (u ij ,v ij ) is the pixel coordinate of the reference structured light image, i represents the row value of the image, and j represents the column value of the image;
[0042] Step (2), perform Helmholtz-Hodge decomposition on the two-dimensional distortion vector field that causes image distortion to obtain the divergence potential D, the curl potential R and the harmonic component :
[0043] Distorting a vector field in two dimensions Perform Helmholtz-Hodge decomposition to distort the vector field in two dimensions Its divergence and curl And when the boundary conditions are known, any two-dimensional distorted vector field can be uniquely decomposed into the following three components:
[0044]
[0045] in, is the non-rotational component perpendicular to the boundary, such as Figure 2e As shown, is the incompressible component parallel to the boundary, such as Figure 2f As shown, For the harmonic components, Figure 2g As shown, is the gradient operator, operator J is a two-dimensional vector rotated counterclockwise by π / 2. The divergence potential D and the curl potential R are obtained by solving the following Poisson equation, and the harmonic component is obtained by calculation ,
[0046]
[0047] Step (3), the divergence potential D, curl potential R and harmonic component obtained in step (2) Interpolation processing obtains a continuous and smooth divergence component , a continuous and smooth curl component and a continuous and smooth harmonic component , and estimate the two-dimensional distortion vector field of the structured light image by the continuous and smooth divergence component, the continuous and smooth curl component and the continuous and smooth harmonic component;
[0048] Step (4), perform image registration on the distorted structured light image and output the restored structured light image:
[0049] The two-dimensional distortion vector field of the structured light image estimated in step (3) is used to solve the spatial transformation model between the distorted structured light image and the reference structured light image:
[0050]
[0051] Among them, (U ij ,V ij ) is the pixel coordinate of the distorted structured light image, (u ij ,v ij ) is the pixel coordinate of the reference structured light image, i represents the row value of the image, and j represents the column value of the image; the restored structured light image is then obtained by image registration, such as Figure 2h As shown;
[0052] Step (5), perform image registration on the distorted target scene image and output the restored target scene image:
[0053] Based on the spatial transformation model of the distorted structured light image and the reference structured light image in step 4), the spatial transformation model of the distorted target scene image and the restored target scene image is further solved.
[0054]
[0055] in, is the pixel coordinate of the distorted target scene image, is the pixel coordinate of the restored target scene image, i represents the row value of the image, j represents the column value of the image, and λ is the physical quantity of the displacement ratio relationship between the reflected and refracted light; the restored target scene image is then obtained through image registration, such as Figure 2i After the above steps, the underwater distorted image can be restored through a distorted image containing wave surface information.
[0056] The preferred embodiments of the present invention disclosed above are only to help explain the present invention and are not intended to limit the present invention to the specific embodiments described. Obviously, many modifications and variations can be made according to the contents of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can better understand and use the present invention.
Claims
1. An image restoration method based on Helmholtz-Hodge decomposition, It is characterized in that The steps include: Step 1), projecting an adaptively adjustable reference structured light image onto the surface of the fluid, distorting the structured light due to the fluctuation of the fluid surface to obtain a distorted structured light image, and simultaneously obtaining a distorted target scene image through the same wavefront, extracting feature points from the distorted structured light image and the reference structured light image, and obtaining a two-dimensional distorted vector field of structured light feature points; Step 2), perform Helmholtz-Hodge decomposition on the two-dimensional distortion vector field that causes image distortion to obtain the divergence potential D, the curl potential R and the harmonic component In step 2), the two-dimensional distorted vector field Perform Helmholtz-Hodge decomposition to distort the vector field in two dimensions Its divergence and curl And when the boundary conditions are known, any two-dimensional distorted vector field can be uniquely decomposed into the following three components: in, is the non-rotational component perpendicular to the boundary, is the incompressible component parallel to the boundary, To balance the weight, is the gradient operator, operator J is a two-dimensional vector rotated counterclockwise by π / 2. The divergence potential D and the curl potential R are obtained by solving the following Poisson equation, and the harmonic component is obtained by calculation Step 3), the divergence potential D, curl potential R and harmonic components obtained in step 2) Interpolation processing obtains a continuous and smooth divergence component Continuously smoothed curl component and a continuous and smooth harmonic component The two-dimensional distortion vector field of the structured light image is estimated by the continuous smooth divergence component, the continuous smooth curl component and the continuous smooth harmonic component; Step 4), performing image registration on the distorted structured light image and outputting a restored structured light image; In step 4), the two-dimensional distortion vector field of the structured light image estimated in step 3) is used to solve the spatial transformation model between the distorted structured light image and the reference structured light image: Among them, (U ij ,V ij ) is the pixel coordinate of the distorted structured light image, (u ij ,v ij ) is the pixel coordinate of the reference structured light image, i represents the row value of the image, and j represents the column value of the image; and the restored structured light image is obtained by image registration; Step 5), performing image registration on the distorted target scene image, and outputting a restored target scene image; In step 5), the spatial transformation model of the distorted structured light image and the reference structured light image in step 4) is used to further solve the spatial transformation model of the distorted target scene image and the restored target scene image. in, is the pixel coordinate of the distorted target scene image, is the pixel coordinate of the restored target scene image, i represents the row value of the image, j represents the column value of the image, and λ is the physical quantity of the displacement proportional relationship between the reflected and refracted light; the restored target scene image is then obtained through image registration.
2. The image restoration method based on Helmholtz-Hodge decomposition according to claim 1, It is characterized in that In step 1), the two-dimensional distorted vector field The definition is as follows: Among them, (U ij ,V ij ) is the pixel coordinate of the distorted structured light image, (u ij ,v ij ) is the pixel coordinate of the reference structured light image, i represents the row value of the image, and j represents the column value of the image.
