Underwater distorted image restoration method based on nonlinear reconstruction and random image block optimization

Abstract

The invention discloses an underwater distorted image restoration method based on nonlinear reconstruction and random image block optimization. The method comprises the following steps: eliminating periodic distortion in a video sequence according to nonlinear reconstruction; random image block optimization is carried out on the video sequence after periodic distortion elimination; fusing and splicing the selected image blocks to form a complete image; and repeating the steps until a restored image sequence is obtained. According to the method, nonlinear reconstruction and random image block optimization are creatively fused into a whole, the method is extremely sensitive to obvious periodic distortion, and the details and definition of the image are improved by selecting and fusing the optimal block in the image, so that large-amplitude distortion in the image is restored, and the restoration effect of the underwater distorted image is greatly improved.

Description

Technical Field

[0001] The invention belongs to the field of machine vision, relates to image processing technology, and specifically relates to an underwater distorted image restoration method based on nonlinear reconstruction and random image block optimization. Background Art

[0002] AUVs are generally used for long-term, fully automatic exploration of submarine volcanic eruptions, seabed expansion, mantle extension, and other scientific research occasions. Due to uneven temperature, salinity changes, and external water flow disturbances, these occasions will form irregular water changes, affecting the uniform density distribution of the water body and causing underwater image distortion. In such extreme occasions, the traditional distorted image restoration algorithm has poor adaptability. In addition, the AUV's own computing power is limited, and it is difficult for the underwater environment to communicate with the surface platform at high speed in real time. It cannot rely on external supercomputing platforms. Researching an underwater distorted image restoration technology based on the underwater robot itself is extremely important for underwater target recognition, tracking, and positioning based on visual technology.

[0003] There are many types of algorithms for restoring underwater distorted images, such as:

[0004] (1) Restoring underwater distorted images using a three-dimensional model (water flow disturbance): This method captures the distortion of the image caused by water wave motion by capturing a sequence of images disturbed by waves, extracting the centroid trajectory in the image. Then, using synthetic imaging technology, the centroid evolution trajectory of each image point is calculated to compensate for the geometric deformation of the image caused by water surface fluctuations. Finally, the restored image is generated by interpolating and aligning these centroid positions. The advantage of this method is that it has a good processing effect on specific wave interference, but the restoration effect is poor when dealing with the real complex underwater environment when the AUV is working.

[0005] (2) Image registration algorithm to restore underwater distorted images: This algorithm first iteratively registers the distorted video sequence with the average frame of the video sequence to solve the distortion of the underwater image, and then uses the rank minimization method to extract sparse errors from the video sequence to further eliminate the residual sparse noise in the video sequence. However, this algorithm has a large amount of calculation and poor processing effect on severely distorted areas, and is not a lightweight algorithm suitable for application in AUVs.

[0006] (3) Deep learning to restore underwater distorted images: This algorithm uses pairs of distorted images and undistorted images as training samples, and uses a neural network to learn the spatial transformation relationship between the undistorted image and the distorted image, thereby restoring the underwater distorted image. Since this method achieves fast end-to-end output, it takes very little time to eliminate the distortion of underwater images. However, the deep learning-based model requires a large amount of training data and computing time, and the restoration effect is also extremely dependent on the quality of the training data, so it is difficult to achieve an ideal restoration effect. Summary of the invention

[0007] Purpose of the invention: In order to overcome the deficiencies in the prior art, a method for restoring underwater distorted images based on nonlinear reconstruction and random image block optimization is provided.

[0008] Technical solution: To achieve the above purpose, the present invention provides an underwater distorted image restoration method based on nonlinear reconstruction and random image block optimization, comprising the following steps:

[0009] S1: Eliminate periodic distortions in video sequences based on nonlinear reconstruction;

[0010] S2: Randomly optimize the image blocks of the video sequence after eliminating periodic distortion;

[0011] S3: fusing and splicing the selected image blocks to form a complete image;

[0012] S4: Repeat steps S2 and S3 until a restored image sequence is obtained.

[0013] Furthermore, the process of eliminating periodic distortion in the video sequence by nonlinear reconstruction in step S1 includes:

[0014] A1: Find characteristic pixels;

[0015] A2: Track the motion trajectory of feature pixels based on sparse reconstruction technology;

[0016] A3: Construct motion vector fields MVFs based on pixel offsets caused by periodic perturbations;

[0017] A4: Restore the periodic distortion of the image based on MVFs and complete the image reconstruction.

[0018] Furthermore, the step A1 is specifically as follows:

[0019] A1-1: Image Segmentation

[0020] According to the image changes caused by periodic underwater inertial oscillation, a motion vector field (MVFs) is established, which is a set of displacement vectors of each pixel in the video image sequence, and the motion vector field (MVFs) is sparse in discrete Fourier transform. Decompose the underwater distorted video shot by the camera forward and decompose it into an image sequence P in frames. d = {I d1 ,I d2 ,...,I dt}, I dt ∈Z h×w , I dtrepresents the t-th frame distorted image in the video image sequence, d is the image frame number, h and w are the height and width of the image; the undistorted image is I0, the distorted image I d The essence is to distort the undistorted image, so it can be expressed as:

[0021]

[0022] in: represents the gradient of I0 in the x direction, represents the gradient of I0 in the y direction, d x (x,y,t),d y (x, y, t) represents the displacement of pixel point (x, y) at frame t;

[0023] A1-2: Select feature points

[0024] Four algorithms, Harris corner detection, SURF corner detection, FAST algorithm and BRISK feature point detection, are used to extract feature points. Suppose the set of all significant feature points in the first frame of the video image sequence is Among them, (x a ,y a ) is the coordinate of the ath feature point, N is the number of feature points; the KLT optical flow tracking algorithm is used to monitor the selected feature points in subsequent frames And draw its changing trajectory Where t is the number of the current frame of the video image sequence, T is the total number of frames in the video image sequence, and a is the serial number of the feature point in each frame. The periodic external water flow disturbance will produce symmetrical traveling waves. Based on the Cox-Munk law, according to the motion trajectory of the feature point Q a Find the center of the motion trajectory. Since the traveling waves on both sides of the center point are symmetrical, the center point is the feature point with the least distortion or no distortion (x a0 ,y a0 ), whose value is:

[0025]

[0026] Among them, x at and at They represent the x-coordinate and y-coordinate of the a-th feature point in the t-th frame of the video image, respectively. The subscript 0 represents an undistorted image, that is, the position where each feature point has the smallest distortion or no distortion.

[0027] Furthermore, the step A2 includes: after finding the center of the motion trajectory of the feature point, setting the position as the reference point, and calculating the displacement of each feature point based on the reference point:

[0028]

[0029] Among them, d ax and d ay Respectively represent the displacement of the a-th feature point in the x-direction and the y-direction; then construct the three-dimensional motion vector field signal d with a time tag, which can be sparsely decomposed in the discrete Fourier transform domain, and the displacement trajectory In fact, it is a sparse sample in the spatiotemporal continuous domain; therefore, the displacement trajectory is connected It is defined as a measurement vector v containing NT elements, and the displacement trajectory error is obtained based on the sparse reconstruction theory.

[0030] Furthermore, the displacement trajectory error in step A2 is expressed as follows:

[0031]

[0032] Where ε is a measurement matrix of size NT × hwT, and formula (5) is an example of this matrix. 1 indicates sampling and 0 indicates unsampling. In formula (4), G is a three-dimensional Fourier transform basis matrix. When processing high-resolution three-dimensional data, it does not appear in an obvious matrix form, but is dynamically calculated through fast Fourier transform. η is a sparse coefficient vector. δ is the displacement trajectory error detected in the tracking algorithm, which is written into the measurement equation as a noise vector column in formula (4).

[0033] Furthermore, the step A3 comprises:

[0034] Minimizing the displacement trajectory error δ means suppressing the noise vector v as much as possible, so that the measurement vector v tends to zero to ensure that sparse reconstruction can better improve image clarity; minimizing the displacement trajectory error δ requires separating the measurement matrix ε and the three-dimensional Fourier transform basis matrix G so that there is a low linear correlation between the two; therefore, the coherence function μ(εG) is designed:

[0035]

[0036] Among them, θ a and F b Respectively represent the ath row of ε and the bth column of G, and a is not equal to b; introduce weights This makes the image pay more attention to row a and column b when restoring. When μ(εG) approaches 0, it means that the measurement matrix ε can capture more independent signal features and obtain better restoration effect.

[0037] When the MVFs estimate is consistent with the observed value, the actual feature point motion trajectory is also the same as the predicted trajectory. Here, the LASSO estimator is used to design the minimized objective function U(θ). The LASSO estimator can find an optimal solution θ that takes into account both minimization of the measurement vector v and sparsity under the influence of noise. The expression is as follows:

[0038]

[0039] in is the fidelity term, minimizing this term avoids overfitting; is the regularization parameter; in order to calculate the optimal solution, it is necessary to convert the solution of the sparse coefficient vector into l1 norm optimization to maintain sparsity, and use the YALL1 solver to solve the motion vector field MVFs of the video pixel points and the distortion process of the pixel points;

[0040] Furthermore, the step A4 comprises:

[0041] On this basis, the periodic distortion is eliminated by reverse tracking and restoration calculation of the above feature points, and finally the video image sequence Vf = {I f1 ,I f2 ,...,I fN}, where Vf∈Z h×w×T .

[0042] Furthermore, the step S2 comprises:

[0043] Lucky block selection: In the same area of ​​each frame of the video sequence, the image is divided into several blocks, and there is a 50% overlap between adjacent image blocks; according to the structural similarity index SSIM, severely distorted image blocks are selected and discarded from the image block sequence; SSIM evaluates image features from three aspects: the impact of brightness, contrast and structure on the image, and its formula is as follows:

[0044]

[0045] Among them, μ I , μ M are the means of image blocks I and M respectively; ω I ,ω M is the brightness weight; σ I , σ M are the variances of image blocks I and M respectively; σ IMCovariance of the image blocks I and M; C1 and C2 are constants added to prevent instability when the denominator approaches zero; the structural similarity value satisfies SSIM ∈ (0, 1], and the closer the SSIM value is to 1, the more similar the two pictures are; for the image blocks in the same region, a feature vector group is constructed based on the structural similarity SSIM value for K-means++ clustering, and the Euclidean distance is used as the distance measure. Set the number of clustering clusters k = 2, that is, there are two types of clustering clusters: the group of severely distorted image blocks and the group of slightly distorted image blocks; select the cluster with a higher average SSIM value (the cluster with slightly distorted images) as the "lucky block", calculate the average image block of its sequence, and then place all the average blocks in their true positions to form a single overlapping image.

[0046] Further, the step S3 includes:

[0047] Lucky block splicing: To avoid obvious seams at the edges of the image blocks due to boundary effects when splicing the image blocks, the weighted smoothing splicing method is used to splice each part of the image blocks, and the splicing formula is as follows:

[0048]

[0049] where c is the offset coefficient, which can dynamically adjust the splicing offset according to the difference of the image blocks to make the splicing process more natural; P is the pixel value after overlapping; i / d is the image gradient coefficient, i (0 < i < d - 1) is the column serial number corresponding to the overlapping area, d is the width of the image overlapping area, so the value range of i / d is 0 < b < 1; L P is the value of a certain pixel point in the left area of the image block; R P is the value of the mirror pixel point corresponding to L in the right area of the image block; formula (9) uses the method of linear weighting to weight the pixel values in the overlapping area according to the gradient coefficient i / d. The closer the value of i / d is to 1, the closer the spliced pixel value is to the left image block; when the value of i / d is close to 0, the spliced pixel value is closer to the right image block; in this way, the pixel values of different image blocks are gradually transitioned, thus avoiding the abrupt splicing effect and ensuring the smooth transition of the spliced image with better visual effects.

[0050] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0051] 1. The present invention innovatively integrates non-linear reconstruction and random image block optimization into a whole. This method is extremely sensitive to obvious periodic distortions, and by selecting and fusing the best blocks in the image to improve the image details and clarity, the large-scale distortions in the image are restored, greatly improving the lower limit of the image restoration effect.

[0052] ​​2. By comparing and selecting multiple frame image blocks in the same area of ​​the image, sparse lucky reconstruction effectively removes local random noise, making the image restoration process more stable and smooth.

[0053] 3. Since the present invention adopts sparse representation and fast image block stitching method, its calculation amount is relatively concentrated and the processing speed is relatively fast, which is very suitable for AUV working scenarios with extremely high real-time requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is the overall flow chart of the method of the present invention;

[0055] Figure 2 Flowchart for nonlinear reconstruction;

[0056] Figure 3 Flowchart for optimizing random image patches;

[0057] Figure 4 The figure shows the video image sequence obtained after using nonlinear reconstruction and random image block optimization. DETAILED DESCRIPTION

[0058] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent forms of modifications to the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0059] The present invention provides a method for restoring underwater distorted images based on nonlinear reconstruction and random image block optimization. Figure 1 As shown, it includes the following steps:

[0060] S1: Eliminate periodic distortions in video sequences based on nonlinear reconstruction;

[0061] Reference Figure 2 ,The process of nonlinear reconstruction to eliminate periodic distortions in video sequences includes:

[0062] A1: Find characteristic pixels;

[0063] Specifically:

[0064] A1-1: Image Segmentation

[0065] Decompose the underwater distorted video shot by the camera in the forward direction and decompose it into an image sequence P in frames. d = {I d1 ,I d2 ,...,I dt}, I dt ∈Z h×w , Idt represents the t-th frame distorted image in the video image sequence, d is the image frame number, h and w are the height and width of the image; the undistorted image is I0, the distorted image I d The essence is to distort the undistorted image, so it can be expressed as:

[0066]

[0067] in: represents the gradient of I0 in the x direction, represents the gradient of I0 in the y direction, d x (x,y,t),d y (x, y, t) represents the displacement of pixel point (x, y) at frame t;

[0068] A1-2: Select feature points

[0069] In order to select all pixels with periodic distortion, the present invention uses Harris corner detection, SURF corner detection, FAST algorithm and BRISK feature point detection to extract feature points. Suppose the set of all significant feature points in the first frame of the video image sequence is Among them, (x a ,y a ) is the coordinate of the ath feature point, N is the number of feature points; the KLT optical flow tracking algorithm is used to monitor the selected feature points in subsequent frames And draw its changing trajectory Where t is the number of the current frame of the video image sequence, T is the total number of frames in the video image sequence, and a is the serial number of the feature point in each frame. The periodic external water flow disturbance will produce symmetrical traveling waves. Based on the Cox-Munk law, according to the motion trajectory of the feature point Q a Find the center of the motion trajectory. Since the traveling waves on both sides of the center point are symmetrical, the center point is the feature point with the least distortion or no distortion (x a0 ,y a0 ), whose value is:

[0070]

[0071] Among them, x at and at They represent the x-coordinate and y-coordinate of the a-th feature point in the t-th frame of the video image, respectively. The subscript 0 represents an undistorted image, that is, the position where each feature point has the smallest distortion or no distortion.

[0072] A2: Track the motion trajectory of feature pixels based on sparse reconstruction technology;

[0073] After finding the center of the feature point's motion trajectory, set that position as the reference point and calculate the displacement of each feature point based on it:

[0074]

[0075] Among them, d ax and d ay Respectively represent the displacement of the a-th feature point in the x-direction and the y-direction; then construct the three-dimensional motion vector field signal d with a time tag, which can be sparsely decomposed in the discrete Fourier transform domain, and the displacement trajectory In fact, it is a sparse sample in the spatiotemporal continuous domain; therefore, the displacement trajectory is connected It is defined as a measurement vector v containing NT elements, and the displacement trajectory error is obtained according to the sparse reconstruction theory, which is expressed as follows:

[0076]

[0077] Where ε is a measurement matrix of size NT × hwT, and formula (5) is an example of this matrix. 1 indicates sampling and 0 indicates unsampling. In formula (4), G is a three-dimensional Fourier transform basis matrix. When processing high-resolution three-dimensional data, it does not appear in an obvious matrix form, but is dynamically calculated through fast Fourier transform. η is a sparse coefficient vector. δ is the displacement trajectory error detected in the tracking algorithm, which is written into the measurement equation as a noise vector column in formula (4).

[0078] A3: Construct motion vector fields MVFs based on pixel offsets caused by periodic perturbations;

[0079] According to the image changes caused by periodic underwater inertial oscillations, motion vector fields (MVFs) are established, which are the set of displacement vectors of each pixel in the video image sequence. Motion vector fields (MVFs) are sparse in discrete Fourier transform.

[0080] Minimizing the displacement trajectory error δ means suppressing the noise vector v as much as possible, so that the measurement vector v tends to zero to ensure that sparse reconstruction can better improve image clarity; minimizing the displacement trajectory error δ requires separating the measurement matrix ε and the three-dimensional Fourier transform basis matrix G so that there is a low linear correlation between the two; therefore, the coherence function μ(εG) is designed:

[0081]

[0082] Among them, θ a and F b Respectively represent the ath row of ε and the bth column of G, and a is not equal to b; introduce weights This makes the image pay more attention to row a and column b when restoring. When μ(εG) approaches 0, it means that the measurement matrix ε can capture more independent signal features and obtain better restoration effect.

[0083] When the MVFs estimate is consistent with the observed value, the actual feature point motion trajectory is also the same as the predicted trajectory. Here, the LASSO estimator is used to design the minimized objective function U(θ). The LASSO estimator can find an optimal solution θ that takes into account both minimization of the measurement vector v and sparsity under the influence of noise. The expression is as follows:

[0084]

[0085] in is the fidelity term, minimizing this term avoids overfitting; is the regularization parameter; in order to calculate the optimal solution, it is necessary to convert the solution of the sparse coefficient vector into l1 norm optimization to maintain sparsity, and use the YALL1 solver to solve the motion vector field MVFs of the video pixel points and the distortion process of the pixel points;

[0086] A4: Restore the periodic distortion of the image based on MVFs and complete the image reconstruction:

[0087] On this basis, the periodic distortion is eliminated by reverse tracking and restoration calculation of the above feature points, and finally the video image sequence Vf = {I f1 ,I f2 ,...,I fN}, where Vf∈Z h×w×T .

[0088] S2: Randomly optimize the image blocks of the video sequence after eliminating periodic distortion to eliminate local noise;

[0089] Reference Figure 3 ,Lucky block selection: In the same area of ​​each frame of the video sequence, the image is divided into several blocks, and there is a 50% overlap between adjacent image blocks; According to the structural similarity index SSIM, severely distorted image blocks are selected and discarded from the image block sequence; SSIM evaluates image features from three aspects: the impact of brightness, contrast and structure on the image, and its formula is as follows:

[0090]

[0091] Among them, μ I , μ M are the means of image blocks I and M respectively; ω I ,ω M is the brightness weight; σ I , σ MThey are the variances of the image blocks I and M respectively; σ IM is the covariance of the image blocks I and M; C1 and C2 are constants added to prevent instability when the denominator approaches zero; the structural similarity value satisfies SSIM ∈ (0, 1], and the closer the SSIM value is to 1, the more similar the two pictures are; for the image blocks in the same region, a feature vector group is constructed with the structural similarity SSIM value and K-means++ clustering is performed, and the Euclidean distance is used as the distance measure. Set the number of clustering clusters k = 2, that is, there are two types of clustering clusters: severely distorted image block groups and low-distortion image block groups; select the cluster with a higher average SSIM value (the clustering cluster with slight image distortion) as the "lucky block", calculate the average image block of its sequence, and then place all the average blocks in their true positions to form a single overlapping image.

[0092] S3: Fuse and splice the selected image blocks to form a complete image;

[0093] Refer to Figure 3 , lucky block splicing: In order to avoid obvious seams at the edges of the image blocks due to boundary effects when splicing the image blocks, the weighted smoothing splicing method is used to splice each part of the image blocks, and the splicing formula is as follows:

[0094]

[0095] Among them, c is the offset coefficient, which can dynamically adjust the splicing offset according to the difference of the image blocks to make the splicing process more natural; P is the pixel value after overlapping; i / d is the image gradual change coefficient, i (0 < i < d - 1) is the column serial number corresponding to the overlapping area, and d is the width of the image overlapping area, so the value range of i / d is 0 < b < 1; L P is the value of a certain pixel point in the left area of the image block; R P is the value of the mirror pixel point corresponding to L P in the right area of the image block; Formula (9) uses the method of linear weighting to weight the pixel values in the overlapping area according to the gradient coefficient i / d. The closer the value of i / d is to 1, the closer the spliced pixel value is to the left image block; when the value of i / d is close to 0, the spliced pixel value is closer to the right image block; in this way, the pixel values of different image blocks are gradually transitioned, thus avoiding an abrupt splicing effect and ensuring a smooth transition of the spliced image with a better visual effect.

[0096] S4: Repeat steps S2 and S3 until the restored image sequence is obtained.

[0097] In order to verify the effectiveness and actual effect of the method of the present invention, this embodiment conducts an example application and restoration performance analysis of the method of the present invention, specifically as follows:

[0098] Such as Figure 4The figure shows the video image sequence obtained after nonlinear reconstruction and random image block optimization. From the perspective of visual perception, there are a large number of distorted areas in the initial video image sequence frames. After restoration, the degree of distortion is significantly reduced, most pixels are closer to the real position, and the severe blur at the edge of the image is also eliminated.

[0099] Therefore, the effectiveness and practical effect of the method of the present invention are verified.

Claims

1. A method for restoring underwater distorted images based on nonlinear reconstruction and random image block optimization, characterized in that: The steps include: S1: Eliminate periodic distortions in video sequences based on nonlinear reconstruction; S2: Randomly optimize the image blocks of the video sequence after eliminating periodic distortion; S3: fusing and splicing the selected image blocks to form a complete image; S4: Repeat steps S2 and S3 until a restored image sequence is obtained.

2. The underwater distorted image restoration method based on nonlinear reconstruction and random image block optimization according to claim 1, characterized in that: The process of eliminating periodic distortion in the video sequence by nonlinear reconstruction in step S1 includes: A1: Find characteristic pixels; A2: Track the motion trajectory of feature pixels based on sparse reconstruction technology; A3: Construct motion vector fields MVFs based on pixel offsets caused by periodic perturbations; A4: Restore the periodic distortion of the image based on MVFs and complete the image reconstruction.

3. The underwater distorted image restoration method based on nonlinear reconstruction and random image block optimization according to claim 2, characterized in that: The step A1 is specifically as follows: A1-1: Image Segmentation Decompose the underwater distorted video shot by the camera in the forward direction and decompose it into an image sequence P in frames. d = {I d1 ,I d2 ,...,I dt }, I dt ∈Z h×w , I dt represents the t-th frame distorted image in the video image sequence, d is the image frame number, h and w are the height and width of the image; the undistorted image is I0, the distorted image I d The essence is to distort the undistorted image, so it can be expressed as: in: represents the gradient of I0 in the x direction, represents the gradient of I0 in the y direction, d x (x,y,t),d y (x, y, t) represents the displacement of pixel point (x, y) at frame t; A1-2: Select feature points Four algorithms, Harris corner detection, SURF corner detection, FAST algorithm and BRISK feature point detection, are used to extract feature points. Suppose the set of all significant feature points in the first frame of the video image sequence is Among them, (x a ,y a ) is the coordinate of the ath feature point, N is the number of feature points; the KLT optical flow tracking algorithm is used to monitor the selected feature points in subsequent frames And draw its changing trajectory Where t is the number of the current frame of the video image sequence, T is the total number of frames in the video image sequence, and a is the serial number of the feature point in each frame. The periodic external water flow disturbance will produce symmetrical traveling waves. Based on the Cox-Munk law, according to the motion trajectory of the feature point Q a Find the center of the motion trajectory. Since the traveling waves on both sides of the center point are symmetrical, the center point is the feature point with the least distortion or no distortion (x a0 ,y a0 ), whose value is: Among them, x at and at They represent the x-coordinate and y-coordinate of the a-th feature point in the t-th frame of the video image, respectively. The subscript 0 represents an undistorted image, that is, the position where each feature point has the smallest distortion or no distortion.

4. The underwater distorted image restoration method based on nonlinear reconstruction and random image block optimization according to claim 2, characterized in that: The step A2 comprises: after finding the center of the motion trajectory of the feature point, setting the position as a reference point, and calculating the displacement of each feature point based on the reference point: Among them, d ax and d ay Respectively represent the displacement of the a-th feature point in the x-direction and the y-direction; then construct the three-dimensional motion vector field signal d with a time tag, which can be sparsely decomposed in the discrete Fourier transform domain, and the displacement trajectory In fact, it is a sparse sample in the spatiotemporal continuous domain; therefore, the displacement trajectory is connected It is defined as a measurement vector v containing NT elements, and the displacement trajectory error is obtained based on the sparse reconstruction theory.

5. The underwater distorted image restoration method based on nonlinear reconstruction and random image block optimization according to claim 4, characterized in that: The displacement trajectory error in step A2 is expressed as follows: Where ε is a measurement matrix of size NT × hwT, and formula (5) is an example of this matrix. 1 indicates sampling and 0 indicates unsampling. In formula (4), G is a three-dimensional Fourier transform basis matrix. When processing high-resolution three-dimensional data, it does not appear in an obvious matrix form, but is dynamically calculated through fast Fourier transform. η is a sparse coefficient vector. δ is the displacement trajectory error detected in the tracking algorithm, which is written into the measurement equation as a noise vector column in formula (4).

6. The underwater distorted image restoration method based on nonlinear reconstruction and random image block optimization according to claim 5, characterized in that: The step A3 comprises: Minimizing the displacement trajectory error δ means suppressing the noise vector v as much as possible, so that the measurement vector v tends to zero to ensure that sparse reconstruction can better improve image clarity; minimizing the displacement trajectory error δ requires separating the measurement matrix ε and the three-dimensional Fourier transform basis matrix G so that there is a low linear correlation between the two; therefore, the coherence function μ(εG) is designed: Among them, θ a and F b Respectively represent the ath row of ε and the bth column of G, and a is not equal to b; introduce weights This makes the image pay more attention to row a and column b when restoring. When μ(εG) approaches 0, it means that the measurement matrix ε can capture more independent signal features and obtain better restoration effect. Using the LASSO estimator, the objective function U(θ) is designed to be minimized. The LASSO estimator can find an optimal solution θ that takes into account both minimization of the measurement vector v and sparsity under the influence of noise. The expression is as follows: in It is the fidelity item; is the regularization parameter; in order to calculate the optimal solution, it is necessary to convert the solution of the sparse coefficient vector into l1 norm optimization to maintain sparsity, and use the YALL1 solver to solve the motion vector field MVFs of the video pixel points and the distortion process of the pixel points.

7. The underwater distorted image restoration method based on nonlinear reconstruction and random image block optimization according to claim 6, characterized in that: The step A4 comprises: Based on step A3, the periodic distortion is eliminated by reverse tracking and restoration calculation of the above feature points, and finally the video image sequence Vf={I f1 ,I f2 ,...,I fN }, where Vf∈Z h×w×T .

8. The underwater distorted image restoration method based on nonlinear reconstruction and random image block optimization according to claim 1, characterized in that: The step S2 comprises: In the same area of ​​each frame of the video sequence, the image is divided into several blocks, and there is a 50% overlap between adjacent image blocks; according to the structural similarity index SSIM, severely distorted image blocks are selected and discarded from the image block sequence; SSIM evaluates image features from three aspects: brightness, contrast, and the impact of structure on the image, and its formula is as follows: Among them, μ I , μ M are the means of image blocks I and M respectively; ω I ,ω M is the brightness weight; σ I , σ M are the variances of image blocks I and M respectively; σ IM is the covariance of image blocks I and M; C1 and C2 are constants added to prevent instability when the denominator is close to zero; the structural similarity value satisfies SSIM∈(0,1]; K-means++ clustering is performed on the feature vector group constructed by the structural similarity SSIM value of the image blocks in the same region, and the Euclidean distance is used as the distance measure. The type of cluster k is set to 2, that is, there are two types of clusters: severely distorted image block groups and low-distorted image block groups; the cluster with a higher average SSIM value is taken as the "lucky block", the average image block of its sequence is calculated, and then all the average blocks are placed at their true positions to form a single overlapping image.

9. The method for restoring underwater distorted images based on nonlinear reconstruction and random image block optimization according to claim 8, characterized in that: The step S3 comprises: The weighted smoothing stitching method is used to stitch the image blocks together. The stitching formula is as follows: Among them, c is the offset coefficient; P is the pixel value after overlapping; i / d is the image gradient coefficient, where i (0 < i < d - 1) is the column number corresponding to the overlapping area, and d is the width of the image overlapping area. Therefore, the value range of i / d is 0 < b < 1; L P is the value of a certain pixel in the left area of the image block; R P is the value of the mirror pixel corresponding to the right area of the image block and L P ; Formula (9) uses the method of linear weighting to weight the pixel values in the overlapping area according to the gradient coefficient i / d.

Images