A method and system for turbulence repair based on sobel gradient and multi-frame fusion

Abstract

The embodiment of the application provides a kind of turbulent flow repair method and system based on Sobel gradient and multi-frame fusion, turbulent flow is repaired based on Sobel gradient and Laplace operator, image is operated by Sobel gradient to remove blur, while the edge oscillation phenomenon caused by turbulent flow is iteratively repaired by adding the Laplace operator in time domain, the main process of fusion is to fuse the best part of the turbulent flow repair effect of multiple frames together, to obtain more stable turbulent flow repair effect, this process can also be omitted, and the last frame image of turbulent flow repair is directly output to improve the algorithm running speed, the calculation process is simple, and the calculation speed is fast, which can ensure real-time repair of turbulent flow video.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of imaging processing, and particularly relate to a turbulence repair method and system based on Sobel gradient and multi-frame fusion. BACKGROUND

[0002] Before light enters the detector, the atmospheric medium will affect or interfere with the ideal propagation of light, causing the wavefront to be distorted, resulting in diffusion of the intensity distribution of the image point on the focal plane, reduction of the peak value, and beam drift, etc. In an infrared imaging system, when the air temperature is high, the turbulence phenomenon is very obvious. Mainly due to the different refractive indexes of different turbulence blocks in the air, the infrared received by the detector is distorted, mainly manifested as the distortion of the edge of the object and the twisting of the target in the image. This phenomenon will have a great negative impact on the detection, identification and tracking of the infrared camera. The current mainstream turbulence repair algorithm has high time complexity and complex calculation process, and it is difficult to repair turbulence in real-time video. SUMMARY

[0003] Embodiments of the present application provide a turbulence repair method and system based on Sobel gradient and multi-frame fusion, which fuses multi-sensor information based on perception data, and realizes the generation and updating of a local sea chart using perception data.

[0004] In a first aspect, embodiments of the present application provide a turbulence repair method based on Sobel gradient and multi-frame fusion, comprising:

[0005] Step S1, acquiring FrameNum-1 frames of images before a current frame in a video stream, wherein FrameNum is a preset frame number of images required for image fusion;

[0006] Step S2, performing turbulence repair on the current frame and the FrameNum-1 frames of images before the current frame;

[0007] Step S3, fusing the images after turbulence repair to obtain a fused single-frame image, and taking the fused single-frame image as a turbulence repair result of the current frame.

[0008] As a preferred, the step S1 specifically comprises:

[0009] Step S11, determining a frame number T of the current frame input by the video stream, and determining a frame number FrameNum required for fusion;

[0010] Step S12, if it is judged that T<FrameNum, taking the current frame as input without performing turbulence repair;

[0011] Step S13, if it is judged that T≥FrameNum, taking images from the T-FrameNum+1 frame to the i-th frame in the video stream.

[0012] As preferred, the step S2 specifically comprises:

[0013] Step S21, determining the input image sequence as I k , I k contains K images of HxW, where k∈{1,2,3,…,K}, H is the image height, W is the image width, and K equals FrameNum;

[0014] Calculating the coefficient matrix D, the position of any point in the coefficient matrix D is i∈{1,2,3,…,H-1}, j∈{1,2,3,…,W-1}, and each point D (i,j) in the coefficient matrix is:

[0015] D (i,j) = sin(π×i÷H) 2 + sin(π×j÷W) 2 ;

[0016] Step S22, traversing all frame images, calculating the discrete Fourier transform of each frame image, obtaining the image sequence after Fourier transform, and calculating the sum of Sobel gradients of each frame image in the image sequence as the iteration initial condition;

[0017] Step S23, starting iteration, calculating the metric value under the current iteration number, and updating the Fourier transform result; if it is judged that the iteration number is reached, performing inverse Fourier transform on the last updated Fourier transform result, and outputting the inverse Fourier transform result of all frame images as the turbulence repair result.

[0018] As preferred, the step S22 specifically comprises:

[0019] Traversing all frame images in the image sequence, calculating the discrete Fourier transform of each frame image, and obtaining the image sequence after discrete Fourier transform; wherein the real part of the image sequence after discrete Fourier transform is and the imaginary part is

[0020] Calculating the sum of Sobel gradients of the kth frame image in the image sequence:

[0021]

[0022] As preferred, the step S23 specifically comprises:

[0023] Calculating the sum of Sobel gradients normGradu k under the current iteration number t, and the image update coefficient const k , k∈{1,2,3,…,K}, wherein the const of the kth frame image isk is:

[0024]

[0025] determine the frequency domain image of the current iteration number t based on the image update coefficient and use the updated frequency domain image calculate normGradu for the next iteration k and const k ; traverse each frame image, and the image update calculation formula is as follows:

[0026]

[0027] In the above formula, both α and λ are adjustable parameters, and μ is fixedly set to 1;

[0028] When the first iteration is performed, normGradu k = normGrad0 k , and the iteration process will continue until the set maximum iteration number is reached.

[0029] As preferred, in the step S22, the discrete Fourier transform is a two-dimensional discrete Fourier transform, which can be obtained by first performing one-dimensional discrete Fourier transform on the image row by row and then performing one-dimensional discrete Fourier transform on the obtained results column by column;

[0030] In the step S23, the Fourier transform result of the last update is then subjected to inverse Fourier transform, specifically including:

[0031]

[0032]

[0033] As preferred, in the step S3, the image after turbulence repair is fused, specifically including:

[0034] The mean image I mean of all frame images is calculated according to the following formula:

[0035]

[0036] The Sobel gradient image of all frame images is calculated according to the following formula:

[0037] The boundary of all frame images is expanded by one pixel, and the expanded pixel value is equal to the pixel value at the image boundary; traverse each point, and the Sobel gradient image calculation formula of the kth frame image is as follows:

[0038]

[0039]

[0040]

[0041] Square all the gray value differences between the frame images and the reference image, denoted as Traverse each point, and the gray value difference between the kth frame image and the reference image is The calculation formula is as follows:

[0042]

[0043] Square all the gray value differences between the frame images and the reference image, denoted as and perform mean filtering;

[0044] Calculate the mixing weight map of all frames Traverse each point, and the gray value difference between the kth frame image and the reference image is The calculation formula is as follows:

[0045]

[0046] Traverse each point, and calculate the weighted average in the time domain to obtain the final fusion image I fusion The calculation formula is as follows:

[0047]

[0048] In a second aspect, an embodiment of the present application provides a turbulence repairing system based on Sobel gradient and multi-frame fusion, comprising:

[0049] A frame extraction module acquires FrameNum-1 frame images before a current frame in a video stream, wherein FrameNum is a preset frame number of images required for image fusion.

[0050] A turbulence repairing module performs turbulence repairing on the current frame and the FrameNum-1 frame images before the current frame.

[0051] An image synthesis module fuses the images after turbulence repairing to obtain a fused single frame image, which is taken as a turbulence repairing result of the current frame.

[0052] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor implements the steps of the turbulence repairing method based on Sobel gradient and multi-frame fusion according to the first aspect of the present application.

[0053] In a fourth aspect, an embodiment of the present application provides a non-transitory computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the method for repairing turbulence based on Sobel gradient and multi-frame fusion according to the first aspect of the present application.

[0054] The method and system for repairing turbulence based on Sobel gradient and multi-frame fusion provided by the embodiment of the present application repair turbulence based on Sobel gradient and Laplace operator. The Sobel gradient is used to perform a deblurring operation on an image, and a Laplace operator in the time domain is used to iteratively repair edge oscillation caused by turbulence. The fusion process mainly fuses the best parts of the multi-frame turbulence repair effects to obtain a more stable turbulence repair effect. The process can be omitted, and the last frame of the turbulence repair image is directly output to improve the running speed of the algorithm. The calculation process is simple, the calculation speed is fast, and the real-time repair of the turbulence video can be ensured. BRIEF DESCRIPTION OF DRAWINGS

[0055] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0056] Figure 1 The flow chart of the method for repairing turbulence based on Sobel gradient and multi-frame fusion according to the embodiment of the present application is shown in FIG. 1.

[0057] Figure 2 The flow chart of the method for repairing turbulence based on Sobel gradient and multi-frame fusion according to the embodiment of the present application is shown in FIG. 1.

[0058] Figure 3 The flow chart of the method for repairing turbulence based on Sobel gradient and multi-frame fusion according to the embodiment of the present application is shown in FIG. 1.

[0059] Figure 4 The flow chart of the method for repairing turbulence based on Sobel gradient and multi-frame fusion according to the embodiment of the present application is shown in FIG. 1.

[0060] Figure 5 The flow chart of the method for repairing turbulence based on Sobel gradient and multi-frame fusion according to the embodiment of the present application is shown in FIG. 1. DETAILED DESCRIPTION

[0061] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0062] The term "and / or" in the embodiments of the present application merely describes an association relationship of associated objects, and indicates that there can be three relationships, for example, A and / or B can represent the following three cases: A exists alone, A and B exist together, and B exists alone.

[0063] The terms "first" and "second" in the embodiments of the present application are used only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first" and "second" can explicitly or implicitly include at least one of the features. In the description of the present application, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a system, product or device including a series of components or units is not limited to the listed components or units, but can optionally include components or units not listed or can optionally include other components or units inherent to these products or devices. In the description of the present application, the meaning of "a plurality of" is at least two, for example, two, three, etc., unless otherwise explicitly specified.

[0064] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The phrase appears at various places in the specification does not necessarily refer to the same embodiment, nor is it necessarily mutually exclusive of other embodiments. It is explicitly and implicitly understood that the embodiments described herein are capable of combination and that individual features, structures, or characteristics from each of the embodiments can be combined.

[0065] Before the light enters the detector, the atmospheric medium will affect or interfere with the ideal propagation of light, causing the wave front to be distorted, forming a turbulent effect of diffusing the intensity distribution of the image point on the focal plane, reducing the peak value, and drifting the light beam. In an infrared imaging system, when the air temperature is high, the turbulent flow phenomenon is very obvious. Due to the different refractive indexes of different turbulent blocks in the air, the infrared received by the detector is distorted, mainly manifested as the distortion of the edge of the object and the twisting of the target in the image. This phenomenon will have a great negative impact on the detection, identification and tracking of the infrared camera. The current mainstream turbulent flow repair algorithm has high time complexity and complex calculation process, and it is difficult to repair the turbulent flow of real-time video.

[0066] Therefore, this invention provides a turbulence restoration method and system based on Sobol gradient and multi-frame fusion, which can achieve real-time turbulence video restoration. The algorithm mainly consists of two steps: turbulence restoration and multi-frame fusion. The Sobol gradient is used to deblur the image, and a temporal Laplacian operator is added to iteratively repair edge oscillations caused by turbulence. The fusion process mainly involves combining the best parts of the turbulence restoration effects from multiple frames to obtain a more stable turbulence restoration effect. The following will elaborate and describe this through several embodiments.

[0067] Figure 1 This invention provides a turbulence repair method based on Sobol gradient and multi-frame fusion, comprising:

[0068] Step S1: Obtain the FrameNum-1 frame images preceding the current frame in the video stream, where FrameNum is the preset number of frames required for image fusion;

[0069] In this embodiment, as Figure 2 As shown, step S1 specifically includes:

[0070] Step S11: Determine the frame number T of the current frame of the video stream input, and determine the number of frames required for fusion as FrameNum;

[0071] Step S12: If it is determined that T < FrameNum, then the current frame is used as input and turbulence repair is not performed;

[0072] Step S13: If it is determined that T≥FrameNum, then take the images from frame T-FrameNum+1 to frame i in the video stream.

[0073] Step S2: Perform turbulence restoration on the current frame and the FrameNum-1 frames before the current frame to obtain a total of FrameNum frames with turbulence restoration.

[0074] In this embodiment, as Figure 3 As shown, the image is deblurred using the Sobol gradient, while a temporal Laplacian operator is used iteratively to repair edge oscillations caused by turbulence. The turbulence repair algorithm is as follows:

[0075] Step S21: Determine the input image sequence as I k I k It contains K H×W images, where k∈{1,2,3,…,K}, H is the image height, W is the image width, and K is equal to FrameNum;

[0076] Calculate the coefficient matrix D, which is a matrix of the same size as the image, and the value of each element in the matrix depends only on its position in the image. For any point in the coefficient matrix D, the position is i∈{1,2,3,…,H-1}, j∈{1,2,3,…,W-1}, and each point in the coefficient matrix D... (i,j) for:

[0077] D (i,j) =sin(π×i÷H) 2 +sin(π×j÷W) 2 ;

[0078] Step S22: Traverse all frame images, calculate the discrete Fourier transform of each frame image, obtain the image sequence after Fourier transform, and calculate the sum of the Sobol gradients of each frame image in the image sequence as the initial condition for iteration.

[0079] In this embodiment, all frame images are traversed, and the two-dimensional discrete Fourier transform of each frame image is calculated to obtain the image sequence after the two-dimensional discrete Fourier transform; wherein, the image sequence after the two-dimensional discrete Fourier transform is a complex number, and its real part is... The imaginary part is

[0080] In this embodiment, the two-dimensional discrete Fourier transform can be obtained by first performing a one-dimensional discrete Fourier transform on each row of the image, and then performing a one-dimensional discrete Fourier transform on each column of the result. Assume a one-dimensional vector of length N has the following real part: The imaginary part is X In For n∈{1,2,3,…,N-1}, after the Discrete Fourier Transform, the result of the real part is: The result of the imaginary part is x Rn If n∈{1,2,3,…,N-1}, then the calculation process of the one-dimensional discrete Fourier transform can be described as follows:

[0081]

[0082]

[0083] When performing a Fourier transform on an image, the input vector is a real number when performing a one-dimensional Fourier transform row by row. If the value is 0, no calculation is needed.

[0084] Calculate the sum of the Sobol gradients in the frequency domain for all frames of the image, normGrad0. k As the initial iteration condition, we iterate through each frame of the image sequence and calculate the sum of the Sobol gradients of the k-th frame:

[0085]

[0086] Step S23: Start iteration, calculate the metric value at the current iteration number, and update the Fourier transform result; if it is determined that the iteration number has been reached, perform an inverse Fourier transform on the last updated Fourier transform result, and output the inverse Fourier transform results of all frame images as the turbulence repair result.

[0087] In this embodiment, after the iteration begins, the sum of the Sobol gradients, normGradu, at the current iteration number t is calculated. k and image update coefficients const k , k∈{1,2,3,…,K}, where the const of the k-th frame image k for:

[0088]

[0089] Frequency domain image for determining the current iteration number t based on image update coefficients (including the real part) and the virtual part ), and use the updated frequency domain image Calculate normGradu for the next iteration k and const k The image update calculation formula is as follows: Traverse each frame of the image.

[0090]

[0091] In the above formula, α and λ are both adjustable parameters, and μ is fixed at 1;

[0092] All matrix multiplications in the above formula are dot product operations, meaning that the multiplication operation is performed by traversing each pixel. During the first iteration, normGradu... k =normGrad0 k The iteration process will continue until the maximum number of iterations is set.

[0093] The original algorithm process will const all frames k The sum is used as a convergence condition to determine the const of the current iteration number. k The sum of the two sums is compared to the difference between the previous two sums. If the difference is less than the set threshold, the iteration ends prematurely. During the iteration through all frames, when iterating through the first frame... because It does not exist, therefore it has been changed to Similarly, the last frame is changed to

[0094] In this embodiment, the frequency domain image updated in the last iteration is subjected to two-dimensional discrete inverse Fourier transform after the iteration is completed, and the real part data after the transform is taken as the image output after turbulence repair. The calculation process is the same as the two-dimensional discrete Fourier transform, and the one-dimensional discrete inverse Fourier transform can be described as follows:

[0095]

[0096]

[0097] Step S3, the image after turbulence repair is fused to obtain a fused single-frame image, which is taken as the turbulence repair result of the current frame.

[0098] The fusion process mainly fuses the best parts of the turbulence repair effects of multiple frames together to obtain a more stable turbulence repair effect. The process can also be omitted to directly output the last frame of the turbulence repair image to improve the running speed of the algorithm.

[0099] In this embodiment, as shown in Figure 4 , the mean image I mean of all frame images is calculated as follows:

[0100]

[0101] The Sobel gradient image of all frame images is calculated as follows:

[0102] The boundaries of all frame images are expanded by one pixel, and the expanded pixel value is equal to the pixel value at the image boundary. The Sobel gradient image of the kth frame image is calculated as follows:

[0103]

[0104]

[0105]

[0106] The square of the gray scale difference between all frame images and the reference image is calculated, denoted as Each point is traversed, and the kth frame image of is calculated as follows:

[0107]

[0108] The mean filter is performed on all frame images and . This is equivalent to performing weighted average in the spatial domain in advance, and the default window size is 15*15, and the boundary padding method is mirror padding.

[0109] Find the mixed weight map of all frames. Iterate through each point, the k-th frame of the image The calculation formula is as follows:

[0110]

[0111] By iterating through each point and calculating the weighted average in the time domain, the final fused image I is obtained. fusion The calculation formula is as follows:

[0112]

[0113] This invention also provides a turbulence repair system based on Sobol gradient and multi-frame fusion, and a turbulence repair method based on Sobol gradient and multi-frame fusion in the above embodiments, comprising:

[0114] The frame extraction module acquires FrameNum-1 frames of images preceding the current frame in the video stream, where FrameNum is the preset number of frames required for image fusion.

[0115] The turbulence repair module performs turbulence repair on the current frame and the FrameNum-1 frames preceding the current frame.

[0116] The image synthesis module fuses the images after turbulence restoration to obtain a fused single-frame image, which is then used as the turbulence restoration result for the current frame.

[0117] Based on the same concept, this invention also provides a schematic diagram of a physical structure, such as... Figure 5 As shown, the server may include a processor 810, a communications interface 820, a memory 830, and a communication bus 840. The processor 810, communications interface 820, and memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute the steps of the turbulence repair method based on Sobol gradient and multi-frame fusion as described in the above embodiments. For example, this includes:

[0118] Step S1: Obtain the FrameNum-1 frame images preceding the current frame in the video stream, where FrameNum is the preset number of frames required for image fusion;

[0119] Step S2: Perform turbulence repair on the current frame and the FrameNum-1 frames preceding the current frame;

[0120] Step S3: Fuse the images after turbulence restoration to obtain a fused single-frame image, which is then used as the turbulence restoration result for the current frame.

[0121] Moreover, the logic instructions in the memory 830 described above can be implemented in the form of software functional units and sold or used as independent products, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the parts that make contributions to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0122] Based on the same concept, the embodiments of the present application also provide a non-transitory computer readable storage medium, which stores a computer program containing at least one code, which can be executed by a host device to control the host device to implement the steps of the turbulence repair method based on the Sobel gradient and multi-frame fusion as described in the above embodiments. For example, it includes:

[0123] Step S1, obtaining FrameNum-1 frame images before a current frame in a video stream, wherein FrameNum is a preset frame number of images required for image fusion;

[0124] Step S2, performing turbulence repair on the current frame and the FrameNum-1 frame images before the current frame;

[0125] Step S3, fusing the turbulence repaired images to obtain a fused single frame image, and taking the fused single frame image as a turbulence repair result of the current frame.

[0126] Based on the same technical concept, the embodiments of the present application also provide a computer program, which, when executed by a host device, is used to implement the above method embodiments.

[0127] The program can be stored in whole or in part on a storage medium packaged together with the processor, or in part or in whole on a storage medium not packaged together with the processor.

[0128] Based on the same technical concept, the embodiments of the present application also provide a processor, which is used to implement the above method embodiments. The processor can be a chip.

[0129] The embodiments of the present application can be combined in any manner to achieve different technical effects.

[0130] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another, for example, the computer instructions can be transferred from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that includes one or more available media sets. The available media can be magnetic media (for example, floppy disk, hard disk, magnetic tape), optical media (for example, DVD), or semiconductor media (for example, solid state disk) and the like.

[0131] Those of ordinary skill in the art can understand that all or part of the processes in the above embodiments can be implemented by a computer program to instruct the relevant hardware, and the program can be stored in a computer readable storage medium. When the program is executed, it can include the processes of the above method embodiments. The storage medium includes ROM or random access memory (RAM), magnetic disk or optical disk, and various media that can store program codes.

[0132] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for turbulence repair based on Sobel gradient and multi-frame fusion, characterized in that, The method comprises the following steps: Step S1, obtaining FrameNum-1 frame images before a current frame in a video stream, wherein FrameNum is a preset frame number of images required for image fusion; Step S2, implementing turbulence repair on the current frame and the FrameNum-1 frame images before the current frame; Step S3, fusing the images after the turbulence repair to obtain a single frame image after the fusion, and taking the single frame image after the fusion as a final result of the turbulence repair of the current frame; The step S1 specifically comprises: Step S11, determining a frame number T of the current frame input by the video stream, and determining the frame number FrameNum required for the fusion; Step S12, if it is judged that T is less than FrameNum, taking the current frame as the input without implementing the turbulence repair; Step S13, if it is judged that T is greater than or equal to FrameNum, taking images from the (T-FrameNum+1)th frame to the Tth frame in the video stream; The step S2 specifically comprises: Step S21, determine the input image sequence as I k , k contains K HxW images, where k ∈ {1, 2, 3, …, K}, H is the image height, W is the image width, and K is equal to FrameNum; calculating a coefficient matrix D, wherein the position of any point in the coefficient matrix D is i∈{0, 1, 2, 3, …, H-1}, j∈{0, 1, 2, 3, …, W-1}, and each point D(i, j) in the coefficient matrix is: Step S22, traversing all the frame images, calculating the discrete Fourier transform of each frame image to obtain a sequence of images after the Fourier transform, and calculating the sum of Sobel gradients of each frame image in the sequence of images as an initial condition of iteration; D (i,j) = sin(π x i ÷ H) 2 + sin(π x j ÷ W) 2 ; Step S23, starting iteration, calculating a metric value at a current iteration number, and updating the Fourier transform result; if it is judged that the iteration number is reached or the metric value reaches a convergence condition, performing inverse Fourier transform on the last updated Fourier transform result to output the inverse Fourier transform result of all the frame images as the turbulence repair result; The step S22 specifically comprises: calculating the sum of Sobel gradients of the kth frame image in the sequence of images: all the frame images in the image sequence, and obtains a sequence of discrete Fourier transform images; wherein the real part of the sequence of discrete Fourier transform images is the imaginary part is The step S23 specifically comprises: In the above formula, both α and λ are adjustable parameters, and μ is fixedly set as 1; Compute the sum of Sobel gradients normGradu at the current iteration number t k and the image update coefficient const k , k e {1, 2, 3,..., K}, where const k for the k-th image frame is: Determine the frequency domain image of the current iteration number t based on the image update coefficient And use the updated frequency domain image Calculate normGradu for the next iteration k And const k ; Traverse each frame of image, and the image update calculation formula is as follows: In the step S22, the discrete Fourier transform is a two-dimensional discrete Fourier transform, and the two-dimensional discrete Fourier transform can be obtained by first performing one-dimensional discrete Fourier transform on the image row by row and then performing one-dimensional discrete Fourier transform on the obtained result column by column; During the first iteration, normGradu k =normGrad0 k The iterative process will continue until the set maximum number of iterations is reached or the convergence condition is met. The convergence condition is a constant value of the current iteration number. k Check if the difference between the sum and the previous two is less than the set threshold; if it is less than the threshold, end the iteration early. In the step S23, the inverse Fourier transform on the last updated Fourier transform result specifically comprises: In the step S3, the fusion of the images after the turbulence repair specifically comprises:

2. The turbulence repair method based on Sobol gradient and multi-frame fusion according to claim 1, characterized in that, extending the boundaries of all the frame images by one pixel, and the pixel value of the extension is equal to the pixel value at the image boundary; traversing each point, and the Sobel gradient image calculation formula of the kth frame image is as follows: a mean image I of all frame images mean The calculation formula is as follows: Sobel gradient images for all frames of images The processor implements the steps of the turbulence repair method based on the Sobel gradient and the multi-frame fusion according to any one of claims 1 to 2 when executing the program. Square all the gray value difference between each frame image and the reference image, denoted as Traverse each point, the kth frame image The calculation formula is as follows: all frames are filtered by mean filtering and mean filtering find the blending weight map for all frames traversing each point, the k-th frame image of The formula is as follows: The final fused image I is obtained by traversing each point and calculating the weighted average in the time domain fusion The calculation formula is as follows:

3. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The computer program implements the steps of the turbulence repair method based on the Sobel gradient and the multi-frame fusion according to any one of claims 1 to 2 when executed by the processor.

4. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, ​

Images