Three-dimensional reconstruction method based on turbulence-removed image

Abstract

The invention relates to a three-dimensional reconstruction method based on a turbulence-removed image, belongs to the technical field of three-dimensional reconstruction methods, and solves the problem in the prior art that the precision of a three-dimensional reconstruction model is affected due to low precision of a shot image caused by an atmospheric turbulence phenomenon. The method comprises the following steps: acquiring a video image comprising a to-be-reconstructed target; performing real-time turbulence removal on each frame of image of the video image to obtain a turbulence-removed image of each frame of image; and performing three-dimensional reconstruction based on the feature points of the to-be-reconstructed target in the turbulence-removed image of each frame of image to obtain a three-dimensional model. The accuracy of a three-dimensional reconstruction result is improved.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional reconstruction methods, and in particular to a three-dimensional reconstruction method based on deturbulence images. Background Technology

[0002] 3D reconstruction involves using computers to simulate real-world 3D objects and acquire complete 3D information about them, including structure, texture, and scale. Currently, non-contact measurement methods are common approaches to 3D reconstruction. These methods involve obtaining and measuring the 3D space of a target object without physical contact. Image-based and camera parameter-based 3D reconstruction is a typical non-contact measurement method. This method reconstructs the 3D shape of an object using image feature information and camera parameters. Reconstruction accuracy is a crucial indicator for evaluating the reconstruction results.

[0003] However, due to atmospheric turbulence, the image accuracy is low, the target geometry is deformed and blurred during long-distance imaging, which in turn affects the accuracy of the 3D reconstruction model. Summary of the Invention

[0004] Based on the above analysis, the embodiments of the present invention aim to provide a three-dimensional reconstruction method based on deturbulent images, in order to solve the problem that the accuracy of existing three-dimensional reconstruction models is affected by the low accuracy of captured images due to atmospheric turbulence.

[0005] The objective of this invention is mainly achieved through the following technical solutions:

[0006] This invention provides a three-dimensional reconstruction method based on deturbulence images, characterized by the following steps:

[0007] Acquire video images including the target to be reconstructed;

[0008] Real-time deturbulence is performed on each frame of the video image to obtain the deturbulence-free image of each frame;

[0009] A 3D model is obtained by reconstructing the target in the deturbulentized image of each frame.

[0010] Furthermore, the process of obtaining a 3D model by reconstructing the target from feature points in the deturbulentized images of each frame includes:

[0011] Extract feature points of the target to be reconstructed from the deturbulentized images of each frame;

[0012] Feature point matching is performed on the deturbulentized image of each frame and the deturbulentized image of the previous frame to establish the feature trajectory of the target to be reconstructed.

[0013] Obtain the internal and external parameters of the camera unit;

[0014] A projection matrix is ​​constructed based on the feature trajectory of the target to be reconstructed, the internal parameters, and the external parameters;

[0015] Based on the projection matrix, the three-dimensional spatial coordinates of the pixels of the target to be reconstructed in the deturbulent image of each frame are calculated, a three-dimensional point cloud of the target to be reconstructed is generated, and a three-dimensional model of the target is obtained.

[0016] Furthermore, the generated 3D point cloud of the target to be reconstructed is subjected to point cloud densification processing, and the 3D model of the target is obtained through texture mapping.

[0017] Furthermore, obtaining the deturbulentized image of each frame of the video image using a real-time image deturbulence method includes:

[0018] The first deturbulence weight and the second deturbulence weight are used to fuse each frame of the video image with the deturbulence-suppressed image of the previous frame of each frame to obtain the first weighted turbulence suppression image and the second weighted turbulence suppression image of each frame.

[0019] Motion values ​​of each pixel are obtained by detecting moving targets based on the normalized visual weights of each pixel in each frame of the image.

[0020] Based on the motion values ​​of each pixel in each frame image, the first weighted turbulence suppression image of each frame image, and the second weighted turbulence suppression image of each frame image, a weighted superposition is performed to obtain the preliminary deturbulence image of each frame image.

[0021] The initial deturbulence images of each frame are enhanced with details to obtain the final deturbulence images.

[0022] Furthermore, the normalized visual weights of each pixel in each frame of the image are obtained, including:

[0023] Transform each pixel of each frame of the image from the spatial domain to the frequency domain;

[0024] Perform a two-dimensional inverse Fourier transform on the phase portion of each pixel in the frequency domain of each frame image to obtain the transformed pixel value of each pixel in each frame image.

[0025] Based on the transformed pixel values ​​of each pixel in each frame of the image, the visual weight of each pixel in each frame of the image is obtained by using a two-dimensional Gaussian function to perform blur widening.

[0026] The visual weights of each pixel in each frame of the image are normalized to obtain the normalized visual weights belonging to the [0,1] interval.

[0027] Furthermore, the step of converting each pixel of each frame image from the spatial domain to the frequency domain includes:

[0028] The spatial domain of each pixel in each frame of the image is orthogonally decomposed into motion components and intensity components.

[0029] Two-dimensional Fourier transforms are performed on the motion and intensity components of each pixel after decomposition, transforming each pixel from the spatial domain to the frequency domain.

[0030] Furthermore, the step of performing a two-dimensional inverse Fourier transform on the phase portion of each frequency domain pixel in each frame image to obtain the pixel value of each transformed pixel in each frame image includes:

[0031] By setting the magnitude of the motion component and the magnitude of the intensity component of each frequency domain pixel to 1, we obtain the motion component and intensity component of each frequency domain pixel that contain only phase information.

[0032] Two-dimensional inverse Fourier transforms are performed on the motion component and intensity component of each frequency domain pixel that contains only phase information to obtain the transformed pixel value of each pixel.

[0033] Furthermore, a preset threshold is used to normalize the visual weights of each pixel in each frame image to distinguish moving targets, including:

[0034] When the normalized visual weight of each pixel in each frame image is greater than or equal to the preset threshold, the pixel is determined to be a moving point and the motion value of the pixel is set to 1.

[0035] When the normalized visual weight of each pixel in each frame image is less than the preset threshold, the pixel is determined to be a stationary point and the motion value of the pixel is set to 0.

[0036] Furthermore, based on the motion values ​​of each pixel in each frame image, the first weighted turbulence suppression image of each pixel in each frame image, and the second weighted turbulence suppression image of each pixel in each frame image, the following formula is used to perform weighted superposition for deturbulence removal to obtain the preliminary deturbulence image of each frame image:

[0037]

[0038] Among them, Mask N (x,y) represents the motion value of pixel (x,y) in the Nth frame of the image; This represents the first weighted turbulence suppression image of the Nth frame; This represents the second-weighted turbulence suppression image of the Nth frame.

[0039] Furthermore, the step of enhancing the details of the preliminary deturbulence images of each frame to obtain the final deturbulence image includes:

[0040] Based on the preliminary deturbulence images of each frame, the preliminary deturbulence visual weights of each pixel in each frame are obtained.

[0041] Based on the preliminary deturbulence visual weights of each pixel in each frame, the final deturbulence image is obtained using the following formula:

[0042]

[0043] Where λ1 represents the first final deturbulence weight; λ2 represents the second final deturbulence weight; Represents the initial deturbulence-free visual weight of pixel (x,y) in the Nth frame image; This represents the pixel value (x, y) of the initial deturbulence image in frame N.

[0044] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0045] 1. This invention performs deturbulence preprocessing on the images used for 3D reconstruction before the 3D reconstruction process, avoiding the problem that atmospheric turbulence causes low image accuracy, which in turn affects the accuracy of the 3D reconstruction model, thus improving the accuracy of the 3D reconstruction results.

[0046] 2. This invention obtains a visual matrix by detecting and analyzing potential moving targets, and then segments the potential target region based on this matrix, which can effectively suppress motion region trailing and eliminate background region turbulence.

[0047] 3. This invention performs deturbulence fusion on the motion region and background region with different weights, and further enhances the fusion result, which can effectively suppress motion blur and improve detail clarity.

[0048] 4. This invention decomposes the pixel information of an image into a representation of motion components and intensity components. It represents the attention given to the motion of a pixel from both the actual motion of the pixel and the brightness, which can more accurately distinguish the foreground target and background information of the image and improve the accuracy of moving target tracking.

[0049] 5. This invention converts the pixel information of an image from the spatiotemporal domain to the frequency domain. In the field of optics, phase information has direct physical meaning. Therefore, the phase part is regarded as the best representation of information features. The inverse Fourier transform of the phase part is performed to extract moving targets in the image. Processing only phase information is simpler and more efficient than processing amplitude information.

[0050] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description

[0051] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0052] Figure 1 This is a flowchart illustrating the three-dimensional reconstruction method based on deturbulence images in an embodiment of the present invention;

[0053] Figure 2 This is a flowchart illustrating the real-time deturbulence removal method in an embodiment of the present invention;

[0054] Figure 3 This is a schematic diagram of the process for obtaining the visual weights of each pixel in each frame of an image in an embodiment of the present invention.

[0055] Figure 4 This is a schematic diagram of the process of obtaining a three-dimensional model through three-dimensional reconstruction in an embodiment of the present invention. Detailed Implementation

[0056] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0057] A specific embodiment of the present invention discloses a three-dimensional reconstruction method based on deturbulence images, such as... Figure 1 As shown, it includes the following steps S1-S3:

[0058] Step S1: Obtain video images including the target to be reconstructed.

[0059] Specifically, 3D reconstruction requires obtaining images of the target to be reconstructed. During the shooting process, video images are captured according to the planned shooting route. After the video images are captured, OpenCV is used to collect each frame of the video sequence.

[0060] Step S2: Perform real-time deturbulence on each frame of the video image to obtain the deturbulence-free image of each frame.

[0061] Specifically, such as Figure 2 As shown, each frame of the video image is subjected to real-time deturbulence using steps S21-S24.

[0062] Step S21: For each frame of the video image, the first deturbulence weight and the second deturbulence weight are used to fuse with the deturbulence-suppressed image of the previous frame of each frame to obtain the first weighted turbulence suppression image and the second weighted turbulence suppression image of each frame.

[0063] Specifically, in a normal turbulent scene, the pixel information at each position is offset from its true position. Since atmospheric turbulence has a quasi-periodic property, it is statistically stationary in nature, meaning that the offset is statistically stationary over time. Therefore, by simulating short-exposure images through weighted averaging between frames in the sequence, the distortion effect can be reduced, making the background image, which has not undergone actual displacement, relatively stable.

[0064] Furthermore, each frame of the video image is fused with the deturbulence-suppressed image of the previous frame using a first deturbulence weight, resulting in a first-weighted turbulence-suppressed image for each frame. The formula is as follows:

[0065]

[0066] Where α1 represents the first deturbulence weight; f N-1 This represents the image after deturbulence removal in frame N-1; This represents the Nth frame image, i.e., the original Nth frame image.

[0067] The second weighted turbulence suppression image of each frame of the video image is obtained by fusing it with the deturbulence-suppressed image of the previous frame of each frame using the second deturbulence weight, and the formula is as follows:

[0068]

[0069] Where α2 represents the second deturbulence weight.

[0070] It should be noted that when processing the first frame, since there is no previous frame after deturbulence removal, the formula is used directly: As the first weighted turbulence suppression image of the first frame, and This serves as the second weighted turbulence suppression image for the first frame.

[0071] Specifically, when the deturbulence weight is small, more information from the previous frame after deturbulence is introduced, which slows down the turbulence. However, information in areas with motion is prone to remainder and superimpose, resulting in information loss in these areas, i.e., motion blur or strong blurring. When the deturbulence weight is large, more information from the current frame is introduced, reducing motion information from the previous frame and suppressing motion blur. It also introduces more turbulence information from the current frame, i.e., turbulence remains are obvious.

[0072] Therefore, it is necessary to separate the potential moving target region of the image from the background region and assign different deturbulence weight values ​​to different regions. In this embodiment, α1 < α2 is set, that is, the first deturbulence weight is used for the background region and the second deturbulence weight is used for the moving region. For example, α1 = 0.1 and α2 = 0.9 are set.

[0073] Step S22: Based on the normalized visual weights of each pixel in each frame image, perform moving target detection to obtain the motion value of each pixel.

[0074] Furthermore, such as Figure 3 As shown, the visual weights of each pixel in each frame of the image are obtained, including steps S221-S224:

[0075] Step S221: Convert each pixel of each frame image from the spatial domain to the frequency domain.

[0076] Specifically, in the frequency domain of an image, noise is usually present in the high-frequency part, while useful information is mainly distributed in the low-frequency region. By converting the image from the spatial domain to the frequency domain, noise in the image can be removed more effectively.

[0077] Furthermore, the step of converting each pixel of each frame image from the spatial domain to the frequency domain includes:

[0078] The spatial domain of each pixel in each frame of the image is orthogonally decomposed into motion and intensity components using the following formula:

[0079] q N (x,y)=M N (x,y)u1+I N (x,y)·u2

[0080] (u1) 2 =(u2) 2 =-1

[0081] u1⊥u2

[0082] Among them, M N (x,y) represents the motion component of pixel (x,y) in the Nth frame of the image; I N (x,y) represents the intensity component of pixel (x,y) in the Nth frame image; u1 represents the first unit vector; u2 represents the second unit vector.

[0083] Specifically, the motion component of a pixel (x, y) represents the difference between the current frame and the previous frame, that is, the motion change value between two consecutive frames, where M N (x,y)=|g N (x,y)-g N-1 (x,y)|;gN (x,y) represents the pixel value of pixel (x,y) in the Nth frame of the image; g N-1 (x,y) represents the pixel value of pixel (x,y) in the (N-1)th frame of the image.

[0084] On the other hand, the intensity component of a pixel (x,y), i.e., the brightness or color intensity of the pixel, can be used to characterize the difference between the target region and the background region, where I N (x,y)=average(g N (x,y)); when the image is three-channel, average(g) N (x,y)) represents the average of the three channels; when the image is a single-channel image, i.e., a grayscale image, then average(g) = 0. N (x,y))=g N (x,y).

[0085] It should be noted that decomposing the pixel information of an image into motion and intensity components, and representing the attention given to pixel motion from both the actual pixel motion and brightness aspects, can more accurately distinguish between foreground and background information in an image, thereby improving the accuracy of moving target tracking.

[0086] Furthermore, a two-dimensional Fourier transform is performed on the motion component and intensity component of each pixel after decomposition, transforming each pixel from the spatial domain to the frequency domain.

[0087] Specifically, the motion components of each pixel after decomposition are used to obtain the motion components of the pixel in the frequency domain using a two-dimensional Fourier transform, as follows:

[0088]

[0089] Among them, |(F M (u,v)) N | represents the magnitude of the motion component of the frequency domain pixel (u,v) in the Nth frame of the image; The phase information of the motion component of the frequency domain pixel (u,v) of the Nth frame image;

[0090] The intensity components of each pixel after decomposition are used to obtain the intensity components of the pixel in the frequency domain using a two-dimensional Fourier transform, as follows:

[0091]

[0092] Among them, |(F I (u,v)) N | represents the magnitude of the intensity component of the frequency domain pixel (u,v) in the Nth frame image; This represents the phase information of the intensity component of the frequency domain pixel (u,v) in the Nth frame image.

[0093] Step S222: Perform a two-dimensional inverse Fourier transform on the phase part of each frequency domain pixel in each frame image to obtain the transformed pixel value of each pixel in each frame image.

[0094] Specifically, the phase information of an image mainly carries the image's edge and texture information. By preserving the phase information and converting it back to the spatial domain, we can remove noise while preserving as many important structural features of the image as possible.

[0095] Furthermore, the magnitudes of the motion component and intensity component of each frequency domain pixel are set to 1, resulting in motion and intensity components of each frequency domain pixel containing only phase information. A two-dimensional inverse Fourier transform is then performed on the motion and intensity components of each frequency domain pixel containing only phase information to obtain the transformed pixel value. The transformed pixel value is expressed as follows:

[0096] q′ N (x,y)=M′ N (x,y)u1+I′ N (x,y)·u2

[0097] Among them, M′ N (x,y) represents the transformed motion component of pixel (x,y) in the Nth frame image; I′ N (x,y) represents the transformed intensity component of pixel (x,y) in the Nth frame of the image.

[0098] Step S223: Based on the transformed pixel values ​​of each pixel in each frame image, use a two-dimensional Gaussian function to perform blur widening to obtain the visual weight of each pixel in each frame image.

[0099] Furthermore, based on the transformed pixel values, the visual weight of each pixel in each frame image is obtained using the following formula:

[0100]

[0101] Among them, Sa N (x,y) represents the visual weight of pixel (x,y) in the Nth frame image; gauss(r,σ) represents the two-dimensional Gaussian function; q′ N (x,y) represents the transformed pixel value of pixel (x,y) in the Nth frame of the image; |q′ N (x,y)| represents q′ N The modulus of (x,y); This indicates a convolution operation.

[0102] Specifically, the two-dimensional Gaussian function is often used for appropriate blurring and broadening, which can enhance visually important areas in an image, while making the image look smoother, making the details of the image more harmonious, and reducing abrupt transitions.

[0103] Step S224: Assign the visual weight Sa to each pixel in each frame image. N (x,y) is normalized to obtain the normalized visual weight Sa' belonging to the interval [0,1]. N (x,y).

[0104] It should be noted that the closer the visual weight of each pixel is to 1, the more likely it is a potential motion area, and the closer it is to 0, the more likely it is a background area.

[0105] Furthermore, a preset threshold is used to normalize the visual weights of each pixel in each frame image to distinguish moving targets, including:

[0106] When the normalized visual weight of each pixel in each frame image is greater than or equal to the preset threshold, the pixel is determined to be a moving point and the motion value of the pixel is set to 1.

[0107] When the normalized visual weight of each pixel in each frame image is less than the preset threshold, the pixel is determined to be a stationary point and the motion value of the pixel is set to 0.

[0108] Specifically, the following formula is used to determine pixels larger than the threshold Th as moving target regions, thereby achieving the separation of moving targets;

[0109]

[0110] Among them, Mask N (x,y) represents the motion value of pixel (x,y) in the Nth frame of the image; Sa' N (x,y) represents the normalized visual weight of pixel (x,y) in the Nth frame image; pixels with a motion value of 1 are considered moving points, and pixels with a motion value of 0 are considered stationary points; the preset threshold Th can be set to a value between 0 and 1 depending on different situations.

[0111] Step S23: Based on the motion values ​​of each pixel in each frame image, the first weighted turbulence suppression image of each frame image, and the second weighted turbulence suppression image of each frame image, perform weighted superposition to obtain the preliminary deturbulence image of each frame image.

[0112] Furthermore, the following formula is used for weighted superposition to obtain preliminary deturbulence images for each frame:

[0113]

[0114] Among them, Mask N (x,y) represents the motion value of pixel (x,y) in the Nth frame of the image; This represents the first weighted turbulence suppression image of the Nth frame; This represents the second-weighted turbulence suppression image of the Nth frame.

[0115] Specifically, as mentioned earlier, for the background area, i.e., the Mask... N Pixels with (x,y) = 0 use the first deturbulence weight α1; for the motion region, i.e., the Mask... N The pixel (x,y)=1 is subjected to the second deturbulence weight α2 for initial deturbulence removal.

[0116] Step S24: Perform detail enhancement on the preliminary deturbulence images of each frame to obtain the final deturbulence image.

[0117] Furthermore, based on the preliminary deturbulence images of each frame, the preliminary deturbulence visual weights of each pixel in each frame are obtained.

[0118] Specifically, for the preliminary deturbulence images of each frame, the method of steps S221-S224 is used to obtain the preliminary deturbulence normalized visual weights of each pixel in each frame. Among them, the normalized visual weights after preliminary deturbulence removal The values ​​are those that belong to the interval [0,1].

[0119] Furthermore, a weighted enhancement approach is adopted to further enhance image pixels of interest and appropriately suppress image pixels of uninteresting interest, thereby improving image contrast.

[0120] Specifically, based on the preliminary deturbulence visual weights of each pixel in each frame of the image, the final deturbulence image is obtained using the following formula:

[0121]

[0122] in, λ1 represents the final deturbulence pixel value of pixel (x,y) in the Nth frame image; λ2 represents the first final deturbulence weight; λ3 represents the second final deturbulence weight. Represents the initial deturbulence-free normalized visual weight of the pixel (x,y) in the Nth frame image; This represents the pixel value (x, y) of the initial deturbulence image in frame N.

[0123] It should be noted that in this embodiment, λ1 > λ2, and λ1 + λ2 = 1.

[0124] Step S3: Based on the feature points of the target to be reconstructed in the deturbulentized image of each frame, perform three-dimensional reconstruction to obtain a three-dimensional model.

[0125] Specifically, such as Figure 4 As shown, the process of performing three-dimensional reconstruction to obtain a three-dimensional model includes steps S31-S35:

[0126] Step S31: Extract feature points of the target to be reconstructed from the deturbulent images of each frame.

[0127] Specifically, the feature point extraction methods include SURF feature detector, SIFT feature detector, FAST feature detector and ORB feature detector.

[0128] Step S32: Perform feature point matching between the deturbulentized image of each frame and the deturbulentized image of the previous frame to establish the feature trajectory of the target to be reconstructed.

[0129] Specifically, feature point matching involves finding corresponding feature points of the same spatial object in other images based on the feature descriptions of the feature points. The accurate motion trajectory of the camera unit can be obtained through the matching results. The feature point matching methods include the K-Nearest Neighbors (KNN) algorithm and the FLANN algorithm.

[0130] Step S33: Obtain the internal and external parameters of the camera unit.

[0131] Specifically, the internal parameters of the camera unit include f x f y , u0 and v0; where, f x f is the number of pixels on the horizontal axis. y The vertical axis represents the number of pixels; u0 represents the coordinate of the principal point of the camera unit on the u-axis of the pixel coordinate system; v0 represents the coordinate of the principal point of the camera unit on the v-axis of the pixel coordinate system. The internal parameters of the camera unit are related to the focal length and image size of the camera unit.

[0132] The external parameters of the camera unit record the motion information of the camera unit, including the rotation angle and translation of the camera unit with respect to the world coordinate system.

[0133] Preferably, in this embodiment, the OpenCV cv2.calibrateCamera function is used to obtain the internal parameters of the camera unit; the cv2.solvePnP function is used to estimate the pose of the object in the camera coordinate system based on the rotation angle and translation of the camera unit with respect to the world coordinate system, thereby obtaining the rotation matrix and translation vector of the camera unit.

[0134] Step S34: Construct a projection matrix based on the feature trajectory, the internal parameters, and the external parameters.

[0135] Specifically, the formula for the projection matrix is ​​as follows:

[0136]

[0137] Where C is the projection matrix; f / dx is the length of the focal length in the x-axis direction described by pixels; f / dy is the length of the focal length in the y-axis direction described by pixels; f is the function relationship of the camera with respect to the pixel coordinates; (u0,v0) is the pixel offset; R is the rotation matrix; and t is the translation vector.

[0138] Step S35: Based on the projection matrix, calculate the three-dimensional spatial coordinates of the pixels of the target to be reconstructed in the deturbulent image of each frame, generate the three-dimensional point cloud of the target to be reconstructed, and obtain the three-dimensional model of the target.

[0139] Specifically, the three-dimensional spatial coordinates of the pixels corresponding to the target to be reconstructed in the deturbulentized image of each frame are calculated using the following formula:

[0140]

[0141] Where λ is the scaling factor, with a value of Zw; (u,v) are the coordinates of the pixel; C is the projection matrix; and (Xw,Yw,Zw) are the three-dimensional spatial coordinates of the three-dimensional point cloud.

[0142] Furthermore, the generated 3D point cloud is densified, and a 3D model is constructed through texture mapping.

[0143] Specifically, in this embodiment, the point cloud densification process is performed using the unconstrained nearest neighbor interpolation method.

[0144] In summary, the three-dimensional reconstruction method based on deturbulence images according to the embodiments of the present invention has the following beneficial effects:

[0145] 1. This invention performs deturbulence preprocessing on the images used for 3D reconstruction before the 3D reconstruction process, avoiding the problem that atmospheric turbulence causes low image accuracy, which in turn affects the accuracy of the 3D reconstruction model, thus improving the accuracy of the 3D reconstruction results.

[0146] 2. This invention obtains a visual matrix by detecting and analyzing potential moving targets, and then segments the potential target region based on this matrix, which can effectively suppress motion region trailing and eliminate background region turbulence.

[0147] 3. This invention performs deturbulence fusion on the motion region and background region with different weights, and further enhances the fusion result, which can effectively suppress motion blur and improve detail clarity.

[0148] 4. This invention decomposes the pixel information of an image into a representation of motion components and intensity components. It represents the attention given to the motion of a pixel from both the actual motion of the pixel and the brightness, which can more accurately distinguish the foreground target and background information of the image and improve the accuracy of moving target tracking.

[0149] 5. This invention converts the pixel information of an image from the spatiotemporal domain to the frequency domain. In the field of optics, phase information has direct physical meaning. Therefore, the phase part is regarded as the best representation of information features. The inverse Fourier transform of the phase part is performed to extract moving targets in the image. Processing only phase information is simpler and more efficient than processing amplitude information.

[0150] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A three-dimensional reconstruction method based on deturbulence images, characterized in that, Includes the following steps: Acquire video images including the target to be reconstructed; Real-time deturbulence is performed on each frame of the video image to obtain the deturbulence-free image of each frame; A 3D model is obtained by reconstructing the target in the deturbulentized image of each frame.

2. The method according to claim 1, characterized in that, The method of obtaining a 3D model by reconstructing the target in the deturbulentized image of each frame of the image includes: Extract feature points of the target to be reconstructed from the deturbulentized images of each frame; Feature point matching is performed on the deturbulentized image of each frame and the deturbulentized image of the previous frame to establish the feature trajectory of the target to be reconstructed. Obtain the internal and external parameters of the camera unit; A projection matrix is ​​constructed based on the feature trajectory of the target to be reconstructed, the internal parameters, and the external parameters; Based on the projection matrix, the three-dimensional spatial coordinates of the pixels of the target to be reconstructed in the deturbulent image of each frame are calculated, a three-dimensional point cloud of the target to be reconstructed is generated, and a three-dimensional model of the target is obtained.

3. The method according to claim 2, characterized in that, The generated 3D point cloud of the target to be reconstructed is subjected to point cloud densification processing, and the 3D model of the target is obtained through texture mapping.

4. The method according to claim 1, characterized in that, The process of obtaining the deturbulent images of each frame of the video image using a real-time image deturbulence method includes: The first deturbulence weight and the second deturbulence weight are used to fuse each frame of the video image with the deturbulence-suppressed image of the previous frame of each frame to obtain the first weighted turbulence suppression image and the second weighted turbulence suppression image of each frame. Motion values ​​of each pixel are obtained by detecting moving targets based on the normalized visual weights of each pixel in each frame of the image. Based on the motion values ​​of each pixel in each frame image, the first weighted turbulence suppression image of each frame image, and the second weighted turbulence suppression image of each frame image, a weighted superposition is performed to obtain the preliminary deturbulence image of each frame image. The initial deturbulence images of each frame are enhanced with details to obtain the final deturbulence images.

5. The method according to claim 4, characterized in that, The normalized visual weights of each pixel in each frame of the image are obtained, including: Transform each pixel of each frame of the image from the spatial domain to the frequency domain; Perform a two-dimensional inverse Fourier transform on the phase portion of each pixel in the frequency domain of each frame image to obtain the transformed pixel value of each pixel in each frame image. Based on the transformed pixel values ​​of each pixel in each frame of the image, the visual weight of each pixel in each frame of the image is obtained by using a two-dimensional Gaussian function to perform blur widening. The visual weights of each pixel in each frame of the image are normalized to obtain the normalized visual weights belonging to the [0,1] interval.

6. The method according to claim 5, characterized in that, The step of converting each pixel of each frame image from the spatial domain to the frequency domain includes: The spatial domain of each pixel in each frame of the image is orthogonally decomposed into motion components and intensity components. Two-dimensional Fourier transforms are performed on the motion and intensity components of each pixel after decomposition, transforming each pixel from the spatial domain to the frequency domain.

7. The method according to claim 6, characterized in that, The step of performing a two-dimensional inverse Fourier transform on the phase portion of each frequency domain pixel in each frame image to obtain the pixel value of each transformed pixel in each frame image includes: By setting the magnitude of the motion component and the magnitude of the intensity component of each frequency domain pixel to 1, we obtain the motion component and intensity component of each frequency domain pixel that contain only phase information. Two-dimensional inverse Fourier transforms are performed on the motion component and intensity component of each frequency domain pixel that contains only phase information to obtain the transformed pixel value of each pixel.

8. The method according to any one of claims 4-7, characterized in that, Moving targets are distinguished using the normalized visual weights of each pixel in each frame of the image based on a preset threshold, including: When the normalized visual weight of each pixel in each frame image is greater than or equal to the preset threshold, the pixel is determined to be a moving point and the motion value of the pixel is set to 1. When the normalized visual weight of each pixel in each frame image is less than the preset threshold, the pixel is determined to be a stationary point and the motion value of the pixel is set to 0.

9. The method according to claim 8, characterized in that, Based on the motion values ​​of each pixel in each frame, the first-weighted turbulence suppression image of each pixel in each frame, and the second-weighted turbulence suppression image of each pixel in each frame, the following formula is used to perform weighted superposition for deturbulence removal to obtain the preliminary deturbulence image of each frame: Among them, Mask N (x,y) represents the motion value of pixel (x,y) in the Nth frame of the image; This represents the first weighted turbulence suppression image of the Nth frame; This represents the second-weighted turbulence suppression image of the Nth frame.

10. The method according to claim 9, characterized in that, The process of enhancing the details of the preliminary deturbulence images of each frame to obtain the final deturbulence image includes: Based on the preliminary deturbulence images of each frame, the preliminary deturbulence visual weights of each pixel in each frame are obtained. Based on the preliminary deturbulence visual weights of each pixel in each frame, the final deturbulence image is obtained using the following formula: Where λ1 represents the first final deturbulence weight; λ2 represents the second final deturbulence weight; Represents the initial deturbulence-free visual weight of pixel (x,y) in the Nth frame image; This represents the pixel value (x, y) of the initial deturbulence image in frame N.

Images