Image-based motion deblurring and 2d gaussian point cloud based scene surface reconstruction method, device, equipment and medium

Abstract

The application relates to a scene surface reconstruction method, device, equipment and medium based on image de-motion blurring and two-dimensional Gaussian point painting, which comprises the following steps: performing gradient back propagation according to luminosity loss, smoothness loss, normal consistency loss and depth consistency loss to adjust camera poses of real motion blurring images and Gaussian point cloud parameters; performing surface estimation on clear images corresponding to the real motion blurring images based on ray tracing, transparency accumulation and depth regularization constraint to restore a two-dimensional Gaussian point painting scene of the clear images corresponding to the real motion blurring images, wherein the two-dimensional Gaussian point painting scene comprises object surface depth and object surface shape; and performing scene surface reconstruction on the clear images corresponding to the real motion blurring images according to the object surface depth and the object surface shape by using a preset truncated signed distance function. The application can enhance the reality of scene reconstruction and realize accurate two-dimensional Gaussian point painting scene reconstruction.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a scene surface reconstruction method based on image motion deblurring and two-dimensional Gaussian point rendering, a corresponding device, an electronic device and a computer-readable storage medium. Background Art

[0002] With the widespread adoption of cameras (such as smartphones, drones, and self-driving car cameras), the demand for cameras to capture clear images in dynamic scenes continues to increase. However, the relative motion between the camera and the scene during capture can cause image blur, which is particularly noticeable in low-light environments, fast-moving scenes, or high-resolution video. This blurring degrades image quality, complicating subsequent computer vision tasks such as image recognition and 3D reconstruction.

[0003] Motion blur is typically caused by the relative motion of the camera and the scene during exposure, resulting in a spatially non-uniform blur kernel (i.e., point spread function, PSF). In the blind deblurring problem, the goal is to simultaneously estimate the pixel-wise blur kernel and the corresponding sharp image. Early research on motion deblurring focused on spatially uniform blur in images, assuming that the blur is uniformly distributed throughout the image. However, in real-world scenes, due to varying depth of field and complex camera motions (such as roll or jitter), the assumption of uniform blur often fails, making deblurring more difficult.

[0004] Recently, many methods have been proposed to address the problem of non-uniform deblurring, particularly those leveraging multi-view information to improve the accuracy of depth estimation. However, existing methods still have shortcomings in addressing non-uniform blur caused by depth variations. This is especially true when the camera motion is complex and the scene depth varies significantly. Traditional methods struggle to accurately reconstruct the scene surface and depth information.

[0005] In summary, there are still deficiencies in adapting to the non-uniform blur caused by depth of field changes in the existing technology. In particular, when the camera motion trajectory is complex and the scene depth changes significantly, traditional methods find it difficult to accurately reconstruct the scene surface and depth information. The applicant has made corresponding explorations to solve this problem. Summary of the Invention

[0006] The purpose of this application is to solve the above problems and provide a scene surface reconstruction method based on image motion deblurring and two-dimensional Gaussian point rendering, a corresponding device, an electronic device and a computer-readable storage medium.

[0007] In order to meet the various objectives of this application, this application adopts the following technical solutions:

[0008] A scene surface reconstruction method based on image motion deblurring and two-dimensional Gaussian point rendering is proposed to meet one of the purposes of this application, including:

[0009] In response to a scene surface reconstruction event, obtaining a camera pose and Gaussian point cloud parameters corresponding to a real motion blurred image, and using a preset Bezier curve model to fit a camera motion trajectory of the real motion blurred image according to the camera pose corresponding to the real motion blurred image;

[0010] Sampling clear images at different time points along the camera motion trajectory to construct a clear image sequence, and constructing a synthetic motion blurred image based on the camera poses corresponding to the clear images at different time points and the exposure time of the camera motion trajectory, wherein the clear image is a differentiable rendering of a two-dimensional Gaussian point rendering scene;

[0011] Calculating and determining a photometric loss between the synthesized motion blurred image and the real motion blurred image, a smoothness loss, a normal consistency loss, and a depth consistency loss between the clear images of adjacent frames in the clear image sequence, and performing gradient backpropagation based on the photometric loss, smoothness loss, normal consistency loss, and depth consistency loss to adjust the camera pose and Gaussian point cloud parameters of the real motion blurred image to determine a clear image corresponding to the real motion blurred image;

[0012] performing surface estimation on the clear image corresponding to the real motion blurred image based on ray tracing, transparency accumulation, and depth regularization constraints to restore a two-dimensional Gaussian point rendering scene of the clear image corresponding to the real motion blurred image, wherein the two-dimensional Gaussian point rendering scene includes object surface depth and object surface shape;

[0013] A preset truncated signed distance function is used to reconstruct the scene surface of the clear image corresponding to the real motion blurred image according to the surface depth and surface shape of the object, so as to complete the scene surface reconstruction based on image de-motion blur and two-dimensional Gaussian point rendering.

[0014] Optionally, the luminosity loss between the synthesized motion blurred image and the real motion blurred image is expressed as:

[0015] L image =(1-λ dssim )|I blur -I gt |1+λ dssim (1-SSIM(I blur ,I gt )),

[0016] Among them, L imagerepresents the photometric loss λ between the synthetic motion blurred image and the real motion blurred image dssim is the similarity loss weight value, I blur represents the synthesized motion blurred image, I gt Represents a real motion blurred image, SSIM(I blur ,I gt ) is used to measure the similarity between the synthesized motion blurred image and the real motion blurred image, |I blur -I gt |1 represents the pixel difference between the synthetic motion blurred image and the real motion blurred image, (1-λ dssim )|I blur -I gt |1 represents the pixel-level loss term, λ dssim (1-SSIM(I blur ,I gt )) represents the structural similarity loss term;

[0017] The smoothness loss between the clear images of adjacent frames in the clear image sequence is expressed as follows:

[0018]

[0019] Among them, L smooth represents the smoothing loss between the clear images of adjacent frames in the clear image sequence, N represents the total number of pixels in the image sequence, i is the pixel index, which represents the position of each pixel in the image, j represents the frame index, which represents the subframe in the clear image sequence, represents the value of the i-th pixel in the j-th frame;

[0020] The normal consistency loss between the clear images of adjacent frames in the clear image sequence is expressed as follows:

[0021]

[0022] Among them, L normal It is expressed as the normal consistency loss between the clear images of adjacent frames in the clear image sequence, represents the normal rendered at point j of the i-th clear image sequence, represents the normal calculated at point j of the i-th clear image sequence, ω i represents the weight of the i-th clear image sequence;

[0023] The depth consistency loss between the clear images of adjacent frames in the clear image sequence is expressed as follows:

[0024]

[0025] Among them, L dist represents a depth consistency loss between clear images of adjacent frames in the clear image sequence; represents the depth value of the i-th clear image sequence at point j, represents the depth value of the i-th clear image sequence at point k, ω i represents the weight of the i-th clear image sequence, ω j The weight of the i-th clear image sequence at point j, ω k Represents the weight of the i-th clear image sequence at point k.

[0026] Optionally, the step of performing gradient back propagation according to the photometric loss, smoothness loss, normal consistency loss, and depth consistency loss to adjust the camera pose and Gaussian point cloud parameters of the real motion blurred image to determine a clear image corresponding to the real motion blurred image includes:

[0027] Obtaining a first weight corresponding to a photometric loss between the synthesized motion blurred image and the real motion blurred image, a second weight corresponding to a smoothness loss between clear images of adjacent frames in the clear image sequence, a third weight corresponding to a normal consistency loss between clear images of adjacent frames in the clear image sequence, and a fourth weight corresponding to a depth consistency loss between clear images of adjacent frames in the clear image sequence;

[0028] Calculating and determining a first product between the photometric loss and the first weight, a second product between the smoothness loss and the second weight, a third product between the normal consistency loss and the third weight, and a fourth product between the depth consistency loss and the fourth weight;

[0029] calculating and determining a sum of the first product, the second product, the third product, and the fourth product to determine a total loss between the synthesized motion blurred image and the clear image;

[0030] Gradient back propagation is performed based on the total loss between the synthesized motion blurred image and the clear image to adjust the camera pose and Gaussian point cloud parameters of the real motion blurred image to determine the clear image corresponding to the real motion blurred image.

[0031] Optionally, the total loss between the synthesized motion blurred image and the clear image is expressed as:

[0032] L total =λ image L image +λ smooth(t) L smooth +λnormal(t) L normal +

[0033] λ dist(t) L dist ,

[0034] Among them, λ image represents a first weight corresponding to the photometric loss between the synthesized motion blurred image and the real motion blurred image, λ smooth(t) represents the second weight corresponding to the smoothness loss between the clear images of adjacent frames in the clear image sequence, λ normal(t) represents the third weight corresponding to the normal consistency loss between the clear images of adjacent frames in the clear image sequence, λ dist(t) represents the fourth weight corresponding to the depth consistency loss between the clear images of adjacent frames in the clear image sequence, λ smooth(t) ,λ normal(t) and λ dist(t) It changes with the number of iterations t.

[0035] Optionally, the step of using a preset Bezier curve model to fit the camera motion trajectory of the real motion blurred image according to the camera pose corresponding to the real motion blurred image includes:

[0036] The expression of the Bezier curve model is:

[0037]

[0038] Among them, T t represents the camera motion trajectory, T j Represents the control points of the Bezier curve, Represents the basis function of the Bezier curve, which represents the combination coefficient and defines the weight of each control point in the curve, log(T j ) represents the Lie algebra logarithmic mapping of the camera pose, exp() represents the exponential mapping of the Lie group, which means mapping the elements in the Lie algebra back to the elements in the group, and u represents the sampling parameter, which is used to interpolate at different positions on the Bezier curve.

[0039] Optionally, the step of sampling clear images at different time points along the camera motion trajectory to construct a clear image sequence, and constructing a synthetic motion blurred image based on the camera poses corresponding to the clear images at different time points and the exposure time of the camera motion trajectory includes:

[0040] The synthetic motion blurred image is fitted from the clear image, and its expression is:

[0041]

[0042] Among them, B is the synthetic motion blurred image, I is the clear image, and P τ is the camera pose, I(P τ ) represents the camera pose P τ A clear image, where the exposure time τ = [τ o ,τ c ], τ o Indicates the start time of the exposure time interval for motion blur, τ c Indicates the end time of the exposure time interval for motion blur.

[0043] Optionally, the step of performing surface estimation on the clear image corresponding to the real motion blurred image based on ray tracing, transparency accumulation, and depth regularization constraints to restore a two-dimensional Gaussian point rendering scene of the clear image corresponding to the real motion blurred image includes:

[0044] The camera emits a ray from each pixel and performs depth estimation along the path of the ray. Each ray intersects with multiple elliptical Gaussian patches, where the Gaussian patches represent the local area of ​​the object. The intersection of the ray and the Gaussian patch will be used as a potential candidate point for surface estimation;

[0045] When the ray intersects the Gaussian patch, each intersection point has a transparency value. The transparency values ​​of all intersection points are gradually accumulated along the ray direction. When the accumulated transparency reaches 0.5, the intersection point is regarded as the surface of the object to determine the depth value corresponding to the ray, where the depth value represents the distance from the object surface to the camera. The transparency value ranges from 0 to 1.

[0046] Deep regularization constraints are used to constrain the compactness of the distribution between Gaussian patches on rays and constrain the normals of adjacent Gaussian patches, so that the distribution of Gaussian points tends to be closer to the surface of the object;

[0047] Based on the ray tracing, transparency accumulation and depth regularization constraint, a two-dimensional Gaussian point rendering scene of a clear image corresponding to the real motion blurred image is restored.

[0048] A scene surface reconstruction device based on image motion blur removal and two-dimensional Gaussian point rendering is provided to adapt to another purpose of the present application, including:

[0049] a camera trajectory fitting module configured to respond to a scene surface reconstruction event, obtain a camera pose and Gaussian point cloud parameters corresponding to a real motion blurred image, and use a preset Bezier curve model to fit the camera motion trajectory of the real motion blurred image based on the camera pose corresponding to the real motion blurred image;

[0050] a blurred image synthesis module, configured to sample clear images at different time points along the camera motion trajectory to construct a clear image sequence, and construct a synthesized motion blurred image based on the camera poses corresponding to the clear images at different time points and the exposure time of the camera motion trajectory, wherein the clear image is a differentiable rendering of a two-dimensional Gaussian point rendering scene;

[0051] a motion deblurring module configured to calculate and determine a photometric loss between the synthesized motion blurred image and the real motion blurred image, a smoothness loss, a normal consistency loss, and a depth consistency loss between the clear images of adjacent frames in the clear image sequence, and perform gradient backpropagation based on the photometric loss, smoothness loss, normal consistency loss, and depth consistency loss to adjust the camera pose and Gaussian point cloud parameters of the real motion blurred image to determine a clear image corresponding to the real motion blurred image;

[0052] a surface estimation module configured to perform surface estimation on the clear image corresponding to the real motion blurred image based on ray tracing, transparency accumulation, and depth regularization constraints, so as to restore a two-dimensional Gaussian point rendering scene of the clear image corresponding to the real motion blurred image, wherein the two-dimensional Gaussian point rendering scene includes the surface depth and the surface shape of the object;

[0053] The scene surface reconstruction module is configured to use a preset truncated signed distance function to reconstruct the scene surface of the clear image corresponding to the real motion blurred image according to the surface depth and surface shape of the object, so as to complete the scene surface reconstruction based on image de-motion blur and two-dimensional Gaussian point rendering.

[0054] An electronic device provided to meet another purpose of the present application includes a central processing unit and a memory, wherein the central processing unit is used to call and run a computer program stored in the memory to execute the steps of the scene surface reconstruction method based on image de-motion blur and two-dimensional Gaussian point rendering described in the present application.

[0055] A computer-readable storage medium is provided to meet another purpose of the present application, which stores a computer program implemented according to the scene surface reconstruction method based on image de-motion blur and two-dimensional Gaussian point rendering in the form of computer-readable instructions. When the computer program is called and executed by a computer, the steps included in the corresponding method are executed.

[0056] Compared with the existing technology, this application addresses the shortcomings of the existing technology in terms of non-uniform blur caused by depth of field changes, especially when the camera motion trajectory is complex and the scene depth changes significantly. Traditional methods have difficulty in accurately reconstructing scene surface and depth information. This application includes but is not limited to the following beneficial effects:

[0057] This application, by combining the Bezier curve model and Gaussian point cloud parameters, can accurately reconstruct the scene surface, extract the camera pose and clear image corresponding to the real motion blurred image, thereby improving the image clarity and reconstruction accuracy. By jointly optimizing the photometric loss, smoothness loss, normal consistency loss and depth consistency loss, motion blur can be effectively eliminated and the clear image sequence can be accurately restored. In addition, based on ray tracing and transparency accumulation, combined with depth regularization constraints, the realism of scene reconstruction is enhanced, and ultimately high-quality deblurring effects and accurate two-dimensional Gaussian point rendering scene reconstruction are achieved. It is suitable for surface reconstruction of dynamic scenes and augmented reality applications to solve the shortcomings of traditional technologies in terms of non-uniform blur caused by depth of field changes, especially when the camera motion trajectory is complex and the scene depth changes significantly, traditional methods are difficult to accurately reconstruct the scene surface and depth information. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0059] Figure 1 Schematic diagram of the process of a scene surface reconstruction method based on image motion deblurring and two-dimensional Gaussian point rendering in an embodiment of the present application;

[0060] Figure 2 This is a schematic diagram of a process for determining a clear image corresponding to a real motion blurred image based on the total loss in an embodiment of the present application;

[0061] Figure 3 A schematic diagram of a process for restoring a clear image corresponding to a real motion blurred image in a two-dimensional Gaussian point rendering scene according to an embodiment of the present application;

[0062] Figure 4 This is a principle block diagram of a scene surface reconstruction device based on image motion deblurring and two-dimensional Gaussian point rendering in an embodiment of the present application;

[0063] Figure 5 Schematic diagram of the structure of the computer device in the embodiment of the present application. DETAILED DESCRIPTION

[0064] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and are not to be construed as limiting the present application.

[0065] It will be understood by those skilled in the art that, unless expressly stated otherwise, the singular forms "a", "an", "said" and "the" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, "connected" or "coupled" as used herein may include wireless connections or wireless couplings. The term "and / or" used herein includes all or any units and all combinations of one or more associated listed items.

[0066] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with their meanings in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0067] It will be understood by those skilled in the art that the terms "client," "terminal," and "terminal device" as used herein include both devices that are wireless signal receivers, i.e., devices that only have wireless signal receivers without transmission capabilities, and devices that have receiving and transmitting hardware capable of two-way communication over a two-way communication link. Such devices may include: cellular or other communication devices such as personal computers and tablet computers, which have single-line displays, multi-line displays, or cellular or other communication devices without multi-line displays; PCS (Personal Communications Service), which may combine voice, data processing, fax, and / or data communication capabilities; PDA (Personal Digital Assistant), which may include a radio frequency receiver, a pager, Internet / Intranet access, a web browser, a notepad, a calendar, and / or a GPS (Global Positioning System) receiver; and conventional laptop and / or palmtop computers or other devices, which have and / or include a radio frequency receiver. As used herein, the terms "client," "terminal," or "terminal device" may be portable, transportable, installed in a vehicle (air, sea, and / or land), or adapted and / or configured to operate locally, and / or in a distributed manner, at any other location on Earth and / or in space. As used herein, the terms "client," "terminal," or "terminal device" may also refer to a communication terminal, an Internet terminal, or a music / video playback terminal, such as a PDA, an MID (Mobile Internet Device), and / or a mobile phone with a music / video playback function, or may refer to a smart TV, a set-top box, or other device.

[0068] The hardware referred to by names such as "server", "client", and "service node" in this application is essentially an electronic device with capabilities equivalent to those of a personal computer. It is a hardware device that has the necessary components revealed by the von Neumann principle, such as a central processing unit (including an arithmetic unit and a controller), a memory, an input device, and an output device. Computer programs are stored in its memory, and the central processing unit loads the program stored in the external memory into the internal memory for execution, executes the instructions in the program, and interacts with the input and output devices to complete specific functions.

[0069] It should be noted that the concept of "server" referred to in this application can also be extended to server clusters. Based on the network deployment principles understood by those skilled in the art, the servers described should be logically divided. In physical space, these servers can be independent of each other but callable through interfaces, or integrated into a single physical computer or a computer cluster. Those skilled in the art should understand this flexibility and should not use it to constrain the implementation of the network deployment method of this application.

[0070] Unless expressly specified, one or more technical features of the present application can be deployed on a server for implementation and accessed by a client through a remote call to obtain an online service interface provided by the server, or can be directly deployed and run on a client for implementation.

[0071] Unless expressly specified otherwise, the neural network models referenced or may be referenced in this application may be deployed on a remote server and remotely called on the client, or may be deployed and directly called on a client with sufficient device capabilities. In some embodiments, when it runs on the client, its corresponding intelligence may be obtained through transfer learning to reduce the requirements for the client's hardware operating resources and avoid excessive occupation of the client's hardware operating resources.

[0072] Unless explicitly specified, the various data involved in this application can be stored remotely on a server or on a local terminal device, as long as they are suitable for being called by the technical solution of this application.

[0073] Those skilled in the art should be aware that although the various methods of this application are described based on the same concept and thus exhibit commonality, unless otherwise specified, these methods can be independently executed. Similarly, the various embodiments disclosed in this application are all based on the same inventive concept. Therefore, concepts with the same expression, as well as concepts that are appropriately transformed for convenience despite different expression, should be understood as equivalent.

[0074] Unless expressly stated to be mutually exclusive, the various embodiments disclosed in this application may be cross-combined with the relevant technical features of the various embodiments to flexibly construct new embodiments, as long as such combination does not deviate from the creative spirit of this application and can meet the needs of the prior art or resolve certain deficiencies in the prior art. Those skilled in the art should be aware of such flexibility.

[0075] See also Figure 1 In one embodiment, the scene surface reconstruction method based on image motion deblurring and two-dimensional Gaussian point rendering of the present application includes:

[0076] Step S10: In response to a scene surface reconstruction event, obtaining a camera pose and Gaussian point cloud parameters corresponding to the real motion blurred image, and using a preset Bezier curve model to fit the camera motion trajectory of the real motion blurred image according to the camera pose corresponding to the real motion blurred image;

[0077] The image scene surface reconstruction system in the terminal device can respond to the scene surface reconstruction event, obtain the camera pose and Gaussian point cloud parameters corresponding to the real motion blurred image, and use a preset Bezier curve model to fit the camera motion trajectory of the real motion blurred image according to the camera pose corresponding to the real motion blurred image, wherein motion blur is usually caused by the relative movement of the camera during the shooting process, resulting in image blur.

[0078] In some embodiments, before fitting the camera motion trajectory of the real motion blurred image using a preset Bezier curve model based on the camera pose corresponding to the real motion blurred image, the motion blurred image is preprocessed using the COLMAP tool to establish a sparse point cloud and camera pose corresponding to the real motion blurred image. COLMAP is a powerful open source image reconstruction and 3D reconstruction tool widely used in computer vision, robotics, photogrammetry, and computer graphics. It focuses on reconstructing the camera pose and dense and sparse 3D structure of the scene from a set of images and is currently one of the most popular and advanced multi-view stereo vision (MVS) reconstruction tools. COLMAP supports recovering the camera pose and sparse 3D point cloud of the scene from multiple input images.

[0079] In some embodiments, the step of using a preset Bezier curve model to fit the camera motion trajectory of the real motion blurred image according to the camera pose corresponding to the real motion blurred image includes:

[0080] The expression of the Bezier curve model is:

[0081]

[0082] Among them, T t represents the camera motion trajectory, T j Represents the control points of the Bezier curve, Represents the basis function of the Bezier curve, which represents the combination coefficient and defines the weight of each control point in the curve, log(T j) represents the Lie algebra logarithmic mapping of the camera pose, exp() represents the exponential mapping of the Lie group, which represents mapping the elements in the Lie algebra back to the elements in the group, and u represents the sampling parameter used to interpolate at different positions on the Bezier curve. Specifically, in the Bezier curve, the parameter u varies between [0,1], indicating a smooth transition from the initial position to the final position of the camera. For each sampling position u, according to the basis function of the Bezier curve Calculate the corresponding camera pose. Then, use the Lie algebra logarithmic mapping to convert these poses into elements in the Lie algebra space, and then convert them back to the actual camera pose through the exponential mapping of the Lie group. In this way, the system can gradually fit the complete motion trajectory of the camera. After completing the Bezier curve fitting, the system will obtain a smooth camera motion trajectory T t , this trajectory represents the path of the camera's movement throughout the entire process. This motion trajectory can be used to compensate for the original motion-blurred image, thereby restoring the image's clarity for further tasks such as image scene surface reconstruction.

[0083] Step S20: Sampling clear images at different time points along the camera motion trajectory to construct a clear image sequence, and constructing a synthetic motion blurred image based on the camera poses corresponding to the clear images at different time points and the exposure time of the camera motion trajectory, wherein the clear image is a differentiable rendering of a two-dimensional Gaussian point rendering scene;

[0084] Using a preset Bezier curve model to fit the camera motion trajectory of the real motion blurred image according to the camera pose corresponding to the real motion blurred image, sampling clear images at different time points on the camera motion trajectory to construct a clear image sequence, and constructing a synthetic motion blurred image based on the camera poses corresponding to the clear images at different time points and the exposure time of the camera motion trajectory, wherein the clear image is a differentiable rendering of a two-dimensional Gaussian point rendering scene;

[0085] In some embodiments, the steps of sampling clear images at different time points along the camera motion trajectory to construct a clear image sequence, and constructing a synthetic motion blurred image based on the camera poses corresponding to the clear images at different time points and the exposure time of the camera motion trajectory include:

[0086] The synthetic motion blurred image is fitted from the clear image, and its expression is:

[0087]

[0088] Among them, B is the synthetic motion blurred image, I is the clear image, and P τ is the camera pose, I(P τ) represents the camera pose P τ A clear image, where the exposure time τ = [τ o ,τ c ], τ o Indicates the start time of the exposure time interval for motion blur, τ c Indicates the end time of the exposure time interval for motion blur.

[0089] Specifically, after fitting the camera motion trajectory of the real motion blurred image using a preset Bezier curve model according to the camera pose corresponding to the real motion blurred image, the corresponding camera pose can be determined by different time points on the trajectory. Based on the fitted camera motion trajectory, clear images at different time points are sampled on the motion trajectory, and these time points represent clear images at different exposure times. The camera pose corresponding to each clear image sequence is based on the sampling point of the motion trajectory. Since motion blur is caused by the rapid movement of the camera, multiple clear images are superimposed to generate a motion blurred image. It should be noted that these clear images are modeled using two-dimensional Gaussian point rendering scene rendering to ensure that they are differentiable renderings.

[0090] Step S30: calculating and determining the photometric loss between the synthesized motion blurred image and the real motion blurred image, the smoothness loss, the normal consistency loss, and the depth consistency loss between the clear images of adjacent frames in the clear image sequence, and performing gradient backpropagation according to the photometric loss, the smoothness loss, the normal consistency loss, and the depth consistency loss to adjust the camera pose and Gaussian point cloud parameters of the real motion blurred image to determine a clear image corresponding to the real motion blurred image;

[0091] Sampling clear images at different time points on the camera motion trajectory to construct a clear image sequence, constructing a synthetic motion blurred image based on the camera poses corresponding to the clear images at different time points and the exposure time of the camera motion trajectory, calculating and determining the photometric loss between the synthetic motion blurred image and the real motion blurred image, the smoothness loss, the normal consistency loss, and the depth consistency loss between the clear images of adjacent frames in the clear image sequence, performing gradient backpropagation based on the photometric loss, the smoothness loss, the normal consistency loss, and the depth consistency loss to adjust the camera pose and Gaussian point cloud parameters of the real motion blurred image, so as to determine a clear image corresponding to the real motion blurred image;

[0092] In some embodiments, the luminosity loss between the synthesized motion blurred image and the real motion blurred image is expressed as follows:

[0093] L image =(1-λdssim )|I blur -I gt |1+λ dssim (1-SSIM(I blur ,I gt )),

[0094] Among them, L image represents the photometric loss between the synthetic motion blurred image and the real motion blurred image, λ dssim is the similarity loss weight value, I blur represents the synthesized motion blurred image, I gt Represents a real motion blurred image, SSIM(I blur ,I gt ) is used to measure the similarity between the synthesized motion blurred image and the real motion blurred image, |I blur -I gt |1 represents the pixel difference between the synthetic motion blurred image and the real motion blurred image, (1-λ dssim )|I blur -I gt |1 represents the pixel-level loss term, λ dssim (1-SSIM(I blur ,I gt )) represents the structural similarity loss term;

[0095] The smoothness loss between the clear images of adjacent frames in the clear image sequence is expressed as follows:

[0096]

[0097] Among them, L smooth represents the smoothing loss between the clear images of adjacent frames in the clear image sequence, N represents the total number of pixels in the image sequence, i is the pixel index, which represents the position of each pixel in the image, j represents the frame index, which represents the subframe in the clear image sequence, represents the value of the i-th pixel in the j-th frame;

[0098] The normal consistency loss between the clear images of adjacent frames in the clear image sequence is expressed as follows:

[0099]

[0100] Among them, L normal It is expressed as the normal consistency loss between the clear images of adjacent frames in the clear image sequence, represents the normal rendered at point j of the i-th clear image sequence, represents the normal calculated at point j of the i-th clear image sequence, ωi represents the weight of the i-th clear image sequence;

[0101] The depth consistency loss between the clear images of adjacent frames in the clear image sequence is expressed as follows:

[0102]

[0103] Among them, L dist represents a depth consistency loss between clear images of adjacent frames in the clear image sequence; represents the depth value of the i-th clear image sequence at point j, represents the depth value of the i-th clear image sequence at point k, ω i represents the weight of the i-th clear image sequence, ω j The weight of the i-th clear image sequence at point j, ω k Represents the weight of the i-th clear image sequence at point k.

[0104] For further examples, please refer to Figure 2 The step of performing gradient back propagation according to the photometric loss, smoothness loss, normal consistency loss, and depth consistency loss to adjust the camera pose and Gaussian point cloud parameters of the real motion blurred image to determine a clear image corresponding to the real motion blurred image includes:

[0105] Step S301: Obtain a first weight corresponding to the photometric loss between the synthesized motion blurred image and the real motion blurred image, a second weight corresponding to the smoothness loss between the clear images of adjacent frames in the clear image sequence, a third weight corresponding to the normal consistency loss between the clear images of adjacent frames in the clear image sequence, and a fourth weight corresponding to the depth consistency loss between the clear images of adjacent frames in the clear image sequence;

[0106] Step S302: Calculate and determine a first product between the photometric loss and the first weight, a second product between the smoothness loss and the second weight, a third product between the normal consistency loss and the third weight, and a fourth product between the depth consistency loss and the fourth weight;

[0107] Step S303: Calculate and determine the sum of the first product, the second product, the third product, and the fourth product to determine a total loss between the synthesized motion blurred image and the clear image;

[0108] Step S304: performing gradient back propagation based on the total loss between the synthesized motion blurred image and the clear image to adjust the camera pose and Gaussian point cloud parameters of the real motion blurred image to determine the clear image corresponding to the real motion blurred image.

[0109] Specifically, the total loss between the synthesized motion blurred image and the clear image is expressed as:

[0110] L total =λ image L image +λ smooth(t) L smooth +λ normal(t) L normal +

[0111] λ dist(t) L dist ,

[0112] Among them, λ image represents a first weight corresponding to the photometric loss between the synthesized motion blurred image and the real motion blurred image, λ smooth(t) represents the second weight corresponding to the smoothness loss between the clear images of adjacent frames in the clear image sequence, λ normal(t) represents the third weight corresponding to the normal consistency loss between the clear images of adjacent frames in the clear image sequence, λ dist(t) represents the fourth weight corresponding to the depth consistency loss between the clear images of adjacent frames in the clear image sequence, λ smooth(t) ,λ normal(t) and λ dist(t) It changes with the number of iterations t.

[0113] More specifically, once the total loss L is calculated total , gradient backpropagation can be performed. Through the backpropagation algorithm, the camera pose and Gaussian point cloud parameters in the network are optimized according to the total loss. The specific process is as follows: Calculate the gradient of the camera pose and point cloud parameters in the model according to the total loss. Update the camera pose and point cloud parameters through optimization algorithms (such as SGD, Adam, etc.) to reduce the value of the loss function. Repeat the gradient backpropagation multiple times, update the parameters in each iteration, and gradually reduce the motion blur. Through the iterative optimization of the above process, the best match between the synthesized motion blurred image and the clear image can be determined, and the corresponding clear image can be restored. This process simultaneously optimizes the parameters of the camera pose and the three-dimensional point cloud, so that the quality of the clear image is gradually improved. After several optimization iterations, a clear image with motion blur removed is finally obtained, and the result is output. At this point, all loss terms have been effectively optimized.

[0114] Step S40: performing surface estimation on the clear image corresponding to the real motion blurred image based on ray tracing, transparency accumulation, and depth regularization constraints to restore a two-dimensional Gaussian point rendering scene of the clear image corresponding to the real motion blurred image, wherein the two-dimensional Gaussian point rendering scene includes the surface depth and surface shape of the object;

[0115] Performing gradient backpropagation according to the photometric loss, smoothness loss, normal consistency loss, and depth consistency loss to adjust the camera pose and Gaussian point cloud parameters of the real motion blurred image to determine a clear image corresponding to the real motion blurred image, and then performing surface estimation on the clear image corresponding to the real motion blurred image based on ray tracing, transparency accumulation, and depth regularization constraints to restore a two-dimensional Gaussian point rendering scene of the clear image corresponding to the real motion blurred image, wherein the two-dimensional Gaussian point rendering scene includes object surface depth and object surface shape;

[0116] In some embodiments, see Figure 3 The step of performing surface estimation on the clear image corresponding to the real motion blurred image based on ray tracing, transparency accumulation, and depth regularization constraints to restore a two-dimensional Gaussian point rendering scene of the clear image corresponding to the real motion blurred image includes:

[0117] Step S401: The camera emits a ray from each pixel and performs depth estimation along the path of the ray. Each ray intersects with multiple elliptical Gaussian patches, where the Gaussian patches represent local areas of the object. The intersection points of the ray and the Gaussian patches are used as potential candidate points for surface estimation.

[0118] Step S402: When the ray intersects the Gaussian patch, each intersection point has a transparency value. The transparency values ​​of all intersection points along the ray direction are gradually accumulated. When the accumulated transparency reaches 0.5, the intersection point is regarded as the surface of the object to determine the depth value corresponding to the ray. The depth value represents the distance from the object surface to the camera, and the transparency value range is between 0 and 1.

[0119] Step S403: Using depth regularization constraints to constrain the compactness of the distribution of Gaussian patches on the ray and constrain the normals of adjacent Gaussian patches, so that the distribution of Gaussian points tends to be closer to the surface of the object;

[0120] Step S404 : Based on the ray tracing, transparency accumulation, and depth regularization constraint, a two-dimensional Gaussian point rendering scene of a clear image corresponding to the real motion blurred image is restored.

[0121] This embodiment effectively extracts surface geometry from real motion-blurred images through ray tracing, transparency accumulation, and depth regularization constraints, restoring a clear, two-dimensional Gaussian point-rendered scene. This approach combines the accuracy of ray tracing, the feasibility of transparency accumulation, and the optimization power of depth regularization to significantly enhance the accuracy and efficiency of scene surface reconstruction.

[0122] In some embodiments, a Gaussian patch generally refers to a Gaussian function that is used to distribute weights on a given area to achieve the purpose of smoothing, filtering, or approximating the area.

[0123] Step S50: Using a preset truncated signed distance function to reconstruct the scene surface of the clear image corresponding to the real motion blurred image according to the surface depth and shape of the object, so as to complete the scene surface reconstruction based on image de-motion blurring and two-dimensional Gaussian point rendering.

[0124] The surface of the clear image corresponding to the real motion blurred image is estimated based on ray tracing, transparency accumulation, and depth regularization constraints to restore the two-dimensional Gaussian point rendering scene of the clear image corresponding to the real motion blurred image. Then, a preset truncated signed distance function is used to reconstruct the scene surface of the clear image corresponding to the real motion blurred image based on the object surface depth and object surface shape, thereby completing the scene surface reconstruction based on image motion deblurring and two-dimensional Gaussian point rendering. The object surface depth refers to the spatial distance between each point on the object surface and the camera. The object surface shape refers to the geometric characteristics of each point on the object surface, such as curvature, shape contour, etc.

[0125] Specifically, the truncated signed distance function (TSDF) in the Open3D library is used to reconstruct the surface of the scene. TSDF is widely used in 3D reconstruction. Its main function is to reconstruct the complete three-dimensional object surface by calculating the distance to the nearest surface for each point in the scene. The truncated signed distance function (TSDF) avoids excessive noise and unnecessary calculations by limiting the distance of each point within a certain threshold, thereby improving the efficiency and accuracy of the reconstruction process. The surface depth and surface shape of the object will be used as input, and the truncated signed distance function (TSDF) will calculate the three-dimensional reconstruction data of the scene based on the surface depth and surface shape of the object. Ultimately, an accurate three-dimensional model can be obtained, representing a clear scene after removing motion blur.

[0126] As can be seen from the above embodiments, compared with the prior art, the present application addresses the shortcomings of the prior art in terms of non-uniform blur caused by depth of field changes. In particular, when the camera motion trajectory is complex and the scene depth changes significantly, traditional methods are difficult to accurately reconstruct the scene surface and depth information. The present application has, but is not limited to, the following beneficial effects:

[0127] This application, by combining the Bezier curve model and Gaussian point cloud parameters, can accurately reconstruct the scene surface, extract the camera pose and clear image corresponding to the real motion blurred image, thereby improving the image clarity and reconstruction accuracy. By jointly optimizing the photometric loss, smoothness loss, normal consistency loss and depth consistency loss, motion blur can be effectively eliminated and the clear image sequence can be accurately restored. In addition, based on ray tracing and transparency accumulation, combined with depth regularization constraints, the realism of scene reconstruction is enhanced, and ultimately high-quality deblurring effects and accurate two-dimensional Gaussian point rendering scene reconstruction are achieved. It is suitable for surface reconstruction of dynamic scenes and augmented reality applications to solve the shortcomings of traditional technologies in terms of non-uniform blur caused by depth of field changes, especially when the camera motion trajectory is complex and the scene depth changes significantly, traditional methods are difficult to accurately reconstruct the scene surface and depth information.

[0128] See also Figure 4A scene surface reconstruction device based on image motion deblurring and two-dimensional Gaussian point rendering is provided to meet one of the purposes of the present application, including a camera trajectory fitting module 1100, a blurred image synthesis module 1200, a motion deblurring module 1300, a surface estimation module 1400 and a scene surface reconstruction module 1500. Among them, the camera trajectory fitting module 1100 is configured to respond to the scene surface reconstruction event, obtain the camera pose and Gaussian point cloud parameters corresponding to the real motion blurred image, and use a preset Bezier curve model to fit the camera motion trajectory of the real motion blurred image according to the camera pose corresponding to the real motion blurred image; the blurred image synthesis module 1200 is configured to sample clear images at different time points on the camera motion trajectory to construct a clear image sequence, and construct a synthesized motion blurred image according to the camera pose corresponding to the clear image at different time points and the exposure time of the camera motion trajectory, wherein the clear image is a differentiable rendering of a two-dimensional Gaussian point rendering scene; the motion deblurring module 1300 is configured to calculate and determine the photometric loss between the synthesized motion blurred image and the real motion blurred image, the smoothness loss, the normal consistency loss and the depth loss between the clear images of adjacent frames in the clear image sequence. The invention relates to a method for reconstructing a scene surface of a clear image corresponding to the real motion blurred image by performing a surface estimation on the clear image corresponding to the real motion blurred image based on ray tracing, transparency accumulation and depth regularization constraints, and performing gradient backpropagation according to the photometric loss, smoothness loss, normal consistency loss and depth consistency loss to adjust the camera pose and Gaussian point cloud parameters of the real motion blurred image, so as to determine the clear image corresponding to the real motion blurred image; a surface estimation module 1400 is configured to perform surface estimation on the clear image corresponding to the real motion blurred image based on ray tracing, transparency accumulation and depth regularization constraints to restore the two-dimensional Gaussian point rendering scene of the clear image corresponding to the real motion blurred image, wherein the two-dimensional Gaussian point rendering scene includes the surface depth and the surface shape of the object; a scene surface reconstruction module 1500 is configured to perform scene surface reconstruction on the clear image corresponding to the real motion blurred image according to the surface depth and the surface shape of the object using a preset truncated signed distance function to complete the scene surface reconstruction based on image de-motion blurring and two-dimensional Gaussian point rendering.

[0129] Based on any embodiment of this application, please refer to Figure 5 Another embodiment of the present application further provides an electronic device, which can be implemented by a computer device, such as Figure 5As shown, a schematic diagram of the internal structure of a computer device. The computer device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. Among them, the computer-readable storage medium of the computer device stores an operating system, a database, and computer-readable instructions, and the database may store a control information sequence. When the computer-readable instructions are executed by the processor, the processor can implement a scene surface reconstruction method based on image de-motion blur and two-dimensional Gaussian point rendering. The processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device. The memory of the computer device may store computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor can execute the scene surface reconstruction method based on image de-motion blur and two-dimensional Gaussian point rendering of the present application. The network interface of the computer device is used to connect and communicate with the terminal. Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0130] In this embodiment, the processor is used to execute the specific functions of each module and its submodule in 4, and the memory stores the program code and various data required to execute the above modules and submodules. The network interface is used to transmit data between user terminals or servers. The memory in this embodiment stores the program code and data required to execute all modules and submodules in the scene surface reconstruction device based on image motion deblurring and two-dimensional Gaussian point rendering of this application. The server can call the server's program code and data to execute the functions of all submodules.

[0131] The present application also provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the scene surface reconstruction method based on image de-motion blur and two-dimensional Gaussian point rendering described in any embodiment of the present application.

[0132] The present application also provides a computer program product, including a computer program / instruction, which, when executed by one or more processors, implements the steps of the scene surface reconstruction method based on image de-motion blurring and two-dimensional Gaussian point rendering described in any embodiment of the present application.

[0133] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments of the present application can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above-mentioned embodiments of the method. The aforementioned storage medium can be a computer-readable storage medium such as a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0134] The above description is only part of the implementation methods of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

[0135] In summary, this application, by combining the Bezier curve model and Gaussian point cloud parameters, can accurately reconstruct the scene surface, extract the camera pose and clear image corresponding to the real motion blurred image, thereby improving the image clarity and reconstruction accuracy. By jointly optimizing the photometric loss, smoothness loss, normal consistency loss and depth consistency loss, motion blur can be effectively eliminated and the clear image sequence can be accurately restored. In addition, based on ray tracing and transparency accumulation, combined with depth regularization constraints, the realism of scene reconstruction is enhanced, and ultimately high-quality deblurring effects and accurate two-dimensional Gaussian point rendering scene reconstruction are achieved. It is suitable for surface reconstruction of dynamic scenes and augmented reality applications to solve the shortcomings of traditional technologies in terms of non-uniform blur caused by depth of field changes, especially when the camera motion trajectory is complex and the scene depth changes significantly, traditional methods are difficult to accurately reconstruct the scene surface and depth information.

Claims

1. A scene surface reconstruction method based on image motion blur removal and two-dimensional Gaussian point rendering, characterized in that: include: In response to a scene surface reconstruction event, obtaining a camera pose and Gaussian point cloud parameters corresponding to a real motion blurred image, and using a preset Bezier curve model to fit a camera motion trajectory of the real motion blurred image according to the camera pose corresponding to the real motion blurred image; Sampling clear images at different time points along the camera motion trajectory to construct a clear image sequence, and constructing a synthetic motion blurred image based on the camera poses corresponding to the clear images at different time points and the exposure time of the camera motion trajectory, wherein the clear image is a differentiable rendering of a two-dimensional Gaussian point rendering scene; Calculating and determining a photometric loss between the synthesized motion blurred image and the real motion blurred image, a smoothness loss, a normal consistency loss, and a depth consistency loss between the clear images of adjacent frames in the clear image sequence, and performing gradient backpropagation based on the photometric loss, smoothness loss, normal consistency loss, and depth consistency loss to adjust the camera pose and Gaussian point cloud parameters of the real motion blurred image to determine a clear image corresponding to the real motion blurred image; performing surface estimation on the clear image corresponding to the real motion blurred image based on ray tracing, transparency accumulation, and depth regularization constraints to restore a two-dimensional Gaussian point rendering scene of the clear image corresponding to the real motion blurred image, wherein the two-dimensional Gaussian point rendering scene includes object surface depth and object surface shape; A preset truncated signed distance function is used to reconstruct the scene surface of the clear image corresponding to the real motion blurred image according to the surface depth and surface shape of the object, so as to complete the scene surface reconstruction based on image de-motion blur and two-dimensional Gaussian point rendering.

2. The scene surface reconstruction method based on image motion deblurring and two-dimensional Gaussian point rendering according to claim 1, characterized in that: The luminosity loss between the synthesized motion blurred image and the real motion blurred image is expressed as: ， in, represents the photometric loss between the synthetic motion blurred image and the real motion blurred image, is the similarity loss weight value, represents the synthesized motion blurred image, represents a real motion blurred image, Used to measure the similarity between the synthesized motion blurred image and the real motion blurred image, represents the pixel difference between the synthesized motion blurred image and the real motion blurred image, represents the pixel-level loss term, Represents the structural similarity loss term; The smoothness loss between the clear images of adjacent frames in the clear image sequence is expressed as follows: ， in, represents the smoothness loss between the clear images of adjacent frames in the clear image sequence, represents the total number of pixels in the image sequence, is the pixel index, which represents the position of each pixel in the image, represents a frame index, which represents a subframe in a clear image sequence, Indicates the Frame No. The value of the pixel; The normal consistency loss between the clear images of adjacent frames in the clear image sequence is expressed as follows: ， in, It is expressed as the normal consistency loss between the clear images of adjacent frames in the clear image sequence, Indicates the A clear image sequence at point Normals rendered on Indicates the A clear image sequence at point The normal calculated above is Indicates the The weight of a clear image sequence; The depth consistency loss between the clear images of adjacent frames in the clear image sequence is expressed as follows: ， in, represents a depth consistency loss between clear images of adjacent frames in the clear image sequence; Indicates the A clear image sequence at point The depth value at Indicates the A clear image sequence at point The depth value at Indicates the The weight of a clear image sequence, Indicates the A clear image sequence at point The weight of Indicates the A clear image sequence at point The weight of .

3. The scene surface reconstruction method based on image motion deblurring and two-dimensional Gaussian point rendering according to claim 1, characterized in that: The step of performing gradient back propagation according to the photometric loss, smoothness loss, normal consistency loss, and depth consistency loss to adjust the camera pose and Gaussian point cloud parameters of the real motion blurred image to determine a clear image corresponding to the real motion blurred image includes: Obtaining a first weight corresponding to a photometric loss between the synthesized motion blurred image and the real motion blurred image, a second weight corresponding to a smoothness loss between clear images of adjacent frames in the clear image sequence, a third weight corresponding to a normal consistency loss between clear images of adjacent frames in the clear image sequence, and a fourth weight corresponding to a depth consistency loss between clear images of adjacent frames in the clear image sequence; Calculating and determining a first product between the photometric loss and the first weight, a second product between the smoothness loss and the second weight, a third product between the normal consistency loss and the third weight, and a fourth product between the depth consistency loss and the fourth weight; calculating and determining a sum of the first product, the second product, the third product, and the fourth product to determine a total loss between the synthesized motion blurred image and the clear image; Gradient back propagation is performed based on the total loss between the synthesized motion blurred image and the clear image to adjust the camera pose and Gaussian point cloud parameters of the real motion blurred image to determine the clear image corresponding to the real motion blurred image.

4. The scene surface reconstruction method based on image motion deblurring and two-dimensional Gaussian point rendering according to claim 3, characterized in that: The total loss between the synthesized motion blurred image and the sharp image is expressed as: ， in, a first weight representing a photometric loss between the synthesized motion blurred image and the real motion blurred image, represents a second weight corresponding to a smoothness loss between clear images of adjacent frames in the clear image sequence, represents a third weight corresponding to the normal consistency loss between the clear images of adjacent frames in the clear image sequence, represents a fourth weight corresponding to a depth consistency loss between clear images of adjacent frames in the clear image sequence, 、 and With the number of iterations Change and vary.

5. The scene surface reconstruction method based on image motion deblurring and two-dimensional Gaussian point rendering according to claim 1, characterized in that: The step of using a preset Bezier curve model to fit the camera motion trajectory of the real motion blurred image according to the camera pose corresponding to the real motion blurred image includes: The expression of the Bezier curve model is: ， in, represents the camera motion trajectory, Represents the control points of the Bezier curve, Represents the basis function of the Bezier curve, which represents the combination coefficient and defines the weight of each control point in the curve. The Lie algebraic logarithmic mapping representing the camera pose, represents the exponential map of the Lie group, which maps elements in the Lie algebra back to elements in the group, Represents the sampling parameters used to interpolate at different positions on the Bezier curve.

6. The scene surface reconstruction method based on image motion deblurring and two-dimensional Gaussian point rendering according to claim 1, characterized in that: The steps of sampling clear images at different time points on the camera motion trajectory to construct a clear image sequence, and constructing a synthetic motion blurred image according to the camera poses corresponding to the clear images at different time points and the exposure time of the camera motion trajectory include: The synthetic motion blurred image is fitted from the clear image, and its expression is: ， in, is the synthetic motion blurred image, It is a clear image. is the camera pose, Indicates the camera pose A clear image, where the exposure time belong , Indicates the start time of the exposure time interval for motion blur, Indicates the end time of the exposure time interval for motion blur.

7. The scene surface reconstruction method based on image motion deblurring and two-dimensional Gaussian point rendering according to any one of claims 1 to 6, characterized in that: The step of performing surface estimation on the clear image corresponding to the real motion blurred image based on ray tracing, transparency accumulation, and depth regularization constraints to restore a two-dimensional Gaussian point rendering scene of the clear image corresponding to the real motion blurred image includes: The camera emits a ray from each pixel and performs depth estimation along the path of the ray. Each ray intersects with multiple elliptical Gaussian patches, where the Gaussian patches represent the local area of ​​the object. The intersection of the ray and the Gaussian patch will be used as a potential candidate point for surface estimation; When the ray intersects the Gaussian patch, each intersection point has a transparency value. The transparency values ​​of all intersection points along the ray direction are gradually accumulated. When the accumulated transparency reaches 0.5, the intersection point is regarded as the surface of the object to determine the depth value corresponding to the ray, where the depth value represents the distance from the object surface to the camera. The transparency value ranges from 0 to 1. Deep regularization constraints are used to constrain the compactness of the distribution between Gaussian patches on rays and constrain the normals of adjacent Gaussian patches, so that the distribution of Gaussian points tends to be closer to the surface of the object; Based on the ray tracing, transparency accumulation and depth regularization constraint, a two-dimensional Gaussian point rendering scene of a clear image corresponding to the real motion blurred image is restored.

8. A scene surface reconstruction device based on image motion blur removal and two-dimensional Gaussian point rendering, characterized in that: include: a camera trajectory fitting module configured to respond to a scene surface reconstruction event, obtain a camera pose and Gaussian point cloud parameters corresponding to a real motion blurred image, and use a preset Bezier curve model to fit the camera motion trajectory of the real motion blurred image based on the camera pose corresponding to the real motion blurred image; a blurred image synthesis module, configured to sample clear images at different time points along the camera motion trajectory to construct a clear image sequence, and construct a synthesized motion blurred image based on the camera poses corresponding to the clear images at different time points and the exposure time of the camera motion trajectory, wherein the clear image is a differentiable rendering of a two-dimensional Gaussian point rendering scene; a motion deblurring module configured to calculate and determine a photometric loss between the synthesized motion blurred image and the real motion blurred image, a smoothness loss, a normal consistency loss, and a depth consistency loss between the clear images of adjacent frames in the clear image sequence, and perform gradient backpropagation based on the photometric loss, smoothness loss, normal consistency loss, and depth consistency loss to adjust the camera pose and Gaussian point cloud parameters of the real motion blurred image to determine a clear image corresponding to the real motion blurred image; a surface estimation module configured to perform surface estimation on the clear image corresponding to the real motion blurred image based on ray tracing, transparency accumulation, and depth regularization constraints, so as to restore a two-dimensional Gaussian point rendering scene of the clear image corresponding to the real motion blurred image, wherein the two-dimensional Gaussian point rendering scene includes the surface depth and the surface shape of the object; The scene surface reconstruction module is configured to use a preset truncated signed distance function to reconstruct the scene surface of the clear image corresponding to the real motion blurred image according to the surface depth and surface shape of the object, so as to complete the scene surface reconstruction based on image de-motion blur and two-dimensional Gaussian point rendering.

9. An electronic device comprising a central processing unit and a memory, characterized in that: The central processing unit is configured to call and run a computer program stored in the memory to execute the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that It stores a computer program implemented according to the method described in any one of claims 1 to 7 in the form of computer-readable instructions, and when the computer program is called and executed by a computer, the steps included in the corresponding method are executed.

Images