Image processing method and device, electronic equipment and storage medium

Abstract

The invention provides an image processing method and device, electronic equipment and a storage medium. The image processing method comprises the following steps: acquiring scene images of a target scene at different visual angles; obtaining an interferent mask matrix; obtaining a modified scene image; establishing a three-dimensional point cloud; converting each three-dimensional point in the three-dimensional point cloud into a three-dimensional Gaussian primitive to obtain a three-dimensional Gaussian primitive set; projecting the three-dimensional Gaussian primitive to the plane of the scene image according to the camera parameter to obtain a two-dimensional Gaussian projection, and reducing the contribution weight of the two-dimensional Gaussian projection overlapped with the interferent mask matrix to the generation of a rendered image under the view angle of the camera parameter; and generating a rendered image under the camera parameters according to the set of the three-dimensional Gaussian primitives and the contribution weight, and adjusting the parameters of the three-dimensional Gaussian primitives according to the non-interferent areas of the rendered image and the scene image. According to the invention, through identifying the interferent in the scene image, the influence of the interferent in the scene image on three-dimensional modeling is removed.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more particularly to an image processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] Scene reconstruction technology is an important research direction in the field of computer vision and graphics, and it is widely used in virtual reality, augmented reality, autonomous driving, and urban modeling. Traditional neural rendering methods (such as NeRF, Neural Radiance Fields) achieve high-fidelity 3D scene reconstruction by synthesizing new perspectives from multi-view 2D images, but they suffer from problems such as slow rendering speed and limited control over specific components of the scene.

[0003] In recent years, 3D Gaussian Splatting (3DGS) has become an important alternative to NeRF. It explicitly represents a scene using a set of colored 3D Gaussian primitives, combined with differentiable sputtering to achieve efficient rasterization. While maintaining high-quality reconstruction, it significantly improves training and rendering speed. 3DGS encodes a scene as a collection of 3D Gaussian primitives, each containing parameters such as position, opacity, and scale. Through viewpoint transformation, these 3D Gaussian primitives are projected onto a 2D Gaussian map, and then the image is rendered using alpha blending.

[0004] Real-world images often contain obstructions such as pedestrians, leaves, and snowflakes. These objects not only obscure key content and reduce visual quality but also cause inconsistencies in multi-view observations, affecting downstream tasks (such as object detection and dynamic scene analysis). Therefore, removing dynamic small objects and preserving the integrity of static regions during reconstruction has become a critical requirement. Summary of the Invention

[0005] This disclosure provides an image processing method, apparatus, electronic device, and storage medium.

[0006] The following technical solution is adopted in this disclosure.

[0007] In some embodiments, this disclosure provides an image processing method, comprising:

[0008] Acquire scene images of the target scene from different perspectives;

[0009] Obtain the interference mask matrix of the scene image;

[0010] The pixels in the area covered by the interference mask matrix in the scene image are set to be transparent, while the pixels in the remaining areas are set to be opaque, thereby obtaining the modified scene image;

[0011] Based on the modified scene image, camera parameters are estimated and a 3D point cloud is constructed.

[0012] Each 3D point in the 3D point cloud is converted into a 3D Gaussian element to obtain a set of 3D Gaussian elements;

[0013] The three-dimensional Gaussian primitive is projected onto the plane of the scene image according to the camera parameters to obtain a two-dimensional Gaussian projection, and the contribution weight of the two-dimensional Gaussian projection that overlaps with the interference mask matrix to the generated rendering image is reduced from the perspective of the camera parameters.

[0014] A rendered image under the camera parameters is generated based on the set of three-dimensional Gaussian primitives and the contribution weights, and the parameters of the three-dimensional Gaussian primitives are adjusted based on the non-interference areas of the rendered image and the scene image.

[0015] In some embodiments, this disclosure provides an image processing apparatus, comprising:

[0016] The acquisition unit is used to acquire scene images of the target scene from different perspectives;

[0017] A processing unit is used to obtain the interference mask matrix of the scene image;

[0018] The processing unit is also configured to set the pixels in the area covered by the interference mask matrix in the scene image to be transparent, and the pixels in the remaining areas to be opaque, thereby obtaining the modified scene image;

[0019] The processing unit is also used to estimate camera parameters and build a 3D point cloud based on the modified scene image;

[0020] The processing unit is also used to convert each three-dimensional point in the three-dimensional point cloud into three-dimensional Gaussian primitives to obtain a set of three-dimensional Gaussian primitives;

[0021] The processing unit is further configured to project the three-dimensional Gaussian primitive onto the plane of the scene image according to the camera parameters to obtain a two-dimensional Gaussian projection, and reduce the contribution weight of the two-dimensional Gaussian projection that overlaps with the interference mask matrix to the generated rendered image from the perspective of the camera parameters.

[0022] The processing unit is further configured to generate a rendered image under the camera parameters based on the set of three-dimensional Gaussian primitives and the contribution weights, and adjust the parameters of the three-dimensional Gaussian primitives based on the non-interference areas of the rendered image and the scene image.

[0023] In some embodiments, this disclosure provides an electronic device, including: at least one memory and at least one processor;

[0024] The memory is used to store program code, and the processor is used to call the program code stored in the memory to execute the above method.

[0025] In some embodiments, this disclosure provides a computer-readable storage medium for storing program code that, when executed by a processor, causes the processor to perform the methods described above.

[0026] In some embodiments of this disclosure, by identifying and removing distracting objects in the scene image, the impact of randomly distributed dynamic distracting objects on 3D modeling is improved. Attached Figure Description

[0027] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0028] Figure 1 This is a flowchart of an image processing method according to some embodiments of the present disclosure.

[0029] Figure 2 This is a schematic diagram illustrating the effect of setting the interfering object to be transparent and non-transparent on three-dimensional point clouds and images according to some embodiments of this disclosure.

[0030] Figure 3 This is a schematic diagram of an image processing method according to some embodiments of the present disclosure.

[0031] Figure 4 This is a schematic diagram comparing the effect of this method in some embodiments of the present disclosure with other methods.

[0032] Figure 5 This is a schematic diagram of the structure of an electronic device according to some embodiments of the present disclosure. Detailed Implementation

[0033] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0034] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0035] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0036] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0037] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0038] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0039] It should be understood that the various steps described in the method embodiments of this disclosure can be performed in sequence and / or in parallel. Furthermore, method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0040] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0041] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0042] It should be noted that the use of the word "a" in this disclosure is illustrative rather than restrictive, and those skilled in the art should understand that it should be understood as "one or more" unless otherwise expressly indicated in the context.

[0043] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0044] The solutions provided by the embodiments of this disclosure will be described in detail below with reference to the accompanying drawings.

[0045] Multi-view images of a scene can be used to generate a 3D model of the scene, such as 3D Gaussian primitives. However, scene images often contain distracting elements such as pedestrians, fallen leaves, and snowflakes. These distractions not only obscure key content and reduce visual quality, but also, because they are dynamic, their positions differ across images, leading to inconsistencies in multi-view observations and affecting modeling quality and downstream tasks. Therefore, removing these dynamic distractions and preserving static areas during modeling is a common requirement for improving modeling quality.

[0046] The following problems exist in the related technologies:

[0047] It struggles to handle diverse and dispersed interference. Related technologies are mainly designed for removing single objects, and have low accuracy in recognizing randomly distributed small objects (such as leaves and snowflakes), making them prone to missed or false detections.

[0048] Multiview Figure 1 Poor consistency: Implicit methods (such as NeRF-based methods) and some 3DGS-based methods do not make full use of multi-view association, resulting in blurry and inconsistent local area reconstruction after the removal of interference objects.

[0049] Efficiency and quality are difficult to balance: NeRF-based methods reconstruct high quality but are slow to render (training a single scene takes 30 to 40 minutes), while native 3DGS is fast but cannot effectively remove interference.

[0050] Insufficient optimization of interference areas: Related technologies have weak ability to restore static area details in areas covered by interference, and are prone to texture loss or artifacts.

[0051] In some embodiments of this disclosure, diverse and scattered small dynamic disturbances in multi-view images can be efficiently identified and removed while ensuring real-time rendering speed and static region reconstruction quality, and at the same time improving the multi-view performance of disturbance areas. Figure 1 Consistency and completeness of details.

[0052] like Figure 1 As shown, Figure 1 This is a flowchart of an image processing method according to an embodiment of the present disclosure, which includes the following steps.

[0053] S11. Obtain scene images of the target scene from different perspectives.

[0054] In some embodiments, the method proposed in this disclosure can be implemented by an electronic device, such as a computer, tablet, or server. The target scene can be an indoor or outdoor scene, such as the area where a building is located. Color images of the target scene are captured from different perspectives to obtain scene images. There are multiple scene images. The scene images can be represented as I1 to I k , k represents the k-th scene image, which includes distracting objects such as snowflakes, pedestrians, and fallen leaves.

[0055] S12. Obtain the interference mask matrix of the scene image.

[0056] In some embodiments, there are multiple scene images, and different scene images have their own interference mask matrix D. k Let k represent the k-th scene image. These object masks form an object mask matrix, which is used to label the pixels of the object in each scene image. Each object mask matrix D... k It is a picture I k The binary image has the same resolution (containing only 0 and 1). In the binary image, the position of a certain value indicates that the pixel in the scene image at that position is a static background (non-interference area), which should be retained for reconstructing the 3D model. The position of a different value in the binary image indicates that the pixel in the scene image at that position is an interference (interference area), which needs to be removed.

[0057] In some embodiments of this disclosure, obtaining the interference mask matrix of a scene image includes: obtaining reference interference features; segmenting the scene image to generate object features and object masks for all objects in the scene image; calculating the similarity between the object features and the reference interference features; determining the interference in the scene image based on the similarity; and generating the interference mask matrix of the scene image based on the object masks of the interference.

[0058] Specifically, an image encoder, such as the SegmentAnythingModel-v2 (SAMv2) image encoder, can be used to extract features from each scene image. The user provides N point prompts P1, P2, ..., PN, where each point Pj is a point manually clicked by the user on a reference distractor (e.g., a pedestrian, a snowflake), allowing the user to select a small subset of distractors in the scene image as reference distractors. Utilizing SAMv2's interactive segmentation capabilities, a corresponding binary segmentation mask Mj∈{0,1} is generated for each point prompt Pj. H×W Here, 1 represents the pixel region of the interfering object, and H×W represents the resolution of the scene image. All these masks form a set M. ref ={M j} Nj=1 This is called the reference interference mask. Then, the reference interference features are obtained based on the reference mask Mref={Mj} and the features of the scene image. Then, the reference interference features are obtained through the reference interference mask M... ref The feature F of the scene image generates the feature F of the reference interference map. ref Specifically, each M can be... j After element-wise multiplication with F, a new feature map is obtained, in which only the region covered by Mj retains the original features, and the rest of the region is set to zero. Then, for all non-zero positions (|Mj) ref The average of the pixels with a mask of 1 (|) is taken to obtain a vector representing the average feature of the j-th reference interference object. Then, the average features of all N reference interference objects are averaged again to obtain the final reference feature F. ref The specific calculation formula is as follows:

[0059]

[0060] It is important to note that reference interference objects are categorized by type, and reference features are calculated for each category. The scene image is segmented using full-image grid point cues to obtain the object mask M for all objects in the scene image. N and object features F N Then use the reference interference feature F ref With object feature F N Calculate the pre-selected similarity between the two objects. If the cosine similarity between the object features of one object and the features of a reference interfering object is above a preset threshold (e.g., 0.6 to 0.8, specifically 0.7), then the object is considered an interfering object in the scene image. The object masks of interfering objects in a scene image constitute the interfering object mask matrix D of that scene image. k This identifies the pixel location (distraction area) of the interference objects in the scene image, that is, it represents the area where interference objects such as snowflakes, pedestrians, and fallen leaves are located in the scene image. Each scene image has its own corresponding binary interference mask matrix.

[0061] S13. Set the pixels in the area covered by the interference mask matrix in the scene image to be transparent, and the pixels in the remaining areas to be opaque, thereby obtaining the modified scene image.

[0062] In some embodiments, the area covered by the interference mask matrix refers to the region where the interference is located, that is, the pixels at the location of the interference in the scene image, while the remaining area is the non-interference region. The pixels in the interference region of the scene image are set to transparent, for example, by setting the pixels in the interference region to (0,0,0), or this can be achieved by adjusting the transparency, while the non-interference regions remain opaque. This way, the interference region will be skipped during subsequent 3D point cloud generation, avoiding incorrect matching caused by the introduction of interference.

[0063] S14. Estimate camera parameters and build a 3D point cloud based on the modified scene image.

[0064] In some embodiments, the modified scene image can be input into the COLMAP tool, which can reconstruct camera parameters and sparse 3D point clouds. Because the interference areas are transparent, multi-view matching utilizes only information from static regions (non-interference areas), avoiding the generation of erroneous point clouds. If the transparency of the interference areas is not set to 0, multiple cameras may generate incorrect information in these areas, leading to inaccurate reconstruction. By setting the interference areas to transparent, the reconstructed 3D point clouds yield satisfactory results. Figure 2 The effects of setting the interference region to transparent and setting it to opaque (marking only the interference region) on the 3D point cloud and the image regenerated later from 3D Gaussian primitives are shown. It can be seen that if the interference region is not set to transparent, ghosting or holes appear in the 3D point cloud and the image, resulting in a significant decrease in quality.

[0065] S15. Convert each 3D point in the 3D point cloud into a 3D Gaussian primitive to obtain a set of 3D Gaussian primitives.

[0066] In some embodiments, each 3D point in the 3D point cloud is converted into a 3D Gaussian unit G. i The three-dimensional Gaussian element includes the position p i (x) i、 y i、 z i Opacity α i The covariance matrix ∑ obtained by rotation R and scaling S decomposition i (∑) i =RSS T R T ) as a parameter. G i It can be represented as:

[0067]

[0068] Here, x represents a point in space, and Gi is the 3D distribution in space. The set of 3D Gaussian primitives represents the 3D model of the scene, where most interfering elements have been excluded, but a small number of residual elements may require further processing.

[0069] S16. Based on the camera parameters, project the three-dimensional Gaussian primitives onto the plane of the scene image to obtain the two-dimensional Gaussian projection, and reduce the contribution weight of the two-dimensional Gaussian projection that overlaps with the interference mask matrix to the generated rendered image from the perspective of the camera parameters.

[0070] S17. Generate a rendered image under camera parameters based on the set of 3D Gaussian primitives and contribution weights, and adjust the parameters of the 3D Gaussian primitives according to the non-interference areas of the rendered image and the scene image.

[0071] In some embodiments, to preserve the efficient rasterization capabilities of 3DGS while ensuring that interference areas are not rendered and that static areas are reconstructed with high quality, the shape of the 3D Gaussian primitives projected from the world coordinate system onto the plane of the scene view needs to be calculated (i.e., the 2D covariance Σ′). The 2D covariance Σ′ of each 3D Gaussian primitive in the camera coordinate system is calculated using the viewpoint transformation W (W is a rigid body transformation from the world coordinate system to the camera coordinate system) and the Jacobian matrix J, where Σ′ = JW∑W T J T The 2D covariance ∑′ determines the 2D Gaussian projection of the 3D Gaussian primitives onto the image. The shape, in two-dimensional Gaussian projection, can be represented as follows (x' is the center of the 3D Gaussian element p). i (2D projection center point after W transformation)

[0072]

[0073] 3DGS is based on a 3D Gaussian model with center point p. i The distances to the planes of the scene image are sorted in depth order and then further processed using fast alpha blending, as follows:

[0074]

[0075] This formula is used to calculate the final rendered color C(x) at any pixel position x in the rendered image, where N is the number of three-dimensional Gaussian units, and c i Let α be the Gaussian color of the i-th 3D Gaussian element, which is related to the viewpoint. i Let T be the opacity of the i-th 3D Gaussian element. i Cumulative transmittance represents the proportion of light that has not been absorbed by preceding Gaussian elements when it reaches the i-th 3D Gaussian element from infinity, and 1−wj represents the proportion of light that is not blocked by the j preceding 3D Gaussian elements.i Let be the contribution weight of the two-dimensional Gaussian projection of the i-th three-dimensional Gaussian element to the pixel position x. In some embodiments of this disclosure, the contribution weight is calculated by the projected two-dimensional Gaussian projection and the interference mask matrix D. k The contribution weight is adjusted based on whether there is overlap. Specifically, in some embodiments, if the projected 2D Gaussian projection overlaps with the interference mask matrix, the contribution weight w of the 2D Gaussian projection to the generated rendered image under the viewpoint of the camera parameters is adjusted. i Setting it to 0 means that the 2D Gaussian projection does not contribute to the generated rendered image from the current viewpoint. If the 2D Gaussian projection does not overlap with the object mask matrix, the contribution weight w of the 2D Gaussian projection to the generated rendered image from the viewpoint of the camera parameters is set. i Set to 1; where the contribution weight describes the degree of contribution in generating the rendered image. This allows us to minimize the impact of removing distracting elements on the rendered image.

[0076] The above formula can be used to calculate the final rendered color of each pixel position in the rendered image, thus obtaining the rendered image under camera parameters (e.g., viewpoint). At this point, the non-interference areas in the scene image are used as ground truth, thereby adjusting the parameters of the 3D Gaussian primitives. The adjusted set of 3D Gaussian primitives becomes the 3D model of the target scene, thus allowing the generation of images of the target scene from any specified viewpoint based on the set of 3D Gaussian primitives.

[0077] In some embodiments of this disclosure, such as Figure 3 The diagram illustrates a flowchart of the method proposed in this embodiment. By identifying and removing distracting elements in a scene image, the method improves the impact of randomly distributed dynamic distractions on 3D modeling. In some embodiments, based on reference distraction features, cosine similarity matching is used to automatically identify diverse and dispersed distractions without requiring an additional classifier. In some embodiments, setting distraction regions to transparent improves the accuracy of reconstructed 3D point clouds and avoids incorrect matching introduced by distractions. In some embodiments, the two-dimensional Gaussian projection of distraction regions is directly removed by contribution weights, suppressing the influence of distractions in real time.

[0078] In some embodiments of this disclosure, adjusting the parameters of the three-dimensional Gaussian unit based on the non-interference regions of the rendered image and the scene image includes: calculating a loss function between the non-interference regions in the rendered image and the scene image; and adjusting the parameters of the three-dimensional Gaussian unit to reduce the loss function.

[0079] In some embodiments, the rendered image is a color image of the target scene after removing distracting elements, while the non-distracting areas in the scene image serve as a real reference. The difference between the two indicates whether the 3D Gaussian primitives accurately reflect the actual target scene. Then, the parameters of the 3D Gaussian primitives are adjusted, the rendered image is regenerated, and the loss function is calculated. The entire process is iterative: during iteration, the Gaussian primitives are adjusted, the rendered image is regenerated, and the loss function is calculated again, completing one iteration. This iterative process continues until the loss function is minimized or the required number of iterations is reached.

[0080] In some embodiments of this disclosure, the loss function includes: refining loss L RMAE Multi-view dynamic loss L MVDL and structural loss L D-SSIM Refining loss L RMAE Used to describe the color error between the rendered image and non-interfering areas; Multi-view dynamic loss L MVDL Used to describe the consistency of the same region in images from different viewpoints; structural loss L D-SSIM Used to describe the structural similarity between a rendered image and a scene image.

[0081] In some embodiments, refining loss L RMAE The formula for calculating it is as follows:

[0082]

[0083] in, α j Let α be the opacity of the 3D Gaussian element j. k Let N(x) be the opacity of the Gaussian elements preceding j, N(x) be the set of three-dimensional Gaussian elements covering pixel x, N be the set of pixels in the non-interference region of the scene image, |N| be the pixel count of this set, and I / M represent the non-interference region. To render the color of the image at pixel x0, C(x0) represents the color of the scene image at pixel x0. A refined loss is used to accurately fit the color in non-disturbing areas, while global opacity information is used for weighting to prevent Gaussian drift into distracting areas.

[0084] In some embodiments, the multi-view dynamic loss L MVDL The calculation formula is as follows:

[0085] , ,

[0086] in, n represents the set of pixels in the interference region. kL represents the number of non-interference areas, and s is the scaling factor (which can be set to 1.2). Interference objects (such as pedestrians) typically only appear in part of the viewpoint. For a pixel x∈I that is occluded by an interference object, other viewpoints may see the non-interference area behind it. MVDL This cross-view visibility is used to guide the 3D model to recover non-interference areas consistent with other viewpoints within interference areas. MVDL The interference area is optimized by using multi-view association, and the contribution of the visible area is compensated by dynamic weight α.

[0087] L D−SSIM Measure the structural similarity between rendered images and real information to improve overall visual consistency.

[0088] In some embodiments, the loss function L total Equal to refining loss L RMAE Multi-view dynamic loss L MVDL and structural loss L D-SSIM The weighted sum. L total =L RMAE +λ1L MVDL +λ2L D−SSIM λ represents the weight. This loss function can improve the reconstruction accuracy of non-interference areas and the restoration of static details in interference areas.

[0089] In some embodiments of this disclosure, a hybrid loss function design is used, combining RMAE (optimizing non-interference regions) and multi-view dynamic loss (enhancing cross-view dynamics). Figure 1 This method combines D-SSIM (for improved visual quality) and 3DGS (for improved consistency) to synergistically enhance reconstruction accuracy. In some embodiments, this method is based on the 3DGS framework, maintaining the efficiency of training (5-30 minutes per scene) and rendering (100+ FPS), which is superior to NeRF-based methods.

[0090] In some embodiments of this disclosure, confetti was scattered in indoor and outdoor scenes and videos were recorded. These real distractors were captured using an Apple iPhone 16, with videos lasting 2 minutes and a resolution of 1276×1276. The videos were then split into 20 to 30 images, with each image labeled as a distractor. All images were jointly processed using Colmap to recover camera parameters in a shared coordinate system. This embodiment compares the proposed method with IDDR-NGP (Huang et al., 2023), GS-Inpainting, SpotLessSplats (Sabour et al., 2024), and two desnow baseline methods. IDDR-NGP is the first method to directly remove distractors on Instant-NGP. GS-Inpainting is a custom baseline method that fills distractor regions using a latent diffusion model (Rombach et al., 2022) and then retrains a Gaussian Splatting model on top of that. SpotLessSplats is a recent state-of-the-art work on outlier removal. This embodiment employs... Figure 1 The method described herein further specifies that if the 2D Gaussian projection overlaps with the object mask matrix, the contribution weight of the 2D Gaussian projection to the generated rendered image under the viewpoint of the camera parameters is set to 0; if the 2D Gaussian projection does not overlap with the object mask matrix, the contribution weight of the 2D Gaussian projection to the generated rendered image under the viewpoint of the camera parameters is set to 1. Furthermore, in this embodiment, adjusting the parameters of the 3D Gaussian primitives based on the non-interference regions of the rendered image and the scene image includes: calculating a loss function between the non-interference regions in the rendered image and the scene image; and adjusting the parameters of the 3D Gaussian primitives to reduce the loss function. The loss function includes: refinement loss, multi-view dynamic loss, and structural loss.

[0091] Figure 4This document demonstrates the results of re-rendering distraction-free images using SpotlessSplats, GS-Inpainting, IDDR-NGP, and the proposed method (SSOR-GS) after inputting a scene image with distractors (GT). All experiments were conducted on an RTX 3090 GPU with 24GB of memory. This embodiment synthesizes several types of distractors (e.g., snowflakes, petals, and fallen leaves) on multiple public datasets, including Mip-NeRF360 (Barron et al., 2022), 3D Gaussian Splatting (Kerbl et al., 2023b), and the Blender dataset (Mildenhall et al., 2021). Furthermore, to evaluate the effectiveness of SSOR-GS on real-world distractors, the proposed method (SSOR-GS) is compared with SpotLessSplats, GS-Inpainting, and IDDR-NGP from the same viewpoint. The comparison areas are enlarged within the red boxes. Among them, the overall reconstruction quality of SpotLessSplats (SLS) is blurry and inconsistent. Figure 4 In the garden and truck scenes, only a small number of distracting elements (such as petals and leaves) were removed. This is likely because these distracting elements are larger and more numerous compared to the typically smaller scattered paper scraps. Furthermore, the pre-trained diffusion and classification network of SLS performed poorly in identifying synthetic distracting elements, such as failing to remove the largest leaves in the truck scene. In contrast, native GS-Inpainting generated photorealistic rendering results and relatively consistent visual effects, demonstrating the powerful ability of Gaussian splashing to aggregate multi-view information. Although GS-Inpainting and IDDR-NGP rendered paper scraps almost imperceptibly in both indoor and outdoor scenes, the detail and texture were inferior to the method in this embodiment (SSOR-GS) in outdoor scenes with marble patterns. Overall, the reconstruction quality of the method in this embodiment (SSOR-GS) is comparable to IDDR-NGP, but the rendering speed is significantly better, with IDDR-NGP taking approximately 30-40 minutes per scene for training. Figure 4 The average time was 37 minutes, while the method in this embodiment only took 11 minutes.

[0092] In this embodiment, accurate identification and removal of both synthetic (snowflakes, petals) and realistic (paper scraps, pedestrians) distraction scenes are achieved. The peak signal-to-noise ratio of the method in this embodiment is improved by 14.2% to 21.5% compared to the SpotLessSplats model. Through multi-view dynamic loss, the cross-view reconstruction consistency of distraction regions is significantly improved, with a structural similarity index of 0.919 on the LLFF (LocalLightFieldFusion) dataset, which is 5.4% higher than the existing SnowFormer model (Chen et al., 2022a). The rendering speed of the method in this embodiment reaches 103 FPS, close to the speed of native 3DGS, and the training time is only 1 / 3 to 1 / 4 of that of the IDDR-NGP model, while maintaining high reconstruction quality. In complex scenes such as marble textures, this method can still retain the fine texture of the static scene after distraction removal, which is better than models such as GS-Inpainting. In summary, the method in this embodiment effectively solves the problems of low efficiency, poor consistency, and loss of detail in the removal of small objects, providing a practical solution for high-quality 3D reconstruction of dynamic scenes.

[0093] This disclosure also proposes an image processing apparatus, comprising:

[0094] The acquisition unit is used to acquire scene images of the target scene from different perspectives;

[0095] The processing unit is used to obtain the interference mask matrix of the scene image;

[0096] The processing unit is also used to set the pixels in the area covered by the interference mask matrix in the scene image to be transparent, and the pixels in the remaining areas to be opaque, so as to obtain the modified scene image.

[0097] The processing unit is also used to estimate camera parameters and build a 3D point cloud based on the modified scene image;

[0098] The processing unit is also used to convert each 3D point in the 3D point cloud into 3D Gaussian primitives to obtain a set of 3D Gaussian primitives;

[0099] The processing unit is also used to project the three-dimensional Gaussian primitives onto the plane of the scene image according to the camera parameters to obtain the two-dimensional Gaussian projection, and to reduce the contribution weight of the three-dimensional Gaussian primitives corresponding to the two-dimensional Gaussian projection that overlaps with the interference mask matrix to the generated rendering image from the perspective of the camera parameters.

[0100] The processing unit is also used to generate a rendered image under camera parameters based on the set of three-dimensional Gaussian primitives and contribution weights, and to adjust the parameters of the three-dimensional Gaussian primitives based on the non-interference areas of the rendered image and the scene image.

[0101] In some embodiments, obtaining the interference mask matrix of the scene image includes: obtaining reference interference features; segmenting the scene image to generate object features and object masks for all objects in the scene image; calculating the similarity between the object features and the reference interference features; determining interference objects in the scene image based on the similarity; and generating the interference mask matrix of the scene image based on the object masks of the interference objects.

[0102] In some embodiments, obtaining reference interference features includes: acquiring features of the scene image using an image encoder; determining the reference interference selected by the user and generating a reference mask for the reference interference; and obtaining the reference interference features based on the reference mask and the features of the scene image.

[0103] In some embodiments, reducing the contribution weight of the two-dimensional Gaussian projection that overlaps with the interference mask matrix to the generated rendered image under the viewpoint of the camera parameters includes: if the two-dimensional Gaussian projection overlaps with the interference mask matrix, setting the contribution weight of the two-dimensional Gaussian projection to the generated rendered image under the viewpoint of the camera parameters to 0; if the two-dimensional Gaussian projection does not overlap with the interference mask matrix, setting the contribution weight of the two-dimensional Gaussian projection to the generated rendered image under the viewpoint of the camera parameters to 1.

[0104] In some embodiments, adjusting the parameters of the three-dimensional Gaussian unit based on the non-interference regions of the rendered image and the scene image includes: calculating a loss function between the non-interference regions in the rendered image and the scene image; and adjusting the parameters of the three-dimensional Gaussian unit to reduce the loss function.

[0105] In some embodiments, the loss function includes: refinement loss, multi-view dynamic loss, and structural loss; the refinement loss is used to describe the color error between the rendered image and the non-interference area; the multi-view dynamic loss is used to describe the consistency of the same area in images from different viewpoints; and the structural loss is used to describe the structural similarity between the rendered image and the scene image.

[0106] In some embodiments, after adjusting the parameters of the three-dimensional Gaussian elements, the processing unit is further configured to: generate an image from a specified viewpoint based on the set of three-dimensional Gaussian elements.

[0107] For embodiments of the apparatus, since they basically correspond to the method embodiments, relevant details can be found in the descriptions of the method embodiments. The apparatus embodiments described above are merely illustrative, and the modules described as separate modules may or may not be separate. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0108] The methods and apparatus of this disclosure have been described above based on embodiments and application examples. Furthermore, this disclosure also provides an electronic device and a computer-readable storage medium, which are described below.

[0109] The following is for reference. Figure 5 The figure illustrates a structural schematic of an electronic device (e.g., a terminal device or server) 800 suitable for implementing embodiments of the present disclosure. The terminal device in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. The electronic device shown in the figure is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present disclosure.

[0110] Electronic device 800 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 802 or a program loaded from storage device 808 into random access memory (RAM) 803. RAM 803 also stores various programs and data required for the operation of electronic device 800. The processing device 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0111] Typically, the following devices can be connected to I / O interface 805: input devices 806 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 807 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 808 including, for example, magnetic tapes, hard disks, etc.; and communication devices 809. Communication device 809 allows electronic device 800 to communicate wirelessly or wiredly with other devices to exchange data. Although an electronic device 800 with various devices is shown in the figure, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0112] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 809, or installed from a storage device 808, or installed from a ROM 802. When the computer program is executed by a processing device 801, it performs the functions defined in the methods of embodiments of this disclosure.

[0113] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0114] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0115] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0116] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods of the present disclosure.

[0117] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0118] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0119] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0120] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0121] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0122] According to one or more embodiments of this disclosure, an image processing method is provided, comprising:

[0123] Acquire scene images of the target scene from different perspectives;

[0124] Obtain the interference mask matrix of the scene image;

[0125] The pixels in the area covered by the interference mask matrix in the scene image are set to be transparent, while the pixels in the remaining areas are set to be opaque, thereby obtaining the modified scene image;

[0126] Based on the modified scene image, camera parameters are estimated and a 3D point cloud is constructed.

[0127] Each 3D point in the 3D point cloud is converted into a 3D Gaussian element to obtain a set of 3D Gaussian elements;

[0128] The three-dimensional Gaussian primitive is projected onto the plane of the scene image according to the camera parameters to obtain a two-dimensional Gaussian projection, and the contribution weight of the two-dimensional Gaussian projection that overlaps with the interference mask matrix to the generated rendering image is reduced from the perspective of the camera parameters.

[0129] A rendered image under the camera parameters is generated based on the set of three-dimensional Gaussian primitives and the contribution weights, and the parameters of the three-dimensional Gaussian primitives are adjusted based on the non-interference areas of the rendered image and the scene image.

[0130] According to one or more embodiments of this disclosure, an image processing method is provided to obtain an interference mask matrix of the scene image, including:

[0131] Obtain the characteristics of the reference interference;

[0132] Segment the scene image to generate object features and object masks for all objects in the scene image;

[0133] Calculate the similarity between the object features and the reference interference features;

[0134] Based on the similarity, interference objects in the scene image are determined;

[0135] An interference object mask matrix is ​​generated for the scene image based on the object mask of the interference object.

[0136] According to one or more embodiments of this disclosure, an image processing method is provided to obtain reference interference features, including:

[0137] The scene image is obtained using an image encoder;

[0138] Determine the reference interfering object selected by the user, and generate a reference mask for the reference interfering object;

[0139] The reference interference features are obtained based on the reference mask and the features of the scene image.

[0140] According to one or more embodiments of this disclosure, an image processing method is provided to reduce the contribution weight of the two-dimensional Gaussian projection overlapping with the interference mask matrix to the generated rendered image under the viewpoint of the camera parameters, including:

[0141] If the two-dimensional Gaussian projection overlaps with the interference mask matrix, the contribution weight of the two-dimensional Gaussian projection to the generated rendered image under the viewpoint of the camera parameters is set to 0.

[0142] If the two-dimensional Gaussian projection does not overlap with the interference mask matrix, the contribution weight of the two-dimensional Gaussian projection to the generated rendered image under the viewpoint of the camera parameters is set to 1.

[0143] According to one or more embodiments of this disclosure, an image processing method is provided, which adjusts the parameters of the three-dimensional Gaussian elements based on the non-interference regions of the rendered image and the scene image, including:

[0144] Calculate the loss function between the rendered image and the non-interference region in the scene image;

[0145] The parameters of the three-dimensional Gaussian primitive are adjusted to reduce the loss function.

[0146] According to one or more embodiments of this disclosure, an image processing method is provided, wherein the loss function includes: refinement loss, multi-view dynamic loss, and structural loss;

[0147] The refining loss is used to describe the color error between the rendered image and the non-interference area;

[0148] The multi-view dynamic loss is used to describe the consistency of the same region in images from different viewpoints.

[0149] The structural loss is used to describe the structural similarity between the rendered image and the scene image.

[0150] According to one or more embodiments of this disclosure, an image processing method is provided, which, after adjusting the parameters of the three-dimensional Gaussian elements, further includes:

[0151] An image at a specified viewpoint is generated based on the set of three-dimensional Gaussian elements.

[0152] According to one or more embodiments of the present disclosure, an image processing apparatus is provided, comprising:

[0153] The acquisition unit is used to acquire scene images of the target scene from different perspectives;

[0154] A processing unit is used to obtain the interference mask matrix of the scene image;

[0155] The processing unit is also configured to set the pixels in the area covered by the interference mask matrix in the scene image to be transparent, and the pixels in the remaining areas to be opaque, thereby obtaining the modified scene image;

[0156] The processing unit is also used to estimate camera parameters and build a 3D point cloud based on the modified scene image;

[0157] The processing unit is also used to convert each three-dimensional point in the three-dimensional point cloud into three-dimensional Gaussian primitives to obtain a set of three-dimensional Gaussian primitives;

[0158] The processing unit is further configured to project the three-dimensional Gaussian primitive onto the plane of the scene image according to the camera parameters to obtain a two-dimensional Gaussian projection, and reduce the contribution weight of the two-dimensional Gaussian projection that overlaps with the interference mask matrix to the generated rendered image from the perspective of the camera parameters.

[0159] The processing unit is further configured to generate a rendered image under the camera parameters based on the set of three-dimensional Gaussian primitives and the contribution weights, and adjust the parameters of the three-dimensional Gaussian primitives based on the non-interference areas of the rendered image and the scene image.

[0160] According to one or more embodiments of the present disclosure, an electronic device is provided, including: at least one memory and at least one processor;

[0161] The at least one memory is used to store program code, and the at least one processor is used to call the program code stored in the at least one memory to execute the method described in any one of the above.

[0162] According to one or more embodiments of the present disclosure, a computer-readable storage medium is provided for storing program code that, when executed by a processor, causes the processor to perform the methods described above.

[0163] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0164] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0165] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A method of processing an image, wherein, The method comprises the following steps: acquiring scene images of a target scene at different view angles; acquiring an interference mask matrix of the scene images; setting pixels in a region covered by the interference mask matrix in the scene images as transparent, and setting pixels in the rest region as non-transparent, thereby obtaining a modified scene image; estimating camera parameters and establishing a three-dimensional point cloud according to the modified scene image; converting each three-dimensional point in the three-dimensional point cloud into a three-dimensional Gaussian primitive to obtain a set of three-dimensional Gaussian primitives; projecting the three-dimensional Gaussian primitives onto a plane of the scene image according to the camera parameters to obtain a two-dimensional Gaussian projection, and reducing a contribution weight of the two-dimensional Gaussian projection to generating a rendered image at the view angle of the camera parameters, which overlaps with the interference mask matrix; generating a rendered image at the camera parameters according to the set of three-dimensional Gaussian primitives and the contribution weight, and adjusting parameters of the three-dimensional Gaussian primitives according to the rendered image and a non-interference region of the scene image.

2. The method of claim 1, wherein, The method for acquiring the interference mask matrix of the scene images comprises the following steps: acquiring a reference interference feature; segmenting the scene images to generate object features and object masks of all objects in the scene images; calculating a similarity between the object features and the reference interference feature; determining an interference object in the scene images according to the similarity; generating the interference mask matrix of the scene images according to the object mask of the interference object.

3. The method of claim 2, wherein, The method for acquiring the reference interference feature comprises the following steps: acquiring a feature of the scene images by using an image encoder; determining a reference interference object selected by a user, and generating a reference mask of the reference interference object; obtaining the reference interference feature according to the reference mask and the feature of the scene images.

4. The method of claim 1, wherein, The method for reducing the contribution weight of the two-dimensional Gaussian projection to generating a rendered image at the view angle of the camera parameters, which overlaps with the interference mask matrix, comprises the following steps: if the two-dimensional Gaussian projection overlaps with the interference mask matrix, setting the contribution weight of the two-dimensional Gaussian projection to generating a rendered image at the view angle of the camera parameters as 0; if the two-dimensional Gaussian projection does not overlap with the interference mask matrix, setting the contribution weight of the two-dimensional Gaussian projection to generating a rendered image at the view angle of the camera parameters as 1.

5. The method of claim 1, wherein, The method for adjusting the parameters of the three-dimensional Gaussian primitives according to the rendered image and the non-interference region of the scene image comprises the following steps: calculating a loss function between the rendered image and the non-interference region in the scene images; adjusting the parameters of the three-dimensional Gaussian primitives to reduce the loss function.

6. The method according to claim 5, wherein the loss function comprises a refinement loss, a multi-view dynamic loss and a structure loss; the refinement loss is used to describe a color error between the rendered image and the non-interference region; the multi-view dynamic loss is used to describe consistency of the same region in different view angle images; the structure loss is used to describe a structural similarity between the rendered image and the scene image.

7. The method of claim 1, wherein, After adjusting the parameters of the three-dimensional Gaussian primitives, the method further comprises the following step: generating an image at a specified view angle according to the set of three-dimensional Gaussian primitives.

8. An image processing apparatus, wherein, The method comprises the following steps: an acquiring unit, configured to acquire scene images of a target scene at different view angles; a processing unit configured to obtain an interference mask matrix of the scene image; the processing unit is further configured to set pixels of a region covered by the interference mask matrix in the scene image as transparent and pixels of a remaining region as non-transparent, thereby obtaining a modified scene image; the processing unit is further configured to estimate camera parameters and establish a three-dimensional point cloud according to the modified scene image; the processing unit is further configured to convert each three-dimensional point in the three-dimensional point cloud into a three-dimensional Gaussian primitive to obtain a set of three-dimensional Gaussian primitives; the processing unit is further configured to project the three-dimensional Gaussian primitives to a plane of the scene image according to the camera parameters to obtain two-dimensional Gaussian projections, and reduce a contribution weight of the two-dimensional Gaussian projections overlapping with the interference mask matrix to a rendered image under a view angle of the camera parameters; the processing unit is further configured to generate a rendered image under the camera parameters according to the set of three-dimensional Gaussian primitives and the contribution weight, and adjust parameters of the three-dimensional Gaussian primitives according to the rendered image and a non-interference region of the scene image. 9.An electronic device, comprising: at least one memory and at least one processor; wherein the at least one memory is configured to store program code, and the at least one processor is configured to invoke the program code stored in the at least one memory to execute the method of any one of claims 1 to 7. 10.A computer-readable storage medium configured to store program code, wherein the program code, when executed by a processor, causes the processor to execute the method of any one of claims 1 to 7.

Images