Image processing method and device, electronic equipment and storage medium

Abstract

The invention provides an image processing method and device, electronic equipment and a storage medium. The method comprises the following steps: generating a three-dimensional Gaussian model of a target scene according to a plurality of scene images of the target scene; determining a mask area in the three-dimensional Gaussian model according to the two-dimensional mask of the scene image, and replacing Gaussian primitives in the mask area with randomly initialized Gaussian primitives; repairing the three-dimensional Gaussian model to generate a first repaired image; decomposing the first repaired image into blocks, and screening out target blocks; decomposing the scene image into blocks, and screening out source blocks; obtaining semantic features of a source block, and repairing the target block according to the source block and a source block semantic feature guide diffusion model to generate the target block which does not contain the target object; and updating the first repaired image by using the repaired target block to obtain a second repaired image. According to the method, the semantic conflict problem is solved, and the visual consistency and geometric rationality of cross-view are kept.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more particularly to an image processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the development of computer vision technology, 3D scene reconstruction and editing have been widely applied in fields such as augmented reality (AR), virtual reality (VR), robot navigation, and digital content creation. In recent years, the emergence of Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) technologies has greatly promoted the ability to render high-fidelity and high-efficiency 3D scenes. Among them, 3DGS, by representing scene models as a set of Gaussian units, enables real-time new perspective synthesis and editable 3D scene reconstruction, becoming an important foundation for 3D scene editing. Summary of the Invention

[0003] This disclosure provides an image processing method, apparatus, electronic device, and storage medium.

[0004] The following technical solution is adopted in this disclosure.

[0005] In some embodiments, this disclosure provides an image processing method, comprising:

[0006] A three-dimensional Gaussian model of the target scene is generated based on multiple scene images of the target scene, wherein the three-dimensional Gaussian model includes multiple Gaussian elements;

[0007] Based on the two-dimensional mask of the scene image, the mask region in the three-dimensional Gaussian model is determined and the Gaussian elements of the mask region are replaced with randomly initialized Gaussian elements. The two-dimensional mask is used to identify the target object.

[0008] The three-dimensional Gaussian model is repaired to generate a first repaired image from the same viewpoint as the scene image;

[0009] The first repaired image is decomposed into blocks, and target blocks corresponding to the area where the target object is located in the scene image are selected;

[0010] The scene image is decomposed into blocks, and source blocks with similar semantic features to the target block are selected.

[0011] Obtain the semantic features of the source block, and guide the diffusion model to repair the target block based on the source block and the semantic features of the source block to generate the target block that does not contain the target object;

[0012] The first repaired image is updated using the repaired target block to obtain the second repaired image.

[0013] In some embodiments, this disclosure provides an image processing apparatus, comprising:

[0014] A generation unit is used to generate a three-dimensional Gaussian model of a target scene based on multiple scene images of the target scene, wherein the three-dimensional Gaussian model includes multiple Gaussian elements;

[0015] The processing unit is configured to determine the mask region in the three-dimensional Gaussian model based on the two-dimensional mask of the scene image and replace the Gaussian elements of the mask region with randomly initialized Gaussian elements, wherein the two-dimensional mask is used to identify the target object;

[0016] The processing unit is also used to repair the three-dimensional Gaussian model and generate a first repaired image from the same viewpoint as the scene image;

[0017] The processing unit is further configured to decompose the first repaired image into blocks and filter out the target blocks corresponding to the area where the target object is located in the scene image;

[0018] The processing unit is further configured to decompose the scene image into blocks and filter out source blocks that have similar semantic features to the target block;

[0019] The processing unit is further configured to acquire the semantic features of the source block, and guide the diffusion model to repair the target block based on the source block and the semantic features of the source block to generate the target block that does not contain the target object;

[0020] The processing unit is further configured to update the first repaired image with the repaired target block to obtain a second repaired image.

[0021] In some embodiments, this disclosure provides an electronic device, including: at least one memory and at least one processor;

[0022] The memory is used to store program code, and the processor is used to call the program code stored in the memory to execute the above method.

[0023] In some embodiments, this disclosure provides a computer-readable storage medium for storing program code that, when run by a processor, causes the processor to perform the methods described above.

[0024] The method provided in this disclosure uses a semantically guided block matching mechanism, based on cross-view semantic feature matching such as DINOv2, to achieve semantic consistency alignment between the target block and multiple views. It makes full use of cross-view semantic features for repair, solves semantic conflicts (such as texture mismatch and structural misalignment) of the target block in different views, effectively solves the semantic conflict problem, and completes the occluded area of ​​the target object by using multi-view observation information to generate reasonable content in the unobserved area, maintaining visual consistency and geometric rationality across views. Attached Figure Description

[0025] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0026] Figure 1 This is a flowchart of an image processing method according to an embodiment of the present disclosure.

[0027] Figure 2 This is a schematic diagram of an image processing method according to an embodiment of the present disclosure.

[0028] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0029] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0030] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0031] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0032] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0033] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0034] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0035] It should be understood that the various steps described in the method embodiments of this disclosure can be performed in sequence and / or in parallel. Furthermore, method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0036] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0037] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0038] It should be noted that the use of the word "a" in this disclosure is illustrative rather than restrictive, and those skilled in the art should understand that it should be understood as "one or more" unless otherwise expressly indicated in the context.

[0039] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0040] The solutions provided by the embodiments of this disclosure will be described in detail below with reference to the accompanying drawings.

[0041] 3D object removal is a crucial task in 3D scene editing. Its goal is to remove unwanted objects from the reconstructed 3D scene and realistically repair any resulting voids. This task must satisfy the following requirements: (1) filling in areas occluded by objects using multi-view observation information; (2) generating reasonable content for unobserved occluded areas; and (3) maintaining visual consistency and geometric plausibility across all views. However, these requirements present challenges when dealing with large objects or 360° boundless environments.

[0042] The 3D target removal methods in related technologies are mainly based on NeRF (Neural Radiance Fields) or 3DGS framework, but there are still some limitations. (1) NeRF-based methods (e.g., Mirzaei, et al. SPIN-NeRF: Multiview Segmentation and Perceptual Inpainting with Neural Radiance Fields. arXiv preprint arXiv:2211.12254.) achieve target removal in 3D scenes through 2D image inpainting technology. The steps are: reconstructing the radiation field of the scene using NeRF; masking the target area and completing the single-view mask area through a 2D diffusion model; optimizing NeRF parameters based on the completion results to achieve cross-view Figure 1 Consistency. However, its over-reliance on single vision. Figure 2D information ignores the semantic consistency of multiple views, resulting in poor geometric coherence and blurred texture details in the reconstruction results. (2) 3DGS-based methods (e.g., Chen et al., Gaussianeditor: Swift and controllable 3d editing with gaussian splatting. In Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition, 21476–21485.) optimize the diffusion-guided reconstruction process through hierarchical Gaussian sputtering and semantic tracking. The steps include: initializing Gaussian primitives of the scene based on 3DGS; removing Gaussian primitives in the masked region and replacing them with randomly initialized primitives; and optimizing primitive parameters using fractional distillation sampling (SDS) to complete the target region. However, it relies on the generation prior but lacks explicit geometric guidance, which easily produces overly smooth or semantically irrelevant textures, and it does not perform iterative optimization for low-quality regions. MVInpainter (Source: Cao et al., Mvin-painter: Learning multi-view consistent in painting to bridge 2D and 3D editing. arXiv preprint arXiv:2408.08000.) guides inpainting through multi-view references. The steps are as follows: constructing reference features using multi-view cues from unmasked regions; guiding a diffusion model based on the reference features to complete the masked regions; and optimizing 3DGS parameters to maintain multi-view consistency. Figure 1 However, it struggles to restore fine geometric details, the repair results are prone to overly smoothed patches, and it does not utilize semantic information for cross-view alignment.

[0043] In related technologies, insufficient semantic consistency across multiple views and failure to fully utilize cross-view semantic information lead to semantic conflicts in the repaired area across different views (such as texture mismatch and structural misalignment). Loss of detail and over-smoothing, coupled with a lack of high-frequency feature guidance during prior generation, result in blurred textures in the repaired area, failing to preserve fine geometric details (such as edges and texture undulations). One-time repair has limitations, performing single-step completion of the entire mask area, failing to iteratively optimize local areas with poor visual quality, and easily producing local artifacts. Some methods sacrifice efficiency for quality (such as long training time and large memory consumption), making it difficult to meet real-time editing requirements.

[0044] Some embodiments of this disclosure at least partially solve the above-mentioned problems. In some embodiments of this disclosure, semantically guided cross-view matching and progressive region optimization improve the multi-view capability of 3D target removal. Figure 1 Consistency and detail fidelity.

[0045] like Figure 1 As shown, Figure 1 This is a flowchart of an image processing method according to an embodiment of the present disclosure, which includes the following steps.

[0046] S11. Generate a 3D Gaussian model of the target scene based on multiple scene images of the target scene.

[0047] In some embodiments, the scene images are from multiple (n) different perspectives π i Color images of the target scene captured by I i (i≤n), therefore the scene image consists of multiple images, and the target scene contains the target object. A 3D Gaussian model of the target scene can be generated using any image generation method. The 3D Gaussian model consists of multiple Gaussian elements, which are typically ellipsoids. Each Gaussian element is defined as G. i = (x i , r i , α i , c i ), where i represents the i-th Gaussian element, x is the 3D center coordinate, r is the rotation parameter, α is the opacity, and c is the RGB color represented by the spherical harmonic function.

[0048] S12. Based on the two-dimensional mask of the scene image, determine the mask region in the three-dimensional Gaussian model and replace the Gaussian elements of the mask region with randomly initialized Gaussian elements.

[0049] In some embodiments, the user may mark the target objects to be removed from the scene image. These target objects can be one or a group of objects in the scene image; for example, if the scene image is outdoors, the target objects could be a large rock or a large tree on the ground. A two-dimensional mask M is generated based on the user's markings. i A scene image corresponds to a two-dimensional mask, which is used to identify the target object. The two-dimensional mask can be a two-dimensional image with two values: 0 and 1. 1 indicates the area where the target object is located, and 0 indicates the background area. The two-dimensional mask determines the mask region corresponding to the target object in the 3D Gaussian model; that is, the mask region is the 3D region where the Gaussian primitives of the target object are located. Because the target object needs to be removed, the Gaussian primitives in the mask region are replaced with an equal number of randomly initialized Gaussian primitives. Further processing continues to generate a 3D Gaussian model without the target object, and the randomly initialized Gaussian primitives in the mask region are repaired to match the surrounding environment. The areas occluded by the target object also need to be repaired.

[0050] S13. Repair the three-dimensional Gaussian model to generate a first repaired image from the same viewpoint as the scene image.

[0051] In some embodiments, a preliminary repair is performed on the 3D Gaussian model. During repair, fractional distillation sampling (SDS, source: Poole, B.; Jain, A.; Barron, JT; and Mildenhall, B. 2022. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE / CVF conference on computer vision and pattern recognition, 10684–10695.) is computed from distillation knowledge in the repair backbone network (Source: Rombach, R.; Blattmann, A.; Lorenz, D.; Esser, P.; and Ommer, B. 2022. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE / CVF conference on computer vision and pattern recognition, 10684–10695.). The parameters of randomly initialized Gaussian units are optimized using the loss of the repair backbone network (Source: Poole, B.; Jain, A.; Barron, JT; and Mildenhall, B. 2022. Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209.14988., 2022). Gaussian noise is then added to the image rendered using the 3D Gaussian model, using a pre-trained diffusion model (e.g., Stable Diffusion). v2.1) Predicts noise, minimizes the difference between predicted and actual noise, and uses multi-step noise prediction to improve the rendering effect of the 3D Gaussian model, thereby completing the initial repair. After the initial repair is completed, the first repaired image is generated. It can be represented as: Where Gθ represents a three-dimensional Gaussian model, I k For single-view scene images, π k For the corresponding camera pose, there are a total of n images for the first restoration.

[0052] S14. Decompose the first repaired image into blocks and filter out the target blocks that correspond to the area where the target object is located in the scene image.

[0053] In some embodiments, the first restored image is decomposed into non-overlapping square blocks, and then the block containing the target object is identified as the target block. In some embodiments, step S14 can be performed using a two-dimensional mask, which can be to decompose the first restored image and its corresponding two-dimensional mask into non-overlapping square blocks. The blocks obtained from the decomposition of the first restored image and its corresponding two-dimensional mask correspond one-to-one, and the two corresponding blocks have the same shape and size. Because the two-dimensional mask identifies the target object, the blocks obtained from the decomposition of the two-dimensional mask can be filtered out, and blocks whose mask ratio exceeds a threshold (e.g., 0.1). For blocks whose mask ratio exceeds the threshold (i.e., the area of ​​the mask value 1 occupies more than the threshold of the total area of ​​the block), the block with the same shape and size in the first restored image is the target block. Previously, the Gaussian elements corresponding to the target object were replaced with randomly initialized Gaussian elements. Therefore, the first restored image does not contain the target object. The region in the first restored image corresponding to the original location of the target object is generated after restoration using randomly initialized Gaussian elements. The quality of this region is usually not high and requires further restoration. Therefore, it is necessary to identify the target block in the first restored image that corresponds to the region where the target object is located in the scene image. That is, the target block is a block that contains image content generated using the restored randomly initialized Gaussian elements and whose area ratio exceeds a preset threshold.

[0054] S15. Decompose the scene image into blocks and filter out source blocks that have similar semantic features to the target block.

[0055] In some embodiments, each scene image is also decomposed into the same blocks according to the same decomposition method in step S14. Source blocks similar to the target block are selected from the blocks of each scene image. These source blocks are not the blocks containing the target object. The target block does not contain the target object. When repairing, the 3D Gaussian model typically uses the features of Gaussian elements surrounding the target object to repair the randomly initialized Gaussian elements. Therefore, source blocks similar to the target block usually include source blocks in the scene image of the region adjacent to the target object, as well as other source blocks that, although not adjacent to the target object, have similar semantics to the target block. For example, if the target block is the area in a garden scene after the statue has been removed, which is displayed as grass, then the source blocks might be blocks of the garden grass from other perspectives in the scene image, or blocks of adjacent grass that are not occluded in the same perspective. In some embodiments, a pre-trained self-distilled vision transformer model encoder is used to extract and compute the semantic features of the blocks in the scene image and the semantic features of the target block, for example, using a DINOv2 (Self-Distilled Vision Transformer v2) encoder. Extract semantic features of the target block f i For the semantic features of the i-th target block, B iLet M be the target block, d be the corresponding 2D mask, d be the feature dimension, and ⊙ be the Hadamard product. For the semantic features of blocks in a scene image, the following approach is used: Calculate its semantic features, f j v For the semantic features of blocks in a scene image, B j v Let v represent the v-th scene image and j represent the j-th block. Source blocks similar to the target block are determined based on the similarity between the semantic features of the scene image blocks and the semantic features of the target block. In some embodiments, for each target block... Source blocks are determined by cosine similarity, using, for example... Calculate the cosine similarity between the semantic features of the target block and the blocks in the scene image. If the cosine similarity is greater than the threshold, the block in the scene image is considered to be the source block of the target block. Here, sim(·,·) represents the cosine similarity function between two features, and the matching criterion is the consistency of semantic features.

[0056] S16. Obtain the semantic features of the source block, and guide the diffusion model to repair the target block based on the source block and the semantic features of the source block to generate the target block that does not contain the target object.

[0057] In some embodiments, the diffusion model is guided to repair the target block based on the source block and its semantic features to generate a target block free of the target object. This includes: sampling the projection of the source block into the latent space using the denoising network of the diffusion model to obtain a latent representation; using the latent representation and the semantic features of the source block as input to the diffusion model, and repairing the target block using the diffusion model. Specifically, the semantic features of the source block are injected into each layer of the denoising network of the diffusion model through a cross-attention mechanism. Specifically, a pre-trained VAE (Variational Autoencoder) encoder (such as the VAE of Stable Diffusion) converts the target block into a latent representation (a representation in the latent space), which serves as the initial input to the diffusion model for predicting noise. The diffusion model operates in the latent space, and its main component is the denoising network Unet. The VAE encoder compresses the target block into a low-dimensional representation, avoiding direct processing of high-resolution images. Latent representation z t i The semantic features c of the current time step T and the target block i As input to the diffusion model, the current time step is used to identify the iteration time step; for example, iterations can proceed from 1000 to 0. , For the denoising network Unet, The noise is predicted, and the process of predicting the noise involves multiple iterations. Each iteration updates the latent representation (therefore z). t i The superscript 't' indicates the time step T. The semantic features of the source block replace the text embedding of the diffusion model. These features are then injected into the denoising network layers of the diffusion model via a cross-attention mechanism, guiding the repair of the target block, ensuring semantic consistency of the generated content, and preventing inconsistencies between the final generated content and its surrounding environment. Predicted noise is generated through multiple iterations, and this noise is removed from the target block to obtain the repaired target block. The content of the repaired target block matches the source block and its surrounding area.

[0058] S17. Update the first repaired image with the repaired target block to obtain the second repaired image.

[0059] In some embodiments, after obtaining the repaired target block, the position of the unrepaired target block in the first repaired image is replaced to obtain a second repaired image. The second repaired image is an image with the target object removed, and the content displayed in the area where the target object was originally located or occluded matches the surrounding area. After obtaining the second repaired image, the 3D Gaussian model can be updated based on the second repaired image.

[0060] In some embodiments of this disclosure, a semantic-guided block matching (SBM) mechanism is employed, based on cross-view semantic feature matching such as DINOv2, to achieve semantic consistency alignment between the target block and multiple views. In these embodiments, cross-view semantic features are fully utilized for repair, resolving semantic conflicts (such as texture mismatch or structural misalignment) of the target block in different views, effectively solving the semantic conflict problem. By using multi-view observation information to complete the occluded area of ​​the target object, reasonable content of the unobserved area is generated, maintaining visual consistency and geometric rationality across views.

[0061] Figure 2 Schematic diagrams of some embodiments of this disclosure are shown, such as Figure 2 As shown from left to right, scene images of the target scene were taken from multiple perspectives, and the target objects to be removed are marked in the images. Figure 2 (The area circled by the hexagon on the left), then as follows Figure 2 As shown in the middle section, source blocks with similar semantic features are selected from scene images from multiple perspectives based on semantic features. These source blocks and their semantic features guide the repair of the region containing the target object. The target object is removed, and reasonable content with semantic consistency is added, ultimately generating a result as shown in the image. Figure 2 The second restored image shown on the right shows the target object removed, and the displayed content is reasonable and matches the surroundings.

[0062] In some embodiments of this disclosure, after updating the first repaired image with the repaired target block to obtain the second repaired image, the method further includes: converting the second repaired image into a grayscale image; dividing the grayscale image into blocks and determining the high-frequency features of each block; determining low-quality blocks from the second repaired image based on the high-frequency features; repairing the low-quality blocks using the semantic features of the source blocks and the high-frequency features of the low-quality blocks; and updating the second repaired image with the repaired low-quality blocks to obtain the third repaired image.

[0063] In some embodiments, the second repaired image is converted to grayscale image I. gray Using horizontal Sobel kernel K h and vertical Sobel core K v Perform high-pass filtering (convolution operation) ), combined with the eroded two-dimensional mask M erode (Used to eliminate noise interference at mask boundaries), calculate the high-frequency feature map F of the grayscale image. k =(I gray K h +I gray K v )⊙I⊙M erode I represents the second restored image. The value of each pixel in the high-frequency feature map represents the detail richness at that location; the larger the value, the stronger the detail, such as edges and textures. The grayscale image is divided into blocks to generate multiple non-overlapping squares, and the high-frequency features of each block are determined from the high-frequency feature map. The average of the high-frequency features of all pixels within a block can be used as the high-frequency feature of that block. If the high-frequency feature of a block is lower than the average of the high-frequency features of its surrounding blocks (in a nine-grid example, the central block has eight adjacent surrounding blocks) by a percentage (e.g., more than 15%), then the block corresponding to that block in the second restored image is considered a low-quality block that needs restoration. The low-quality block has a low high-frequency feature value, describing weaker details. Therefore, only low-quality blocks are restored, reducing computation and avoiding damage to other areas. Low-quality blocks are restored using the semantic features of the source block. This iterative optimization of low-quality regions identified by high-frequency features improves local restoration quality and global consistency while maintaining overall efficiency. After obtaining the restored low-quality blocks, their positions in the second restored image are replaced to obtain the third restored image.

[0064] In some embodiments of this disclosure, the low-quality block is repaired using the semantic features of the source block and the high-frequency features of the low-quality block, including: obtaining the high-frequency feature vector of the high-frequency features of the low-quality block; inputting the semantic features of the source block and the high-frequency feature vector into a denoising network, and performing denoising repair on the low-quality block through the diffusion model in which the denoising network is located.

[0065] In some embodiments, when repairing low-quality blocks, instead of a global redraw, details are precisely injected, and only the blurred parts are finely processed, rather than redrawing the entire image. If UNet is used directly for repair, it will be over-smoothed (e.g., restoring leaves to a uniform green, losing texture). Therefore, this embodiment uses high-frequency feature vectors (describing detail information) and semantic features (content information) to guide UNet to accurately restore details. Specifically, for each low-quality block B... k Through the lightweight convolutional encoder E freq Extracting high-frequency feature vector f k =E freq (F k ), F in the formula k =∣▽xI k ∣+∣▽yI k |, where ▽x and ▽y are the Sobel filter gradients in the horizontal and vertical directions, respectively, and I k E represents the original pixels of the low-quality block in the second restored image. freq It is only responsible for extracting details, not for drawing diagrams. It combines the semantic features of the source block. k and the currently extracted high-frequency feature vector f k Input the UNet denoising network and predict the noise. , where f k The intermediate layers of UNet can be injected through cross-attention mechanisms or multilayer perceptron fusion blocks, enabling the network to recover clearer textures under the guidance of global semantics and local detail priors. To improve the performance of UNet, it can be pre-trained for noise prediction to adapt it to the object removal task. In this formula, θ′ represents the UNet parameters fine-tuned after pre-training. Of course, this pre-training step can also be omitted, and UNet can be used directly. Pre-training can improve the accuracy of noise prediction. k Given the current noise of the diffusion model and T as the time step, noise is predicted through a denoising network. The predicted noise is then removed from the low-quality block to obtain the repaired low-quality block. By inputting the semantic features of the source block and the high-frequency feature vector into the denoising network, the repair ensures both semantic consistency and high-frequency details. In some embodiments of this disclosure, generating the third repaired image further includes updating the three-dimensional Gaussian model of the target scene based on the third repaired image.

[0066] In some embodiments of this disclosure, updating the three-dimensional Gaussian model of the target scene based on the third restored image includes: calculating a loss function and adjusting the parameters of the Gaussian elements to reduce the loss function; the loss function includes: Gaussian reconstruction loss and fractional distillation sampling loss.

[0067] In some embodiments, after obtaining the third restored image, the loss function L is calculated. total Specifically, Gaussian reconstruction loss L GS =(1-λ)L1+λL D-SSIM ,in, , This is the third restored image, I is the scene image, L1 represents the pixel-level difference between the third restored image and the scene image, and L... D-SSIM For differentiable SSIM (structural similarity index), fractional distillation sampling loss L SDS , ,in, This represents all possible noise ( The average of the time steps (t) of the diffusion process and the time step (t) is taken, where w(t) is the time step weight. It is noise predicted by the diffusion model. ψ is the version of the noisy image at time step t, y is the conditional input (such as semantic features of the source block), ψ is the parameters of the diffusion model, and t is the current time step. This is the noise actually added during the diffusion process. It is an image rendered by a 3D Gaussian model, where θ is the parameter of the Gaussian unit (including position x). i Rotation r i Opacity α i and color c i L total =L GS +λ SDS ·L SDS , λ SDS The weights are used to control the semantic guidance. After calculating the loss function, the parameters of all Gaussian elements are updated by backpropagating the gradient of the total loss.

[0068] In some embodiments of this disclosure, a semantically guided block matching mechanism is employed, based on DINOv2 cross-view semantic feature matching, to achieve consistent alignment between the repaired region and the semantics of multiple views. A high-frequency feature-guided diffusion model optimization is used, extracting high-frequency details through the Sobel operator to guide the diffusion model in generating sharpened textures, thus addressing the over-smoothing problem. A region-based progressive optimization strategy is employed, selectively iteratively repairing low-quality regions to balance repair quality and computational efficiency. The integration of 3D Gaussian sputtering and composite loss is used, fusing reconstruction loss and SDS loss to ensure 3D structural consistency and visual realism.

[0069] In some embodiments of this disclosure, multi-view functionality is improved. Figure 1Consistency is achieved through cross-view semantic alignment, effectively resolving semantic conflicts. Detail fidelity is enhanced; guided by high-frequency features, the edge and texture clarity of the repaired results are significantly improved. Geometric rationality and visual coherence are maintained even in the removal of large objects and 360° borderless scenes, broadening its applicability. In some embodiments of this disclosure, semantic guidance and progressive optimization outperform related technologies in terms of quality, consistency, and efficiency of 3D target removal, providing a more reliable solution for applications such as AR and VR scene editing.

[0070] Some embodiments of this disclosure also propose an image processing apparatus, comprising:

[0071] A generation unit is used to generate a three-dimensional Gaussian model of a target scene based on multiple scene images of the target scene, wherein the three-dimensional Gaussian model includes multiple Gaussian elements;

[0072] The processing unit is configured to determine the mask region in the three-dimensional Gaussian model based on the two-dimensional mask of the scene image and replace the Gaussian elements of the mask region with randomly initialized Gaussian elements, wherein the two-dimensional mask is used to identify the target object;

[0073] The processing unit is also used to repair the three-dimensional Gaussian model and generate a first repaired image from the same viewpoint as the scene image;

[0074] The processing unit is further configured to decompose the first repaired image into blocks and filter out the target blocks corresponding to the area where the target object is located in the scene image;

[0075] The processing unit is further configured to decompose the scene image into blocks and filter out source blocks that have similar semantic features to the target block;

[0076] The processing unit is further configured to acquire the semantic features of the source block, and guide the diffusion model to repair the target block based on the source block and the semantic features of the source block to generate the target block that does not contain the target object;

[0077] The processing unit is further configured to update the first repaired image with the repaired target block to obtain a second repaired image.

[0078] In some embodiments, the processing unit is further configured to update the three-dimensional Gaussian model based on the second repaired image.

[0079] In some embodiments, after updating the first repaired image with the repaired target block to obtain a second repaired image, the processing unit is further configured to: convert the second repaired image into a grayscale image; divide the grayscale image into blocks and determine the high-frequency features of each block; determine low-quality blocks from the second repaired image based on the high-frequency features; repair the low-quality blocks using the semantic features of the source block and the high-frequency features of the low-quality blocks; and update the second repaired image with the repaired low-quality blocks to obtain a third repaired image.

[0080] In some embodiments, the low-quality block is repaired using the semantic features of the source block and the high-frequency features of the low-quality block, including: obtaining the high-frequency feature vector of the high-frequency features of the low-quality block; inputting the semantic features of the source block and the high-frequency feature vector into a denoising network, and performing denoising repair on the low-quality block through the diffusion model in which the denoising network is located.

[0081] In some embodiments, the processing unit is further configured to: update the three-dimensional Gaussian model of the target scene based on the third repaired image.

[0082] In some embodiments, decomposing the scene image into blocks and filtering out source blocks with similar semantic features to the target block includes: decomposing the scene image into blocks; using a pre-trained self-distilling visual converter model encoder to extract and calculate the semantic features of the blocks of the scene image and the semantic features of the target block; and determining source blocks similar to the target block based on the similarity between the semantic features of the blocks of the scene image and the semantic features of the target block.

[0083] In some embodiments, decomposing the first restored image into blocks and filtering out target blocks corresponding to the area where the target object is located in the scene image includes: decomposing the first restored image and the corresponding two-dimensional mask into one-to-one corresponding blocks; determining blocks in the blocks obtained by decomposing the two-dimensional mask whose mask ratio exceeds a preset threshold; and determining the block in the first restored image corresponding to the block whose mask ratio exceeds the preset threshold as the target block.

[0084] In some embodiments, the diffusion model is guided to repair the target block based on the source block and the semantic features of the source block to generate the target block that does not contain the target object. This includes: sampling the projection of the source block in the latent space through the denoising network of the diffusion model to obtain a latent representation; using the latent representation and the semantic features of the source block as input to the diffusion model, and repairing the target block through the diffusion model, wherein the semantic features of the source block are injected into each layer of the denoising network of the diffusion model through a cross-attention mechanism.

[0085] In some embodiments, updating the 3D Gaussian model of the target scene based on the third restored image includes: calculating a loss function and adjusting the parameters of the Gaussian units to reduce the loss function; the loss function includes: Gaussian reconstruction loss and fractional distillation sampling loss.

[0086] For embodiments of the apparatus, since they basically correspond to the method embodiments, relevant details can be found in the descriptions of the method embodiments. The apparatus embodiments described above are merely illustrative, and the modules described as separate modules may or may not be separate. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0087] The methods and apparatus of this disclosure have been described above based on embodiments and application examples. Furthermore, this disclosure also provides an electronic device and a computer-readable storage medium, which are described below.

[0088] The following is for reference. Figure 3 The figure illustrates a structural schematic of an electronic device (e.g., a terminal device or server) 800 suitable for implementing embodiments of the present disclosure. The terminal device in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. The electronic device shown in the figure is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present disclosure.

[0089] Electronic device 800 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 802 or a program loaded from storage device 808 into random access memory (RAM) 803. RAM 803 also stores various programs and data required for the operation of electronic device 800. The processing device 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0090] Typically, the following devices can be connected to I / O interface 805: input devices 806 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 807 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 808 including, for example, magnetic tapes, hard disks, etc.; and communication devices 809. Communication device 809 allows electronic device 800 to communicate wirelessly or wiredly with other devices to exchange data. Although an electronic device 800 with various devices is shown in the figure, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0091] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 809, or installed from a storage device 808, or installed from a ROM 802. When the computer program is executed by a processing device 801, it performs the functions defined in the methods of embodiments of this disclosure.

[0092] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0093] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0094] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0095] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods of the present disclosure.

[0096] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0097] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0098] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0099] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0100] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0101] According to one or more embodiments of this disclosure, an image processing method is provided, comprising:

[0102] A three-dimensional Gaussian model of the target scene is generated based on multiple scene images of the target scene, wherein the three-dimensional Gaussian model includes multiple Gaussian elements;

[0103] Based on the two-dimensional mask of the scene image, the mask region in the three-dimensional Gaussian model is determined and the Gaussian elements of the mask region are replaced with randomly initialized Gaussian elements. The two-dimensional mask is used to identify the target object.

[0104] The three-dimensional Gaussian model is repaired to generate a first repaired image from the same viewpoint as the scene image;

[0105] The first repaired image is decomposed into blocks, and target blocks corresponding to the area where the target object is located in the scene image are selected;

[0106] The scene image is decomposed into blocks, and source blocks with similar semantic features to the target block are selected.

[0107] Obtain the semantic features of the source block, and guide the diffusion model to repair the target block based on the source block and the semantic features of the source block to generate the target block that does not contain the target object;

[0108] The first repaired image is updated using the repaired target block to obtain the second repaired image.

[0109] According to one or more embodiments of this disclosure, an image processing method is provided, which, after updating the first repaired image with the repaired target block to obtain a second repaired image, further includes:

[0110] Convert the second repaired image to a grayscale image;

[0111] The grayscale image is divided into blocks, and the high-frequency features of each block are determined;

[0112] Low-quality blocks are identified from the second repaired image based on the high-frequency features;

[0113] The low-quality block is repaired using the semantic features of the source block and the high-frequency features of the low-quality block;

[0114] The third repaired image is obtained by updating the second repaired image based on the repaired low-quality blocks.

[0115] According to one or more embodiments of this disclosure, an image processing method is provided to repair a low-quality block using semantic features of the source block and high-frequency features of the low-quality block, including:

[0116] Obtain the high-frequency feature vector of the low-quality block;

[0117] The semantic features of the source block and the high-frequency feature vector are input into the denoising network, and the low-quality block is denoised and repaired by the diffusion model in which the denoising network is located.

[0118] According to one or more embodiments of this disclosure, an image processing method is provided, further comprising:

[0119] The three-dimensional Gaussian model of the target scene is updated based on the third repaired image.

[0120] According to one or more embodiments of this disclosure, an image processing method is provided to decompose a scene image into blocks and filter out source blocks that have similar semantic features to the target blocks, including:

[0121] The scene image is decomposed into blocks;

[0122] A pre-trained self-distillation visual converter model encoder is used to extract and compute the semantic features of the blocks in the scene image and the semantic features of the target block;

[0123] The source block similar to the target block is determined based on the similarity between the semantic features of the blocks in the scene image and the semantic features of the target block.

[0124] According to one or more embodiments of this disclosure, an image processing method is provided, which decomposes a first restored image into blocks and filters out target blocks corresponding to the area where the target object is located in the scene image, including: decomposing the first restored image and the corresponding two-dimensional mask into one-to-one corresponding blocks; determining blocks in the blocks obtained by decomposing the two-dimensional mask where the mask ratio exceeds a preset threshold; and determining the block in the first restored image corresponding to the block where the mask ratio exceeds the preset threshold is a target block.

[0125] And / or,

[0126] The diffusion model guides the source block and its semantic features to repair the target block to generate a target block that does not contain the target object. This includes: sampling the projection of the source block in the latent space through the denoising network of the diffusion model to obtain a latent representation; using the latent representation and the semantic features of the source block as input to the diffusion model, and repairing the target block through the diffusion model, wherein the semantic features of the source block are injected into each layer of the denoising network of the diffusion model through a cross-attention mechanism.

[0127] According to one or more embodiments of this disclosure, an image processing method is provided to update a three-dimensional Gaussian model of a target scene based on the third repaired image, including:

[0128] Calculate the loss function and adjust the parameters of the Gaussian element to reduce the loss function;

[0129] The loss functions include: Gaussian reconstruction loss and fractional distillation sampling loss.

[0130] According to one or more embodiments of the present disclosure, an image processing apparatus is provided, comprising:

[0131] A generation unit is used to generate a three-dimensional Gaussian model of a target scene based on multiple scene images of the target scene, wherein the three-dimensional Gaussian model includes multiple Gaussian elements;

[0132] The processing unit is configured to determine the mask region in the three-dimensional Gaussian model based on the two-dimensional mask of the scene image and replace the Gaussian elements of the mask region with randomly initialized Gaussian elements, wherein the two-dimensional mask is used to identify the target object;

[0133] The processing unit is also used to repair the three-dimensional Gaussian model and generate a first repaired image from the same viewpoint as the scene image;

[0134] The processing unit is further configured to decompose the first repaired image into blocks and filter out the target blocks corresponding to the area where the target object is located in the scene image;

[0135] The processing unit is further configured to decompose the scene image into blocks and filter out source blocks that have similar semantic features to the target block;

[0136] The processing unit is further configured to acquire the semantic features of the source block, and guide the diffusion model to repair the target block based on the source block and the semantic features of the source block to generate the target block that does not contain the target object;

[0137] The processing unit is further configured to update the first repaired image with the repaired target block to obtain a second repaired image.

[0138] According to one or more embodiments of the present disclosure, an electronic device is provided, including: at least one memory and at least one processor;

[0139] The at least one memory is used to store program code, and the at least one processor is used to call the program code stored in the at least one memory to execute the method described in any one of the above.

[0140] According to one or more embodiments of the present disclosure, a computer-readable storage medium is provided for storing program code that, when executed by a processor, causes the processor to perform the methods described above.

[0141] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0142] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0143] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. An image processing method, wherein, include: A three-dimensional Gaussian model of the target scene is generated based on multiple scene images of the target scene, wherein the three-dimensional Gaussian model includes multiple Gaussian elements; Based on the two-dimensional mask of the scene image, the mask region in the three-dimensional Gaussian model is determined and the Gaussian elements of the mask region are replaced with randomly initialized Gaussian elements. The two-dimensional mask is used to identify the target object. The three-dimensional Gaussian model is repaired to generate a first repaired image from the same viewpoint as the scene image; The first repaired image is decomposed into blocks, and target blocks corresponding to the area where the target object is located in the scene image are selected; The scene image is decomposed into blocks, and source blocks with similar semantic features to the target block are selected. Obtain the semantic features of the source block, and guide the diffusion model to repair the target block based on the source block and the semantic features of the source block to generate the target block that does not contain the target object; The first repaired image is updated using the repaired target block to obtain the second repaired image.

2. The method according to claim 1, characterized in that, After updating the first repaired image with the repaired target block to obtain the second repaired image, the method further includes: Convert the second repaired image to a grayscale image; The grayscale image is divided into blocks, and the high-frequency features of each block are determined; Low-quality blocks are identified from the second repaired image based on the high-frequency features; The low-quality block is repaired using the semantic features of the source block and the high-frequency features of the low-quality block; The third repaired image is obtained by updating the second repaired image based on the repaired low-quality blocks.

3. The method according to claim 2, characterized in that, Repairing the low-quality block using the semantic features of the source block and the high-frequency features of the low-quality block includes: Obtain the high-frequency feature vector of the low-quality block; The semantic features of the source block and the high-frequency feature vector are input into the denoising network, and the low-quality block is denoised and repaired by the diffusion model in which the denoising network is located.

4. The method according to claim 2, characterized in that, Also includes: The three-dimensional Gaussian model of the target scene is updated based on the third repaired image.

5. The method according to claim 1, characterized in that, The scene image is decomposed into blocks, and source blocks with similar semantic features to the target block are selected, including: The scene image is decomposed into blocks; A pre-trained self-distillation visual converter model encoder is used to extract and compute the semantic features of the blocks in the scene image and the semantic features of the target block; The source block similar to the target block is determined based on the similarity between the semantic features of the blocks in the scene image and the semantic features of the target block.

6. The method according to claim 1, characterized in that, Decomposing the first restored image into blocks and filtering out target blocks corresponding to the area where the target object is located in the scene image includes: decomposing the first restored image and the corresponding two-dimensional mask into one-to-one corresponding blocks; determining blocks in the blocks obtained by decomposing the two-dimensional mask where the mask ratio exceeds a preset threshold; determining the block in the first restored image corresponding to the block where the mask ratio exceeds the preset threshold as the target block; And / or, The diffusion model guides the source block and its semantic features to repair the target block to generate a target block that does not contain the target object. This includes: sampling the projection of the source block in the latent space through the denoising network of the diffusion model to obtain a latent representation; using the latent representation and the semantic features of the source block as input to the diffusion model, and repairing the target block through the diffusion model, wherein the semantic features of the source block are injected into each layer of the denoising network of the diffusion model through a cross-attention mechanism.

7. The method according to claim 4, characterized in that, The 3D Gaussian model of the target scene is updated based on the third repaired image, including: Calculate the loss function and adjust the parameters of the Gaussian element to reduce the loss function; The loss functions include: Gaussian reconstruction loss and fractional distillation sampling loss.

8. An image processing apparatus, wherein, include: A generation unit is used to generate a three-dimensional Gaussian model of a target scene based on multiple scene images of the target scene, wherein the three-dimensional Gaussian model includes multiple Gaussian elements; The processing unit is configured to determine the mask region in the three-dimensional Gaussian model based on the two-dimensional mask of the scene image and replace the Gaussian elements of the mask region with randomly initialized Gaussian elements, wherein the two-dimensional mask is used to identify the target object; The processing unit is also used to repair the three-dimensional Gaussian model and generate a first repaired image from the same viewpoint as the scene image; The processing unit is further configured to decompose the first repaired image into blocks and filter out the target blocks corresponding to the area where the target object is located in the scene image; The processing unit is further configured to decompose the scene image into blocks and filter out source blocks that have similar semantic features to the target block; The processing unit is further configured to acquire the semantic features of the source block, and guide the diffusion model to repair the target block based on the source block and the semantic features of the source block to generate the target block that does not contain the target object; The processing unit is further configured to update the first repaired image with the repaired target block to obtain a second repaired image.

9. An electronic device, comprising: At least one memory and at least one processor; The at least one memory is used to store program code, and the at least one processor is used to call the program code stored in the at least one memory to execute the method of any one of claims 1 to 7.

10. A computer-readable storage medium for storing program code, which, when executed by a processor, causes the processor to perform the method of any one of claims 1 to 7.

Images