3D Model Editing Method, Device, Storage Medium and Program Product

Abstract

The present application discloses a three-dimensional model editing method, device, storage medium, and program product, which relate to the technical field of image processing. The method includes: completing each non-rigid global sub-editing iteration based on global sparse anchors to obtain global anchors after non-rigid editing. The global anchors after non-rigid editing reflect the target state. Then, iterative densification is performed based on the global anchors after non-rigid editing, so as to obtain target global densified Gaussian points that can describe the appearance characteristics of the target object, and a target three-dimensional model is obtained. At this time, the target three-dimensional model reflects the target state, thereby showing the dynamic changes of the target object. In addition, in the present application, the global sparse anchors can describe the geometric characteristics of the target object, while the target global densified Gaussian points can describe the appearance characteristics, thereby making the target three-dimensional model more accurate and of higher quality.

Description

Technical Field

[0001] The present application relates to the field of image processing technologies, and in particular, to a three-dimensional model editing method, device, storage medium, and program product. Background Art

[0002] Nowadays, creating 3D assets has become a major trend. 3D assets can be applied in games, movies, and virtual worlds. Based on the 3D models in the 3D assets, the morphological appearance of the 3D objects therein can be vividly reflected, thus accelerating the research on 3D models.

[0003] In related technologies, a source 3D model and a text prompt are input into a pre-trained image editing model, so that the pre-trained image editing model performs editing related to the text prompt on the image corresponding to the source 3D model to output an edited image, and then a target 3D model is generated based on the edited image. However, the objects in the 3D model can only achieve color changes through text driving, cannot achieve dynamic changes, and the quality of the generated 3D model is low. Summary of the Invention

[0004] The present application provides a three-dimensional model editing method, device, storage medium, and program product to at least solve the problem in related technologies that only color changes in the three-dimensional model can be achieved.

[0005] In a first aspect, the present application provides a three-dimensional model editing method, including:

[0006] In response to the non-rigid editing being global editing, obtaining global sparse anchor points from the global of the source 3D model; the source 3D model includes a target object; the target object is in a source state; the global sparse anchor points are anchor points describing the geometric features of the target object;

[0007] Generating a plurality of non-rigid global sub-edits according to the non-rigid editing; the editing degree of the subsequent non-rigid global sub-edit is greater than that of the previous non-rigid global sub-edit;

[0008] Iterating on each non-rigid global sub-edit in turn based on the global sparse anchor points and a preset resolution global reconstruction loss function to obtain globally anchor points after non-rigid editing; the globally anchor points after non-rigid editing are anchor points for the target object to form a target state from the source state after non-rigid editing;

[0009] Densifying the globally anchor points after non-rigid editing to obtain target global densified Gaussian points describing the appearance features of the target object; the densified 3D model generated based on the target global densified Gaussian points is the target 3D model, and the target object in the target 3D model forms a target state from the source state after performing a target action; the target 3D model is a global state change.

[0010] In a second aspect, the present application also provides an editing device, including: a memory for storing a computer program; a processor for implementing the steps of any of the above three-dimensional model editing methods when executing the computer program.

[0011] In a third aspect, the present application also provides a computer-readable storage medium storing a computer program, wherein the computer program implements the steps of any of the above three-dimensional model editing methods when executed by a processor.

[0012] In a fourth aspect, the present application also provides a computer program product including a computer program, and the computer program implements the steps of any of the above three-dimensional model editing methods when executed by a processor.

[0013] Through a three-dimensional model editing method, device, storage medium, and program product provided by the present application, first, in response to non-rigid editing being global editing, global sparse anchor points are obtained from the source three-dimensional model, then multiple non-rigid global sub-edits are generated according to the non-rigid editing, and then iterations are sequentially performed on each non-rigid global sub-edit based on the global sparse anchor points and a preset resolution global reconstruction loss function, so as to obtain the global anchor points after non-rigid editing. In the present application, the target object in the source three-dimensional model is in the source state. After iterations on each non-rigid global sub-edit, the global anchor points after non-rigid editing are actually the anchor points of the target object after non-rigid editing from the source state to the target state. Therefore, the global anchor points after non-rigid editing reflect the target state. Further, densification is performed based on the global anchor points after non-rigid editing, so as to obtain target global densified Gaussian points. In the present application, the global sparse anchor points can describe the geometric features of the target object, and the target global densified Gaussian points can describe the appearance features of the target object, thereby making the target three-dimensional model more accurate in terms of geometric features and appearance features; in addition, in the present application, the editing degree of the latter non-rigid global sub-edit is greater than that of the former non-rigid global sub-edit, so the non-rigid editing is hierarchical. In fact, the target state changes in a hierarchical and progressive manner, and then iterations are sequentially performed on each non-rigid global sub-edit, that is, the target state of the target object changes in a hierarchical and progressive manner based on multiple non-rigid global sub-edits, gradually realizing non-rigid editing until the final editing is completed on the last non-rigid global sub-edit, so that iterative editing realizes controllable non-rigid editing, and the global anchor points after non-rigid editing can be more accurate and stable; in addition, in the present application, a preset resolution global reconstruction loss function is used in the non-rigid transformation of the global sparse anchor points, and then the resolution is considered to optimize the global sparse anchor points at the preset resolution, which can reduce the optimization burden. Description of the Drawings

[0014] To more clearly illustrate the embodiments of the present application, the accompanying drawings required for the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0015] Figure 1 It is a diagram of the application scenario of a three-dimensional model editing method provided by the present application;

[0016] Figure 2 It is a schematic flowchart of a three-dimensional model editing method provided for Embodiment 1;

[0017] Figure 3 It is a schematic flowchart of a three-dimensional model editing method provided for Embodiment 3;

[0018] Figure 4 It is a schematic flowchart of a three-dimensional model editing method provided for Embodiment 4;

[0019] Figure 5 It is a schematic flowchart of a three-dimensional model editing method provided for Embodiment 6;

[0020] Figure 6 It is a schematic flowchart of a three-dimensional model editing method provided for Embodiment 8;

[0021] Figure 7 It is a schematic flowchart of a three-dimensional model editing method provided for Embodiment 10;

[0022] Figure 8 It is a schematic flowchart of a three-dimensional model editing method provided for Embodiment 11;

[0023] Figure 9 It is a schematic flowchart of a three-dimensional model editing method provided for Embodiment 12;

[0024] Figure 10 It is a schematic structural diagram of a three-dimensional model editing device provided for Embodiment 13;

[0025] Figure 11 It is a schematic structural diagram of the editing device provided for Embodiment 13. Specific Embodiments

[0026] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the protection scope of the present application.

[0027] It should be noted that in the description of this application, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. The terms "first", "second", etc. in this application are used to distinguish similar objects and not to describe a specific order or sequence.

[0028] To enable those skilled in the art of this technology to better understand the solution of this application, the following further details this application in conjunction with the accompanying drawings and specific implementation manners.

[0029] In combination with the specific application environment architecture or specific hardware architecture on which the execution of the three-dimensional model editing method depends, the specific application environment architecture or specific hardware architecture is described herein.

[0030] In the related art, the source three-dimensional model and text prompts are input into a pre-trained image editing model, so that the pre-trained image editing model performs editing related to the text prompts on the image corresponding to the source three-dimensional model to output an edited image, and then a target three-dimensional model is generated based on the edited image. However, the objects in the three-dimensional model can only achieve color changes through text driving, cannot achieve dynamic changes, and the quality of the generated three-dimensional model is low.

[0031] To solve the deficiencies of the related art, the inventors of this solution have conducted creative research and designed a new solution. This solution provides a three-dimensional model editing method to solve the problem of inability to achieve dynamic changes in the related art. In this solution, based on the global sparse anchor points and the preset resolution global reconstruction loss function, iterations are performed on the non-rigid global sub-editing in sequence to obtain the globally anchored points after non-rigid editing. Among them, the globally anchored points after non-rigid editing are the anchor points of the target object after non-rigid editing from the source state to the target state, thus reflecting the dynamic changes. Then, based on the globally anchored points after non-rigid editing, densification is performed to obtain the target globally densified Gaussian points. At this time, the densified three-dimensional model generated based on the target globally densified Gaussian points is the target three-dimensional model, and the target object in the target three-dimensional model is in the target state, reflecting the state change of the target object from the source three-dimensional model to the target three-dimensional model, that is, reflecting the dynamic changes. Therefore, this solution can achieve dynamic changes. In addition, to solve the problem of low accuracy of the target three-dimensional model in the related art, in this solution, since the target three-dimensional model is generated by the target globally densified Gaussian points, and the target globally densified Gaussian points can describe the appearance features of the target object, and the target globally densified Gaussian points are densified from the global sparse anchor points, the target three-dimensional model combines geometric features and appearance features, thus making the target three-dimensional model more accurate.

[0032] Figure 1 This is an application scenario diagram of a three-dimensional model editing method provided for this application. As Figure 1 shown, this application scenario includes an editing device 101.

[0033] Among them, the editing device 101 can be a computer or a server, and there is no limitation here.

[0034] Specifically, in response to the non-rigid editing being global editing, the editing device 101 obtains global sparse anchor points from the source three-dimensional model. Further, according to the non-rigid editing, multiple non-rigid global sub-edits are generated, and based on the global sparse anchor points and the preset resolution global reconstruction loss function, iterations are performed on each rigid global sub-edit in turn to obtain the global anchor points after non-rigid editing.

[0035] Further, based on the densification of the global anchor points after non-rigid editing, target global densified Gaussian points are obtained, and the three-dimensional model after densification generated by the target global densified Gaussian points is the target three-dimensional model.

[0036] The embodiments of this application provide a three-dimensional model editing method. In combination with the execution process of the three-dimensional model editing method, the method is described in detail.

[0037] Embodiment 1

[0038] The execution subject of Embodiment 1 to Embodiment 12 of this application is a three-dimensional model editing device (hereinafter referred to as the editing device), and this editing device is located in the editing device.

[0039] Figure 2 This is a schematic flowchart of a three-dimensional model editing method provided for Embodiment 1. As Figure 2 shown, it includes:

[0040] S201, in response to the non-rigid editing being global editing, obtain global sparse anchor points from the global of the source three-dimensional model; the source three-dimensional model includes a target object; the target object is in the source state; the global sparse anchor points are anchor points describing the geometric features of the target object.

[0041] Among them, the global sparse anchor points refer to the sparse anchor points corresponding to the source three-dimensional model globally.

[0042] It should be noted that global editing means that the target object in the source three-dimensional model is to undergo a global state change, and thus this non-rigid editing is global editing.

[0043] It should be noted that the global sparse anchor points can describe the global geometric features of the target object.

[0044] In one way, when the user inputs the target text prompt information and the source 3D model, a non-rigid editing type can be attached, where the non-rigid editing type is global editing or local editing. Then, the editing device can determine that the non-rigid editing at this time is global editing from the non-rigid editing type.

[0045] S202. Generate multiple non-rigid global sub-edits according to the non-rigid editing; the editing degree of the subsequent non-rigid global sub-edit is greater than that of the previous non-rigid global sub-edit.

[0046] In one way, the non-rigid editing can be layered to generate multiple non-rigid global sub-edits.

[0047] Among them, the non-rigid global sub-edit is a sub-edit under the non-rigid editing.

[0048] S203. Iterate on each non-rigid global sub-edit in turn based on the global sparse anchor points and the preset resolution global reconstruction loss function to obtain the global anchor points after non-rigid editing; the global anchor points after non-rigid editing are the anchor points for the target object to form the target state from the source state after non-rigid editing.

[0049] Among them, the preset resolution global reconstruction loss function is a pre-set reconstruction loss function considering the resolution. In this embodiment, the preset resolution global reconstruction loss function can consider the influence on the reconstruction loss at low resolution.

[0050] In this embodiment, iterate in turn according to the editing degree on each non-rigid global sub-edit, and then obtain the global anchor points after non-rigid editing (hereinafter referred to as global anchor points) under the last non-rigid global sub-edit. In this embodiment, the global anchor points can represent that the target object forms the target state from the source state after non-rigid editing. Therefore, the target state is represented in the global anchor points.

[0051] S204. Densify based on the global anchor points after non-rigid editing to obtain the target global densified Gaussian points describing the appearance features of the target object; the densified 3D model generated based on the target global densified Gaussian points is the target 3D model, and the target object in the target 3D model forms the target state from the source state after performing the target action; the target 3D model is a global state change.

[0052] Furthermore, densify the global anchor points to obtain target global densified Gaussian points with higher density. Specifically, initialize based on the global anchor points and random Gaussian points to obtain the initial Gaussian points, and then iteratively densify the initial Gaussian points to finally determine the target global densified Gaussian points. Among them, the random Gaussian points can be from the source 3D model or randomly generated.

[0053] In this embodiment, the target 3D model is generated based on the target global dense Gaussian points, and at this time, the target object in the target 3D model is in the target state.

[0054] In this embodiment, the target object in the target 3D model is in a global state change. For example, in the source 3D model, the target object is a cat, and the source state is with hands on the hips and sitting. After non-rigid editing, a cat with hands raised and jumping is obtained. At this time, the target object changes from the state of hands on the hips and sitting to the state of hands raised and jumping, achieving global editing and obtaining a target object with a global state change.

[0055] This embodiment provides a 3D model editing method. In this embodiment, first, in response to the non-rigid editing being global editing, global sparse anchors are obtained from the source 3D model. Then, multiple non-rigid global sub-edits are generated according to the non-rigid editing. Furthermore, based on the global sparse anchors and the preset resolution global reconstruction loss function, iterations are performed on each non-rigid global sub-edit in turn, and then the global anchors after non-rigid editing are obtained. In this application, the target object in the source 3D model is in the source state. After iterations on each non-rigid global sub-edit, the global anchors after non-rigid editing are actually the anchors for the target object to form the target state from the source state after non-rigid editing. Therefore, the global anchors after non-rigid editing reflect the target state. Further, densification is performed based on the global anchors after non-rigid editing, so that target global dense Gaussian points can be obtained. In this application, the global sparse anchors can describe the geometric features of the target object, and the target global dense Gaussian points can describe the appearance features of the target object, making the target 3D model more accurate in terms of geometric features and appearance features. In addition, in this application, the editing degree of the latter non-rigid global sub-edit is greater than that of the former non-rigid global sub-edit. Therefore, the non-rigid editing is hierarchical. In fact, the target state changes in a hierarchical and progressive manner, and then iterations are performed on each non-rigid global sub-edit in turn, that is, the target state of the target object changes in a hierarchical and progressive manner based on multiple non-rigid global sub-edits, gradually realizing non-rigid editing until the final editing is completed on the last non-rigid global sub-edit, enabling the iterative editing to achieve controllable non-rigid editing, and making the global anchors after non-rigid editing more accurate and stable. In addition, in this application, the preset resolution global reconstruction loss function is used in the non-rigid transformation of the global sparse anchors, and then the resolution is considered, and the global sparse anchors are optimized at the preset resolution, which can reduce the optimization burden.

[0056] Embodiment 2

[0057] This embodiment is a further refinement of any of the above embodiments. This embodiment is an optional way to generate multiple non-rigid global sub-edits according to non-rigid editing, and specifically includes:

[0058] Adjusting the magnitudes of the semantic difference parameter and the semantic commonality parameter generates multiple non-rigid global sub-edits for non-rigid editing in ascending order of the editing degree; the last non-rigid global sub-edit has the maximum editing degree; the semantic difference parameter is used to control the importance degree of semantic differences; the semantic commonality parameter is used to control the importance degree of semantic commonalities.

[0059] It should be noted that one non-rigid edit corresponds to one semantic difference parameter and one semantic commonality parameter. To obtain non-rigid edits of different magnitudes, in this embodiment, the semantic difference parameter and the semantic commonality parameter are adjusted to obtain the adjusted semantic difference parameter and semantic commonality parameter. Each set of adjusted semantic difference parameter and semantic commonality parameter corresponds to a non-rigid global sub-edit. When adjusting the semantic difference parameter and the semantic commonality parameter, it is ensured that the previous non-rigid global sub-edit is smaller than the subsequent non-rigid global sub-edit. It can be seen that when adjusting the semantic difference parameter and the semantic commonality parameter, non-rigid global sub-edits are hierarchically generated in ascending order of the editing degree.

[0060] It can be understood that when hierarchically dividing non-rigid edits, the last non-rigid global sub-edit has the maximum editing degree.

[0061] Exemplarily, assume that the non-rigid edit is that the hands of the target object are raised from 0° to 90°. When hierarchically dividing the non-rigid edit, the semantic difference parameter and the semantic commonality parameter are adjusted to obtain non-rigid global sub-edits which are the first non-rigid global sub-edit of 0 - 10°, the second non-rigid global sub-edit of 0 - 30°, the third non-rigid global sub-edit of 0 - 60°, and the fourth non-rigid global sub-edit of 0 - 90°.

[0062] In this embodiment, the editing degree of the subsequent non-rigid global sub-edit is greater than and includes the editing degree of the previous non-rigid global sub-edit, and the last non-rigid global sub-edit is the complete non-rigid edit. Since the editing degree of the subsequent non-rigid global sub-edit in this embodiment includes the editing degree of the previous non-rigid global sub-edit, it can be gradually hierarchical from small to large, enabling non-rigid anchors corresponding to different editing degrees to be gradually obtained on each non-rigid global sub-edit, and the subsequent non-rigid global anchor is obtained based on the previous non-rigid global anchor. Then, through gradual iteration, the subsequent iterative optimization learns the results of the previous iterative optimization, and thus the global anchor after the final non-rigid edit is more accurate.

[0063] This embodiment provides a 3D model editing method. In this embodiment, by adjusting the magnitudes of the semantic difference parameter and the semantic commonality parameter, non-rigid global sub-edits can be accurately generated in ascending order of the editing degree. In this embodiment, the last non-rigid global sub-edit is the non-rigid global sub-edit with the maximum editing degree, that is, the complete non-rigid edit.

[0064] Embodiment III

[0065] This embodiment is a further refinement of any of the above embodiments. This embodiment is an optional way to iteratively obtain the global anchor points after non-rigid editing by iterating on each non-rigid global sub-edit based on global sparse anchor points and a global reconstruction loss function with a preset resolution in sequence.

[0066] Figure 3 It is a schematic flow diagram of a 3D model editing method provided for Embodiment III. As Figure 3 shown, it includes:

[0067] S301, determine the ground truth of the sub-images of each non-rigid global sub-edit.

[0068] Among them, the ground truth of the sub-image is the ground truth image obtained by downsampling the target multi-view sub-image at a preset resolution.

[0069] S302, iteratively optimize the global sparse anchor points based on the global reconstruction loss function with a preset resolution and the ground truth of the sub-images corresponding to the first non-rigid global sub-edit to determine the first non-rigid global anchor points corresponding to the first non-rigid global sub-edit.

[0070] In this embodiment, since iterations need to be performed on multiple non-rigid global sub-edits in sequence, the iteration is first performed on the first non-rigid global sub-edit.

[0071] Among them, the first non-rigid global anchor points refer to the non-rigid global anchor points generated after iterative optimization based on the global sparse anchor points on the first non-rigid global sub-edit. The first non-rigid global anchor points complete the first non-rigid global sub-edit.

[0072] S303, iteratively optimize the previous non-rigid global anchor points based on the global reconstruction loss function with a preset resolution and the ground truth of the sub-images corresponding to the current non-rigid global sub-edit to determine the current non-rigid global anchor points corresponding to the current non-rigid global sub-edit; if the current non-rigid global sub-edit is the second non-rigid global sub-edit, the first non-rigid global anchor points are the previous non-rigid global anchor points in the second non-rigid global sub-edit.

[0073] In this step, when performing the current non-rigid global sub-edit, continue to iteratively optimize the previous non-rigid global anchor points based on the global reconstruction loss function with a preset resolution and the ground truth of the sub-images corresponding to the current non-rigid global sub-edit to determine the current non-rigid global anchor points.

[0074] Among them, the current non-rigid global anchor points are the optimal iteratively optimized anchor points corresponding to the current non-rigid global sub-edit.

[0075] This embodiment provides a 3D model editing method. In this embodiment, based on a preset resolution global reconstruction loss function and the ground truth of each sub-image, iterative optimization is performed in sequence according to the editing degree of non-rigid global sub-editing. When performing the current non-rigid global sub-editing, iterative optimization is performed based on the previous non-rigid global anchor point, so that the current non-rigid global anchor point corresponding to the current non-rigid global sub-editing can be obtained. Since iterative optimization is performed based on the previous non-rigid global anchor point, the current non-rigid global anchor point is more accurate and more in line with the editing degree of the current non-rigid global sub-editing.

[0076] Embodiment 4

[0077] This embodiment is a further refinement of any of the above embodiments, and this embodiment is an optional way to determine the ground truth of each non-rigid global sub-editing.

[0078] Figure 4 It is a schematic flow diagram of a 3D model editing method provided for Embodiment 4. As Figure 4 shown, it includes:

[0079] S401, generating target text embedding data corresponding to each non-rigid global sub-editing based on the semantic difference parameter, semantic commonality parameter, and target text prompt information corresponding to each non-rigid global sub-editing.

[0080] It should be noted that each non-rigid global sub-editing corresponds to its own semantic difference parameter , semantic commonality parameter . Based on the source 3D model, target text prompt information, and a preset text embedding fusion algorithm, the projection component and the vertical component are calculated, and then they are input into a preset text embedding optimization algorithm to obtain the target text embedding data. Among them, the preset text embedding optimization algorithm includes the semantic difference parameter and the semantic commonality parameter.

[0081] Specifically, each non-rigid global sub-editing is represented by , and the target text embedding data corresponding to each non-rigid global sub-editing is represented by .

[0082] Suppose there are three non-rigid global sub-edits, which are respectively , and the generated target text embedding data are respectively and . Among them, is the target text embedding data corresponding to the non-rigid global sub-editing with the largest editing degree, that is, the target text embedding data corresponding to the non-rigid editing.

[0083] S402, input the target text embedding data and the source images rendered from the source 3D model from different perspectives into a multi-perspective image editing model that has been trained to convergence, so as to output a set of target multi-perspective sub-images corresponding to each non-rigid global sub-editing; the set of target multi-perspective sub-images includes target multi-perspective sub-images from different perspectives.

[0084] Further, output from the multi-perspective image editing model that has been trained to convergence and the corresponding set of target multi-perspective sub-images, denoted as , where i represents the i-th perspective.

[0085] Exemplarily, respectively obtain and . Among them, is the image set corresponding to the maximum editing degree, and is actually the set of target multi-perspective images corresponding to non-rigid editing.

[0086] S403, downsample each target multi-perspective sub-image according to a preset resolution to obtain the sub-image ground truth of each non-rigid global sub-editing from different perspectives.

[0087] Among them, the sub-image ground truth in global editing is denoted as .

[0088] Among them, the preset resolution is less than the standard resolution of the source image. Exemplarily, assuming that the standard resolution corresponding to the source image itself is 1024*1024, then the preset resolution is less than the standard resolution, and can be 512*512, or 256*256, or other resolutions less than the standard resolution.

[0089] Exemplarily, for the set of target multi-perspective sub-images corresponding to the first non-rigid global sub-editing, assuming that it includes four target multi-perspective sub-images from different perspectives, then downsample the above target multi-perspective sub-images according to the preset resolution, so as to obtain the sub-image ground truth corresponding to the first non-rigid global sub-editing.

[0090] This embodiment provides a 3D model editing method. In this embodiment, based on different semantic difference parameters and semantic commonality parameters, the target text embedding data corresponding to each non-rigid global sub-editing can be output, and then input into a multi-perspective image editing model that has been trained to convergence, and then a set of target multi-perspective sub-images is output. Then, each target multi-perspective sub-image is downsampled according to a preset resolution, so as to obtain the sub-image ground truth. In this embodiment, downsampling is performed using the preset resolution, which reduces the subsequent optimization burden.

[0091] Embodiment Five

[0092] This embodiment is a further refinement of any of the above embodiments. This embodiment is an optional way to iteratively optimize the global sparse anchor points based on the preset resolution global reconstruction loss function and the sub-image ground truth corresponding to the first non-rigid global sub-edit to determine the first non-rigid global anchor points corresponding to the first non-rigid global sub-edit, specifically including:

[0093] Step 1: For the first non-rigid global sub-edit, generate an edited 3D model corresponding to the first iteration based on the global sparse anchor points under the first non-rigid global sub-edit to obtain the first edited 3D model.

[0094] Specifically, for the first non-rigid global sub-edit, first generate a corresponding edited 3D model for the global sparse anchor points on the basis of the source 3D model according to the first non-rigid global sub-edit, which is the edited 3D model for the first iteration on the first non-rigid global sub-edit, that is, the first edited 3D model.

[0095] Step 2: Iteratively optimize the first edited 3D model based on the preset resolution global reconstruction loss function to determine the first non-rigid global anchor points corresponding to the first non-rigid global sub-edit.

[0096] Specifically, perform iterative optimization on the first non-rigid global sub-edit to obtain the corresponding first non-rigid global anchor points.

[0097] This embodiment provides a 3D model editing method. In this embodiment, the first edited 3D model corresponding to the first non-rigid global sub-edit is obtained based on the global sparse anchor points. Further, iterative optimization is performed on it to determine the first non-rigid global anchor points. It can be seen that in this embodiment, for the first non-rigid global sub-edit, iterative optimization needs to be performed under the preset resolution global reconstruction loss function, thus considering the preset resolution and further reducing the optimization burden; in addition, this embodiment determines the first non-rigid global anchor points corresponding to the first non-rigid global sub-edit, thereby realizing the hierarchical determination of the non-rigid global anchor points corresponding to each non-rigid global sub-edit.

[0098] Embodiment Six

[0099] This embodiment is a further refinement of any of the above embodiments. This embodiment is an optional way to iteratively optimize the first edited 3D model based on the preset resolution global reconstruction loss function to determine the first non-rigid global anchor points corresponding to the first non-rigid global sub-edit.

[0100] Figure 5 It is a schematic flow diagram of a 3D model editing method provided for Embodiment Six. As Figure 5 shown, it includes:

[0101] S501, Render the first edited 3D model from different perspectives to obtain a first global rendering image set; the first global rendering image set includes first global rendering images from different perspectives.

[0102] Among them, the global rendering image set is obtained by rendering the edited 3D model during global editing.

[0103] S502, Input the first global rendering image set and the sub-image ground truth corresponding to the first non-rigid global sub-edit into a preset-resolution global reconstruction loss function to obtain the global editing loss value for the first iteration.

[0104] Among them, the preset-resolution global reconstruction loss function is as shown in (1):

[0105] (1)

[0106] Among them, is the target multi-view sub-image set corresponding to the th non-rigid global sub-edit The sub-image ground truth downsampled according to the preset resolution; is the global rendering image set obtained by rendering the edited 3D model from the i-th perspective; is the global editing loss value.

[0107] Specifically, the first global rendering image set and are input into (1) to obtain , which is the global editing loss value for the first iteration on the first non-rigid global sub-edit. Among them, refers to the sub-image ground truth corresponding to the first non-rigid global sub-edit.

[0108] S503, In response to the global editing loss value for the first iteration not satisfying the preset-resolution global loss condition, continue to perform iterative optimization on the first edited 3D model on the first non-rigid global sub-edit until the global editing loss value of the edited 3D model corresponding to a certain number of iterations satisfies the preset-resolution global loss condition, and determine the iterative optimization anchor point corresponding to the edited 3D model that satisfies the preset-resolution global loss condition as the first non-rigid global anchor point corresponding to the first non-rigid global sub-edit.

[0109] Among them, the preset-resolution global loss condition refers to the preset global loss condition considering the preset resolution.

[0110] Specifically, if The preset resolution global loss condition is not satisfied. For the first non-rigid global sub-edit, continue to iteratively optimize the three-dimensional model after the first edit. During the second iteration, optimize from the global sparse anchor points to obtain the anchor points optimized in the second iteration, and generate the three-dimensional model after editing corresponding to the second iteration based on the anchor points optimized in the second iteration. Calculate the global editing loss value of the second iteration based on the three-dimensional model after editing corresponding to the second iteration, the sub-image ground truth corresponding to the first non-rigid global sub-edit, and the preset resolution global reconstruction loss function. Further determine whether the global editing loss value of the second iteration satisfies the preset resolution global loss condition. If it does not satisfy, continue to iterate according to the above method until the global editing loss value of the three-dimensional model after editing corresponding to a certain iteration number satisfies the preset resolution global loss condition, and then the iteration ends. Determine the anchor points optimized in this iteration as the first non-rigid global anchor points corresponding to the first non-rigid global sub-edit. If it satisfies, determine the anchor points optimized in the second iteration as the first non-rigid global anchor points corresponding to the first non-rigid global sub-edit.

[0111] This embodiment provides a three-dimensional model editing method. In this embodiment, continuous iterative optimization is performed on the first non-rigid global sub-edit, so that the global editing loss value of the three-dimensional model after editing corresponding to a certain iteration number satisfies the preset resolution global loss condition, thereby determining the first non-rigid global anchor points. In this embodiment, due to iterative optimization and the preset resolution global loss condition, the accuracy of the first non-rigid global anchor points is improved.

[0112] Embodiment Seven

[0113] This embodiment is a further refinement of any of the above embodiments. This embodiment includes:

[0114] In response to the global editing loss value of the first iteration satisfying the preset resolution global loss condition, determine the global sparse anchor points as the first non-rigid global anchor points corresponding to the first non-rigid global sub-edit.

[0115] This embodiment provides a three-dimensional model editing method. In this embodiment, as long as the global editing loss value of the first iteration satisfies the preset resolution global loss condition, the first non-rigid global anchor points can be determined. Therefore, based on the preset resolution global loss condition, the first non-rigid global anchor points can be accurately controlled.

[0116] It should be noted that Examples 1 to 7 describe the transformation from non-rigid editing to global editing. In this application, for global editing, the global part of the source 3D model is determined as the model to be edited. When performing non-rigid editing on the global sparse anchor points, the non-rigid editing is hierarchically generated into multiple non-rigid global sub-edits. Further, the sub-image true values corresponding to each non-rigid global sub-edit are determined, and the non-rigid global anchor points corresponding to each non-rigid global sub-edit are determined in ascending order of the editing degree. Since the last non-rigid global sub-edit is the largest non-rigid global sub-edit, the non-rigid global anchor point corresponding to the last non-rigid global sub-edit is the global anchor point after non-rigid editing. The non-rigid global anchor point corresponding to the previous non-rigid global sub-edit is the base anchor point for the subsequent non-rigid global sub-edit, that is, iterative optimization is performed on the subsequent non-rigid global sub-edit based on the base anchor point.

[0117] Exemplarily, assume that the non-rigid editing includes two non-rigid global sub-edits, namely and , where is the largest non-rigid global sub-edit, that is, the complete non-rigid editing. On , first, the edited 3D model corresponding to the first iteration is generated based on the global sparse anchor points, that is, the first edited 3D model. Then, the global editing loss value of the first iteration corresponding to the first edited 3D model is calculated. In response to the global editing loss value of the first iteration not satisfying the preset global resolution loss condition, the global sparse anchor points are optimized for a specified number of time steps to obtain the anchor points optimized in the second iteration. Based on the anchor points optimized in the second iteration, the first edited 3D model is continuously iteratively optimized to obtain the edited 3D model corresponding to the second iteration, and the global editing loss value of the second iteration is calculated. If the global editing loss value of the second iteration satisfies the preset global resolution loss condition, the iteration ends, and the anchor points optimized in the second iteration are determined as the corresponding non-rigid global anchor points. Among them, the edited 3D model corresponding to the second iteration is generated at the editing degree of .

[0118] Further, continue to perform iterative optimization on , using the corresponding non-rigid global anchor points as the base anchor points, and at the editing degree of , using the corresponding non-rigid global anchor points to generate on this non-rigid global sub-edit (that is, The edited three-dimensional model corresponding to the first iteration of ( ) is used to continue calculating the global editing loss value of the first iteration at this time. If the preset global resolution loss condition is not met, the basic anchor points are continuously iteratively optimized until the iteratively optimized anchor points that meet the preset global resolution loss condition are obtained. These iteratively optimized anchor points are determined as The corresponding non-rigid global anchor points. Therefore, the non-rigid global anchor points corresponding to the previous non-rigid global sub-editing serve as the basic anchor points for the subsequent non-rigid global sub-editing.

[0119] In this application, on each non-rigid global sub-editing, the anchor points optimized in the previous iteration are optimized for a specified number of time steps to obtain the anchor points optimized in the subsequent iteration.

[0120] In this application, non-rigid editing is regarded as a progressive change. Therefore, the non-rigid global anchor points corresponding to each non-rigid global sub-editing are obtained in sequence, enabling controllable non-rigid editing of the target object.

[0121] Furthermore, in this application, iterative densification is performed based on the global anchor points after non-rigid editing. During the iterative densification process, a preset global densification reconstruction loss function, a preset global densification loss condition, and the current global densification loss value are used to determine whether the global densification loss result of the current iteration number meets the preset global densification loss condition. If it meets, the densified three-dimensional model corresponding to the current iteration number is determined as the target three-dimensional model; if it does not meet, the current densified Gaussian points are continuously iteratively densified based on the densified three-dimensional model corresponding to the current iteration number until the global densification loss value of the densified three-dimensional model corresponding to a certain iteration number meets the preset global densification loss condition. The iteration ends, and the densified three-dimensional model corresponding to the iteration number that meets the condition is determined as the target three-dimensional model, and the densified Gaussian points corresponding to the densified three-dimensional model corresponding to the iteration number are determined as the target global densification Gaussian points. Among them, the current global densification loss value is the global densification loss value of the densified three-dimensional model corresponding to the current iteration number.

[0122] In this application, the preset global densification reconstruction loss function is the reconstruction loss function at the standard resolution. The preset resolution global reconstruction loss function is the loss function used for non-rigid editing of global sparse anchors to generate non-rigidly edited global anchors. This preset resolution global reconstruction loss function is the reconstruction loss function at the preset resolution, where the preset resolution is less than the standard resolution. Therefore, in the process of generating non-rigidly edited global anchors, this application considers a smaller preset resolution to reduce the optimization burden. It can be seen that in this application, the preset global densification reconstruction loss function corresponding to the standard resolution is used to achieve densification, and the preset resolution global reconstruction loss function is used to achieve non-rigid editing of global sparse anchors. Therefore, editing of the three-dimensional model using multiple resolutions is realized, and the target three-dimensional model is obtained. In this application, global editing of the source three-dimensional model is realized, and the state of the target object in the source three-dimensional model is changed using reconstruction loss functions of multiple resolutions to obtain a target three-dimensional model with global state changes.

[0123] In this application, the generation of the target three-dimensional model is realized in the way of sparse control - dense generation. Among them, the global sparse anchors can describe the global geometric features of the target object, and the obtained target global densification Gaussian points can describe the global appearance features of the target object. Furthermore, the geometry - appearance set not only realizes non-rigid editing of the target object but also ensures the accuracy and quality of the target three-dimensional model. In this application, obtaining the target global densification Gaussian points can improve the quality of the rendered image. This application decouples the geometry and appearance, which is convenient for non-rigid editing of global sparse anchors and also convenient for iterative densification.

[0124] Embodiment VIII

[0125] This embodiment is a further refinement of any of the above embodiments. Figure 6 It is a schematic flowchart of a three-dimensional model editing method provided for Embodiment VIII. As Figure 6 shown, it includes:

[0126] S601, in response to the non-rigid editing being local editing, determine a three-dimensional editing box from the source three-dimensional model. The three-dimensional editing box includes the local model to be edited of the target object, and the local model to be edited is a part of the source three-dimensional model.

[0127] Among them, the three-dimensional editing box refers to the three-dimensional framework structure that encloses the local model to be edited when performing local editing on the source three-dimensional model.

[0128] Exemplarily, assuming that only local editing needs to be performed on the target object in the source three-dimensional model, and the local editing content is that the target object raises its right hand, then the local model to be edited is the spatial area occupied by the right hand of the target object.

[0129] S602. Determine multiple source Gaussian points as local sparse anchor points from the source Gaussian points included in the three-dimensional editing box.

[0130] It should be noted that, assuming that there are a total of 1000 source Gaussian points in the source three-dimensional model and 500 source Gaussian points are included in the three-dimensional editing box, multiple source Gaussian points are then determined from these 500 source Gaussian points as local sparse anchor points.

[0131] S603. Determine the target three-dimensional model with local state changes based on the local sparse anchor points.

[0132] In one approach, perform non-rigid editing on the local sparse anchor points to obtain non-rigidly edited local anchor points, and then perform densification based on the non-rigidly edited local anchor points to obtain the target three-dimensional model with local state editing.

[0133] It should be noted that in this embodiment, the three-dimensional editing box is determined, and only the local sparse anchor points included in the three-dimensional editing box (i.e., the sparse anchor points corresponding to part of the target object) are processed, without processing the model outside the three-dimensional editing box. Therefore, computing resources can be saved and the editing speed can be improved.

[0134] This embodiment provides a three-dimensional model editing method. In this embodiment, local sparse anchor points are obtained from the three-dimensional model editing box, and then the target three-dimensional model with local state changes is determined. In this embodiment, for local editing, only part of the target object included in the three-dimensional editing box needs to be edited to determine the target three-dimensional model, saving computing resources.

[0135] Embodiment Nine

[0136] This embodiment is a further refinement of any of the above embodiments. This embodiment is an optional way to determine the target three-dimensional model with local state changes based on local sparse anchor points, and specifically includes:

[0137] Step 1: Iteratively optimize the local sparse anchor points on each non-rigid local sub-editing based on the preset resolution local reconstruction loss function to obtain the non-rigidly edited local anchor points; the editing degree of the subsequent non-rigid local sub-editing is greater than that of the previous non-rigid local sub-editing.

[0138] Among them, the preset resolution local reconstruction loss function is shown in formula (2):

[0139] (2)

[0140] Among them, is the editing mask obtained by projecting the three-dimensional editing box onto the preset resolution, is element-wise multiplication; is the local editing loss value.

[0141] Specifically, the semantic difference parameter and the semantic commonality parameter are adjusted to generate multiple non-rigid local sub-edits in layers. The editing degree of the subsequent non-rigid local sub-edit is greater than and includes that of the previous non-rigid local sub-edit. The last non-rigid local sub-edit is the non-rigid edit at this time.

[0142] Further, in the same processing manner as for the global edit, iterative optimization is sequentially performed on each non-rigid local sub-edit based on the editing degree. Until on the last non-rigid local sub-edit, the optimal iteratively optimized anchor point is obtained, and this iteratively optimized anchor point is determined as the local anchor point after non-rigid editing. When performing iterative optimization on each non-rigid local sub-edit, when performing iterative optimization on the first non-rigid local sub-edit, the iterative optimization starts based on the local sparse anchor point.

[0143] Specifically, for the th non-rigid local sub-edit, its corresponding sub-image ground truth, the local rendering image set corresponding to the current iteration number, and the editing mask are input into the preset resolution local reconstruction loss function to output the local editing loss value corresponding to the current iteration number. If the local editing loss value at this time does not meet the preset resolution local editing loss condition, continue to optimize the iteratively optimized anchor point corresponding to the current iteration number on the th non-rigid local sub-edit, and generate the corresponding edited 3D model based on the new iteratively optimized anchor point until the local editing loss value corresponding to a certain iteration number meets the preset resolution local editing loss condition. The iteration on the th non-rigid local sub-edit ends, and continue to perform iteration on the next non-rigid local sub-edit until the optimal iteratively optimized anchor point on the last non-rigid local sub-edit is obtained, which is the local anchor point after non-rigid editing (hereinafter referred to as the local anchor point).

[0144] Step 2: Densify based on the local anchor point after non-rigid editing to determine the target 3D model of the local state change.

[0145] Further, specifically, initialize based on the local anchor point and random Gaussian points to obtain the initial Gaussian points, and then perform iterative densification on the initial Gaussian points to finally determine the target local densified Gaussian points. The densified 3D model generated based on the target local densified Gaussian points is the target 3D model of the local state change.

[0146] It should be noted that the random Gaussian points in this step are multiple source Gaussian points within the three-dimensional editing box. Exemplarily, assuming that the source three-dimensional model includes 1000 source Gaussian points and the three-dimensional editing box includes 500 source Gaussian points, then a random mechanism is used to determine multiple source Gaussian points from the 500 source Gaussian points as the random Gaussian points. Among them, the number of random Gaussian points is greater than the number of local sparse anchor points to ensure that the total number of Gaussian points is dense and improve the accuracy.

[0147] This embodiment provides a three-dimensional model editing method. In this embodiment, only the local sparse anchor points within the three-dimensional editing box are densified, thereby ensuring that the rest of the space of the source three-dimensional model is not edited, thus improving the densification speed and obtaining the target three-dimensional model with local state changes faster.

[0148] Embodiment Ten

[0149] This embodiment is a further refinement of any of the above embodiments. This embodiment is an optional way to densify the local anchor points after non-rigid editing to determine the target three-dimensional model with local state changes.

[0150] Figure 7 It is a schematic flowchart of a three-dimensional model editing method provided for Embodiment Ten. As Figure 7 shown, it includes:

[0151] S701, for the first iteration, iteratively densify the local anchor points after non-rigid editing to obtain the three-dimensional model after densification in the first iteration.

[0152] In one way, when performing the first iteration densification on the local anchor points after non-rigid editing, the method of replication and splitting can be used for iterative densification. What is realized in this step is to densify the model within the three-dimensional editing box.

[0153] S702, determine the local densification loss result of the three-dimensional model after densification in the first iteration.

[0154] S703, in response to the local densification loss result not satisfying the preset local densification loss condition, iteratively densify the local densified Gaussian points corresponding to the previous iteration number until the densified three-dimensional model at a certain iteration number satisfies the preset local densification loss condition, and determine the densified three-dimensional model that satisfies the preset local densification loss condition as the target three-dimensional model with local state changes.

[0155] S704, in response to the local densification loss result satisfying the preset local densification loss condition, determine the three-dimensional model after densification in the first iteration as the target three-dimensional model with local state changes.

[0156] This embodiment also includes:

[0157] Determine the locally densified Gaussian points corresponding to the target 3D model that meet the preset local densification loss condition as the target locally densified Gaussian points.

[0158] It should be noted that the target 3D model is generated by the target locally densified Gaussian points and the locally optimized anchor points. Among them, the locally optimized anchor points are the anchor points generated by optimizing the attributes of the locally anchor points after non-rigid editing while keeping the positions of the locally anchor points after non-rigid editing unchanged.

[0159] This embodiment is a further refinement of any of the above embodiments. In this embodiment, for the local densification loss result of the 3D model after densification in the first iteration, if it does not meet the preset local densification loss condition, iterative densification needs to be continued. Densify the locally densified Gaussian points corresponding to the previous iteration number until the preset densification loss condition is met, and the iteration ends. Then, determine the target 3D model with local state changes. Due to continuous iterative densification, the density of the 3D model after densification that meets the preset local densification loss condition is ensured, and thus the target 3D model is more accurate.

[0160] Embodiment Eleven

[0161] This embodiment is a further refinement of any of the above embodiments. This embodiment is an optional way to perform iterative densification on the locally anchor points after non-rigid editing to obtain the 3D model after densification in the first iteration.

[0162] Figure 8 It is a schematic flow diagram of a 3D model editing method provided for Embodiment Eleven. As Figure 8 shown, it includes:

[0163] S801, perform iterative densification on the locally anchor points after non-rigid editing to obtain the locally densified Gaussian points in the first iteration.

[0164] Specifically, for any locally anchor point, first determine whether the position gradient of the locally anchor point meets the first preset gradient, determine the reconstruction type of the area where the locally anchor point that meets the first preset gradient is located. If it is the under-reconstruction type, copy the locally anchor point to obtain the corresponding copied Gaussian points; if it is the transition-reconstruction type, split the locally anchor point to obtain the corresponding split Gaussian points. Thus, the first iteration densification is achieved based on the reconstruction type, and the locally densified Gaussian points in the first iteration are obtained based on each copied Gaussian point, each split Gaussian point, and the locally anchor points that do not meet the first preset gradient.

[0165] S802, generate the corresponding locally densified model based on the locally densified Gaussian points in the first iteration.

[0166] In one way, the editing device may generate a densified local model based on the local part of the target object included in the Gaussian points after the first iteration of local densification within the three-dimensional editing frame.

[0167] S803. Combine the densified local model with the model outside the three-dimensional editing frame in the source three-dimensional model to obtain the densified three-dimensional model after the first iteration.

[0168] Among them, the densified three-dimensional model is a complete three-dimensional model.

[0169] This embodiment provides a three-dimensional model editing method. In this embodiment, only the three-dimensional editing frame is processed, realizing local processing and improving the speed. In addition, by combining the densified local model after the first iteration with the model outside the three-dimensional editing frame, an accurate densified three-dimensional model can be obtained.

[0170] Embodiment Twelve

[0171] This embodiment is a further refinement of any of the above embodiments and is an optional way to determine the local densification loss result of the densified three-dimensional model after the first iteration.

[0172] Figure 9 It is a schematic flowchart of a three-dimensional model editing method provided for Embodiment Twelve. As Figure 9 shown, it includes:

[0173] S901. Render the densified three-dimensional model after the first iteration from different perspectives to obtain the first local rendering image set corresponding to the first iteration; the first local rendering image set includes the first local rendering images from different perspectives.

[0174] Among them, the local rendering image set is obtained by rendering the densified three-dimensional model during local editing.

[0175] S902. Input the first local rendering image set and the target multi-view sub-image set into a preset local densification reconstruction loss function to output the local densification loss value corresponding to the first iteration. The target multi-view sub-image set is the image set corresponding to the maximum non-rigid local sub-editing, and the maximum non-rigid local sub-editing is a complete non-rigid editing.

[0176] Among them, the preset local densification reconstruction loss function is a function that is preset to reconstruct the loss of the densified three-dimensional model.

[0177] Among them, the preset local densification reconstruction loss function is shown in formula (3):

[0178] (3)

[0179] Among them, For the target multi-view sub-image set corresponding to the th non-rigid global sub-edit, and the sub-image ground truth at the standard resolution. Among them, the th non-rigid global sub-edit is the largest non-rigid global sub-edit, that is, the complete non-rigid edit. At this time, the corresponding target multi-view sub-image set is the target multi-view image set; is the local rendering image set rendered from the densified three-dimensional model at the i-th view; is the local densification loss value; The editing mask obtained by projecting the three-dimensional editing frame onto the standard resolution, is element-wise multiplication.

[0180] It should be noted that the preset local densification reconstruction loss function in this embodiment is the reconstruction loss function corresponding to the standard resolution.

[0181] Specifically, the editing device inputs the first local rendering image set and the target multi-view sub-image set into Equation (3) to output , where is the local densification loss value corresponding to the first iteration.

[0182] S903. In response to the local densification loss value satisfying the preset local densification loss condition, determine that the local densification loss result satisfies the preset local loss condition.

[0183] S904. In response to the local densification loss value not satisfying the preset local densification loss condition, determine that the local densification loss result does not satisfy the preset local densification loss condition.

[0184] This embodiment provides a three-dimensional model editing method. By using the preset local densification reconstruction loss function and the preset local densification loss condition in this embodiment, the local densification loss result can be accurately determined.

[0185] Embodiments VIII to XII in this application describe non-rigid editing as global editing. When performing local editing in this application, a three-dimensional editing frame is determined from the source three-dimensional model. The model within the three-dimensional editing frame is the local model to be edited. Further, local sparse anchor points are determined from the local model to be edited, and the non-rigid local anchor points corresponding to the last non-rigid local sub-edit are obtained in the same processing manner as the global sparse anchor points, that is, the local anchor points after non-rigid editing, and also the optimal iteratively optimized anchor points corresponding to the last non-rigid local sub-edit.

[0186] Further, non-rigid local anchors are used to initialize random Gaussian points to obtain the initial Gaussian points corresponding to the 3D editing box. Then, the above initial Gaussian points are iteratively densified to obtain the target locally densified Gaussian points and the corresponding target 3D object with local state changes at this time. Among them, the random Gaussian points are the source Gaussian points within the 3D editing box.

[0187] In this application, only the model within the 3D editing box is processed, saving the processing of the model outside the 3D editing box, so computing resources are saved.

[0188] In this application, during local editing, the model can be edited in a multi-resolution manner.

[0189] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases, the former is a better implementation method.

[0190] Embodiment Thirteen

[0191] The following is the device embodiment of this application. Figure 10 It is a schematic structural diagram of a 3D model editing device provided for Embodiment Thirteen. The 3D model editing device 1000 includes the following modules:

[0192] An obtaining module 1001, configured to obtain global sparse anchors from the global of the source 3D model in response to the non-rigid editing being global editing; the source 3D model includes a target object; the target object is in the source state; the global sparse anchors are the anchors describing the geometric features of the target object;

[0193] A generating module 1002, configured to generate a plurality of non-rigid global sub-edits according to the non-rigid editing; the editing degree of the subsequent non-rigid global sub-edit is greater than that of the previous non-rigid global sub-edit;

[0194] An editing module 1003, configured to iteratively perform operations on each non-rigid global sub-edit based on the global sparse anchors and a preset resolution global reconstruction loss function to obtain the global anchors after non-rigid editing; the global anchors after non-rigid editing are the anchors for the target object to form the target state from the source state after non-rigid editing;

[0195] A densifying module 1004, configured to densify based on the global anchors after non-rigid editing to obtain the target globally densified Gaussian points describing the appearance features of the target object; the densified 3D model generated based on the target globally densified Gaussian points is the target 3D model, and the target object in the target 3D model forms the target state from the source state after performing the target action; the target 3D model is a global state change.

[0196] Optionally, when generating module 1002 generates multiple non-rigid global sub-edits according to non-rigid editing, it is specifically used for:

[0197] Adjust the sizes of the semantic difference parameter and the semantic commonality parameter, and generate multiple non-rigid global sub-edits in ascending order of editing degree for the non-rigid editing; the last non-rigid global sub-edit is the maximum editing degree; the semantic difference parameter is used to control the importance degree of semantic differences; the semantic commonality parameter is used to control the importance degree of semantic commonalities.

[0198] Optionally, when editing module 1003 iteratively processes each non-rigid global sub-edit based on the global sparse anchor points and the preset resolution global reconstruction loss function to obtain the global anchor points after non-rigid editing, it is specifically used for:

[0199] Determine the sub-image ground truth of each non-rigid global sub-edit; iteratively optimize the global sparse anchor points based on the preset resolution global reconstruction loss function and the sub-image ground truth corresponding to the first non-rigid global sub-edit to determine the first non-rigid global anchor point corresponding to the first non-rigid global sub-edit; iteratively optimize the previous non-rigid global anchor point based on the preset resolution global reconstruction loss function and the sub-image ground truth corresponding to the current non-rigid global sub-edit to determine the current non-rigid global anchor point corresponding to the current non-rigid global sub-edit; if the current non-rigid global sub-edit is the second non-rigid global sub-edit, then the first non-rigid global anchor point is the previous non-rigid global anchor point in the second non-rigid global sub-edit.

[0200] Optionally, when editing module 1003 determines the sub-image ground truth of each non-rigid global sub-edit, it is specifically used for:

[0201] Generate the target text embedding data corresponding to each non-rigid global sub-edit based on the semantic difference parameter, the semantic commonality parameter, and the target text prompt information corresponding to each non-rigid global sub-edit; input the target text embedding data and the source images rendered from the source 3D model from different perspectives into the multi-view image editing model that has been trained to convergence to output the target multi-view sub-image set corresponding to each non-rigid global sub-edit; the target multi-view sub-image set includes the target multi-view sub-images from different perspectives; downsample each target multi-view sub-image according to the preset resolution to obtain the sub-image ground truth of each non-rigid global sub-edit from different perspectives.

[0202] When editing module 1003 iteratively optimizes the global sparse anchor points based on the preset resolution global reconstruction loss function and the sub-image ground truth corresponding to the first non-rigid global sub-edit to determine the first non-rigid global anchor point corresponding to the first non-rigid global sub-edit, it is specifically used for:

[0203] For the first non-rigid global sub-edit, based on the global sparse anchors, generate the edited 3D model corresponding to the first iteration under the first non-rigid global sub-edit to obtain the first edited 3D model; perform iterative optimization on the first edited 3D model based on the preset resolution global reconstruction loss function to determine the first non-rigid global anchors corresponding to the first non-rigid global sub-edit.

[0204] Optionally, when the editing module 1003 performs iterative optimization on the first edited 3D model based on the preset resolution global reconstruction loss function to determine the first non-rigid global anchors corresponding to the first non-rigid global sub-edit, it is specifically used for:

[0205] Render the first edited 3D model from different perspectives to obtain the first global rendering image set; the first global rendering image set includes the first global rendering images from different perspectives; input the first global rendering image set and the sub-image ground truth corresponding to the first non-rigid global sub-edit into the preset resolution global reconstruction loss function to obtain the global editing loss value of the first iteration; in response to the global editing loss value of the first iteration not satisfying the preset resolution global loss condition, continue to perform iterative optimization on the first edited 3D model for the first non-rigid global sub-edit until the global editing loss value of the edited 3D model corresponding to a certain number of iterations satisfies the preset resolution global loss condition, and determine the anchor points of the iterative optimization corresponding to the edited 3D model that satisfies the preset resolution global loss condition as the first non-rigid global anchors corresponding to the first non-rigid global sub-edit.

[0206] A 3D model editing device provided in this embodiment further includes: a determination module;

[0207] The determination module is used to, in response to the global editing loss value of the first iteration satisfying the preset resolution global loss condition, determine the global sparse anchors as the first non-rigid global anchors corresponding to the first non-rigid global sub-edit.

[0208] In this embodiment, the determination module is further used for:

[0209] In response to the non-rigid edit being a local edit, determine a 3D editing box from the source 3D model; determine a plurality of source Gaussian points as local sparse anchors from the source Gaussian points included in the 3D editing box; determine the target 3D model with local state changes based on the local sparse anchors.

[0210] When the determination module determines the target 3D model with local state changes based on the local sparse anchors, it is specifically used for:

[0211] Iteratively optimize the local sparse anchors on each non-rigid local sub-edit based on a preset resolution local reconstruction loss function to obtain local anchors after non-rigid editing; the editing degree of the latter non-rigid local sub-edit is greater than that of the previous non-rigid local sub-edit;

[0212] Densify based on the local anchors after non-rigid editing to determine the target 3D model of the local state change.

[0213] Optionally, when determining the module densifies based on the local anchors after non-rigid editing to determine the target 3D model of the local state change, it is specifically used for:

[0214] For the first iteration, iteratively densify the local anchors after non-rigid editing to obtain the 3D model after densification of the first iteration; determine the local densification loss result of the 3D model after densification of the first iteration; in response to the local densification loss result not meeting the preset local densification loss condition, iteratively densify the local Gaussian points after densification corresponding to the previous iteration number until the 3D model after densification of a certain iteration number meets the preset local densification loss condition, and determine the 3D model after densification that meets the preset local densification loss condition as the target 3D model of the local state change; in response to the local densification loss result meeting the preset local densification loss condition, determine the 3D model after densification of the first iteration as the target 3D model of the local state change.

[0215] Optionally, when determining the module iteratively densifies the local anchors after non-rigid editing to obtain the 3D model after densification of the first iteration, it is specifically used for:

[0216] Iteratively densify the local anchors after non-rigid editing to obtain the local Gaussian points after densification of the first iteration; generate the corresponding densified local model based on the local Gaussian points after densification of the first iteration; combine the densified local model with the model within the non-3D editing frame of the source 3D model to obtain the 3D model after densification of the first iteration.

[0217] Optionally, when determining the module determines the local densification loss result of the 3D model after densification of the first iteration, it is specifically used for:

[0218] Render the densified three-dimensional model of the first iteration from different perspectives to obtain a first set of local rendered images corresponding to the first iteration; the first set of local rendered images includes first local rendered images from different perspectives; input the first set of local rendered images and the target multi-view sub-image set into a preset local densification reconstruction loss function to output a local densification loss value corresponding to the first iteration; the target multi-view sub-image set is the image set corresponding to the maximum non-rigid local sub-editing, and the maximum non-rigid local sub-editing is a complete non-rigid editing. In response to the local densification loss value satisfying the preset local densification loss condition, determine that the local densification loss result satisfies the preset local loss condition; in response to the local densification loss value not satisfying the preset local densification loss condition, determine that the local densification loss result does not satisfy the preset local densification loss condition.

[0219] For the description of the features in the corresponding embodiments of the three-dimensional model editing device, reference can be made to the relevant descriptions in the corresponding embodiments of the three-dimensional model editing method, which will not be elaborated here one by one.

[0220] An embodiment of the present application also provides an editing device. Figure 11 It is a schematic structural diagram of the editing device provided in the thirteenth embodiment. As Figure 11 shown, the editing device 1100 includes a processor 1101 and a memory 1102. Among them, the processor 1101, the memory 1102, and the communication component 1103 are connected through a bus 1104. A computer program is stored in the memory 1102, and the processor 1101 is configured to run the computer program to execute the steps in any of the above embodiments of the three-dimensional model editing method.

[0221] An embodiment of the present application also provides a computer-readable storage medium, in which a computer program is stored. Among them, the computer program is configured to execute the steps in any of the above embodiments of the three-dimensional model editing method when running.

[0222] In an exemplary embodiment, the above computer-readable storage medium may include, but is not limited to: USB flash drive, read-only memory (abbreviated as ROM), random access memory (abbreviated as RAM), mobile hard disk, magnetic disk, or optical disc and other various media that can store computer programs.

[0223] An embodiment of the present application also provides a computer program product. The above computer program product includes a computer program, and when the computer program is executed by a processor, it implements the steps in any of the above embodiments of the three-dimensional model editing method.

[0224] Embodiments of the present application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described embodiments of the three-dimensional model editing method.

[0225] Those skilled in the art can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in this document can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0226] The above has introduced in detail a three-dimensional model editing provided by this application. Specific examples are used in this document to elaborate on the principle and implementation manner of this application. The description of the above embodiments is only used to help understand the method and its core idea of this application. It should be noted that for those of ordinary skill in the art in this technical field, without departing from the principle of this application, several improvements and modifications can be made to this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A three-dimensional model editing method, characterized in that, Including: In response to non-rigid editing being global editing, obtaining global sparse anchors from the global of the source 3D model; The source 3D model includes a target object in a source state, and the global sparse anchors are anchors describing the geometric features of the target object; Generating a plurality of non-rigid global sub-edits in ascending order of the editing degree of the non-rigid editing; wherein, the editing degree of each subsequent non-rigid global sub-edit is greater than the previous one until the maximum editing degree is reached; Iteratively performing on each non-rigid global sub-edit based on the global sparse anchors and a global reconstruction loss function with a preset resolution, to obtain global anchors after non-rigid editing; the global anchors after non-rigid editing are anchors for the target object to form a target state from the source state after non-rigid editing; Densifying based on the global anchors after non-rigid editing to obtain target global densified Gaussian points describing the appearance features of the target object; The densified 3D model generated based on the target global densified Gaussian points is a target 3D model, and the target 3D model is used to characterize the global state change of the target object from the source state to the target state after performing the target action; The iteratively performing on each non-rigid global sub-edit based on the global sparse anchors and a global reconstruction loss function with a preset resolution includes: Determining the sub-image ground truth of each non-rigid global sub-edit; For the first non-rigid global sub-edit, iteratively optimizing the global sparse anchors based on the global reconstruction loss function with a preset resolution and the sub-image ground truth corresponding to this sub-edit to determine the first non-rigid global anchors; For the current non-rigid global sub-edit, iteratively optimizing the previous non-rigid global anchors based on the global reconstruction loss function with a preset resolution and the sub-image ground truth corresponding to this sub-edit to determine the current non-rigid global anchors; Repeatedly execute the steps for the current non-rigid global sub-edit until all non-rigid global sub-edits are completed, to obtain global anchors after non-rigid editing.

2. The method according to claim 1, characterized in that, The generating a plurality of non-rigid global sub-edits in ascending order of the editing degree of the non-rigid editing includes: Adjusting the magnitudes of the semantic difference parameter and the semantic commonality parameter to generate a plurality of non-rigid global sub-edits in ascending order of the editing degree of the non-rigid editing; the semantic difference parameter is used to control the importance degree of semantic differences; the semantic commonality parameter is used to control the importance degree of semantic commonalities.

3. The method according to claim 1, characterized in that Wherein, If the current non-rigid global sub-edit is the second non-rigid global sub-edit, then the first non-rigid global anchors are the previous non-rigid global anchors in the second non-rigid global sub-edit.

4. The method according to claim 3, wherein The determining the sub-image ground truth of each non-rigid global sub-edit includes: Generating target text embedding data corresponding to each non-rigid global sub-edit based on the semantic difference parameter, the semantic commonality parameter, and the target text prompt information corresponding to each non-rigid global sub-edit; Input the embedded data of each target text and the source images rendered from the source 3D model from different perspectives into a multi-view image editing model that has been trained to convergence, so as to output a set of target multi-view sub-images corresponding to each non-rigid global sub-editing; the set of target multi-view sub-images includes target multi-view sub-images from different perspectives. Downsample each of the target multi-view sub-images according to a preset resolution to obtain the sub-image ground truth of each non-rigid global sub-editing from different perspectives.

5. The method according to claim 3, wherein For the first non-rigid global sub-editing, based on a preset resolution global reconstruction loss function and the sub-image ground truth corresponding to this sub-editing, iteratively optimize the global sparse anchor points to determine the first non-rigid global anchor points, including: For the first non-rigid global sub-editing, generate an edited 3D model corresponding to the first iteration based on the global sparse anchor points under the first non-rigid global sub-editing to obtain a first edited 3D model; Iteratively optimize the first edited 3D model based on the preset resolution global reconstruction loss function to determine the first non-rigid global anchor points corresponding to the first non-rigid global sub-editing.

6. The method according to claim 5, characterized in that, The iteratively optimizing the first edited 3D model based on the preset resolution global reconstruction loss function to determine the first non-rigid global anchor points corresponding to the first non-rigid global sub-editing includes: Render the first edited 3D model from different perspectives to obtain a first set of global rendered images; the first set of global rendered images includes first global rendered images from different perspectives; Input the first set of global rendered images and the sub-image ground truth corresponding to the first non-rigid global sub-editing into the preset resolution global reconstruction loss function to obtain the global editing loss value of the first iteration; In response to the global editing loss value of the first iteration not satisfying the preset resolution global loss condition, continue to iteratively optimize the first edited 3D model on the first non-rigid global sub-editing until the global editing loss value of the edited 3D model corresponding to a certain number of iterations satisfies the preset resolution global loss condition, and determine the anchor points of the iterative optimization corresponding to the edited 3D model that satisfies the preset resolution global loss condition as the first non-rigid global anchor points corresponding to the first non-rigid global sub-editing.

7. The method according to claim 6, wherein The method further includes: In response to the global editing loss value of the first iteration satisfying the preset resolution global loss condition, determine the global sparse anchor points as the first non-rigid global anchor points corresponding to the first non-rigid global sub-editing.

8. The method according to claim 1, wherein The method further includes: In response to the non-rigid editing being a local editing, determine a 3D editing box from the source 3D model; Determine a plurality of source Gaussian points included in the 3D editing box as local sparse anchor points; Determine a target 3D model with local state changes based on the local sparse anchor points.

9. The method according to claim 8, characterized in that The determining a target 3D model with local state changes based on the local sparse anchor points includes: Iteratively optimize the local sparse anchor points on each non-rigid local sub-editing based on a preset resolution local reconstruction loss function to obtain local anchor points after non-rigid editing; the editing degree of the latter non-rigid local sub-editing is greater than that of the former non-rigid local sub-editing; Densify based on the local anchor points after non-rigid editing to determine the target three-dimensional model of the local state change.

10. The method according to claim 9, wherein The densifying based on the local anchor points after non-rigid editing to determine the target three-dimensional model of the local state change includes: For the first iteration, iteratively densify the local anchor points after non-rigid editing to obtain the three-dimensional model after densification of the first iteration; Determine the local densification loss result of the three-dimensional model after densification of the first iteration; In response to the local densification loss result not satisfying the preset local densification loss condition, iteratively densify the Gaussian points after local densification corresponding to the previous iteration number until the three-dimensional model after densification of a certain iteration number satisfies the preset local densification loss condition, and determine the three-dimensional model after densification that satisfies the preset local densification loss condition as the target three-dimensional model of the local state change; In response to the local densification loss result satisfying the preset local densification loss condition, determine the three-dimensional model after densification of the first iteration as the target three-dimensional model of the local state change.

11. The method according to claim 10, wherein The iteratively densifying the local anchor points after non-rigid editing to obtain the three-dimensional model after densification of the first iteration includes: Iteratively densify the local anchor points after non-rigid editing to obtain the Gaussian points after local densification of the first iteration; Generate the corresponding densified local model based on the Gaussian points after local densification of the first iteration; Combine the densified local model with the model within the non-three-dimensional editing frame in the source three-dimensional model to obtain the three-dimensional model after densification of the first iteration.

12. The method according to claim 10, wherein The determining the local densification loss result of the three-dimensional model after densification of the first iteration includes: Render the three-dimensional model after densification of the first iteration from different perspectives to obtain the first local rendering image set corresponding to the first iteration; the first local rendering image set includes the first local rendering images from different perspectives; Input the first local rendering image set and the target multi-view sub-image set into a preset local densification reconstruction loss function to output the local densification loss value corresponding to the first iteration; the target multi-view sub-image set is the image set corresponding to the maximum non-rigid local sub-editing, and the maximum non-rigid local sub-editing is the complete non-rigid editing; In response to the local densification loss value satisfying the preset local densification loss condition, determine that the local densification loss result satisfies the preset local loss condition; In response to the local densification loss value not satisfying the preset local densification loss condition, determine that the local densification loss result does not satisfy the preset local densification loss condition.

13. An editing device, characterized in that, Includes: A memory for storing a computer program; A processor for implementing the steps of the three-dimensional model editing method according to any one of claims 1 to 12 when executing the computer program.

14. A computer-readable storage medium, characterized in that, A computer program is stored in the computer-readable storage medium, wherein when the computer program is executed by a processor, the steps of the three-dimensional model editing method according to any one of claims 1 to 12 are implemented.

15. A computer program product comprising a computer program, characterized in that, When the computer program is executed by a processor, the steps of the three-dimensional model editing method according to any one of claims 1 to 12 are implemented.

Images