Three-dimensional model editing method and device, storage medium and program product

Abstract

The invention discloses a three-dimensional model editing method and device, a storage medium and a program product, and relates to the technical field of image processing, and the method comprises the steps: completing each non-rigid global sub-editing iteration based on a global sparse anchor point, obtaining a global anchor point after non-rigid editing, and enabling the global anchor point after non-rigid editing to reflect a target state, then iterative densification is carried out on the basis of the global anchor points after non-rigid editing, so that target global densified Gaussian points capable of describing appearance characteristics of the target object are obtained, a target three-dimensional model is obtained, and at the moment, the target three-dimensional model reflects the target state, so that the dynamic change of the target object is shown; in addition, the global sparse anchor points can describe geometric features of the target object, and the target global dense Gaussian points can describe appearance features, so that the target three-dimensional model is more accurate and higher in quality.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a three-dimensional model editing method, device, storage medium and program product. Background Art

[0002] Nowadays, creating three-dimensional assets has become a major trend. Three-dimensional assets can be used in games, movies and virtual worlds. The three-dimensional models based on three-dimensional assets can vividly reflect the shape and appearance of the 3D objects, thereby accelerating the research on three-dimensional models.

[0003] In the related art, the source 3D model and the text prompt are input into a pre-trained image editing model, so that the pre-trained image editing model performs editing of the text prompt-related description on the image corresponding to the source 3D model to output the edited image, and then generates the target 3D model based on the edited image. However, the objects in the 3D model can only achieve color changes through text drive, and cannot achieve dynamic changes, and the quality of the generated 3D model is low. Summary of the invention

[0004] The present application provides a three-dimensional model editing method, device, storage medium and program product to at least solve the problem that only color changes in the three-dimensional model can be achieved in the related art.

[0005] In a first aspect, the present application provides a three-dimensional model editing method, comprising:

[0006] In response to the non-rigid editing being a global editing, a global sparse anchor point is obtained from the global of the source three-dimensional model; the source three-dimensional model includes a target object; the target object is in a source state; the global sparse anchor point is an anchor point that describes the geometric features of the target object;

[0007] generating a plurality of non-rigid global sub-edits according to the non-rigid edit; the editing degree of a latter non-rigid global sub-edit is greater than the editing degree of a former non-rigid global sub-edit;

[0008] Based on the global sparse anchor point and the preset resolution global reconstruction loss function, it is iterated on each non-rigid global sub-edit in turn to obtain the global anchor point after non-rigid editing; the global anchor point after non-rigid editing is the anchor point of the target object from the source state to the target state after non-rigid editing;

[0009] The global anchor points are densified based on non-rigid editing to obtain target global densified Gaussian points that describe the appearance characteristics of the target object; the densified three-dimensional model generated based on the target global densified Gaussian points is the target three-dimensional model, and the target object in the target three-dimensional model forms a target state from a source state after performing a target action; the target three-dimensional model is a global state change.

[0010] In a second aspect, the present application also provides an editing device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any one of the above-mentioned three-dimensional model editing methods when executing the computer program.

[0011] In a third aspect, the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program, when executed by a processor, implements the steps of any of the above-mentioned three-dimensional model editing methods.

[0012] In a fourth aspect, the present application also provides a computer program product, including a computer program, which implements the steps of any of the above-mentioned three-dimensional model editing methods when executed by a processor.

[0013] A three-dimensional model editing method, device, storage medium and program product provided by the present application first responds to non-rigid editing as global editing, and then obtains global sparse anchor points from the source three-dimensional model, then generates multiple non-rigid global sub-edits according to the non-rigid editing, and then iterates on each non-rigid global sub-edit in turn based on the global sparse anchor points and the preset resolution global reconstruction loss function, and then obtains the global anchor point after non-rigid editing. In the present application, the target object in the source three-dimensional model is in the source state. After iterating on each non-rigid global sub-edit, the obtained global anchor point after non-rigid editing is actually the anchor point of the target object that forms the target state from the source state after non-rigid editing. Therefore, the global anchor point after non-rigid editing reflects the target state. Furthermore, based on the global anchor point after non-rigid editing, densification is performed to obtain the target global densified Gaussian point. In the present application, the global sparse anchor point can describe the geometric features of the target object, and the target global dense The Gaussian points can describe the appearance features of the target object, thereby making the target three-dimensional model more accurate in terms of geometric features and appearance features; in addition, in the present application, the editing degree of the latter non-rigid global sub-edit is greater than that of the previous non-rigid global sub-edit, so the non-rigid editing is layered, and in fact, the target state is progressively changed in a layered manner, and then iterated on each non-rigid global sub-edit in turn, that is, the target state of the target object is layered and progressively changed based on multiple non-rigid global sub-edits, and the non-rigid editing is gradually realized until the final editing is completed on the last non-rigid global sub-edit, so that the iterative editing realizes controllable non-rigid editing, and the global anchor point after non-rigid editing can be made more accurate and stable; in addition, the preset resolution global reconstruction loss function is used in the present application in the non-rigid transformation of the global sparse anchor point, and the resolution is taken into account, and the global sparse anchor point is optimized at the preset resolution, which can reduce the optimization burden. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0015] Figure 1 A three-dimensional model editing method application scenario diagram provided in this application;

[0016] Figure 2 A schematic diagram of a three-dimensional model editing method provided in Example 1;

[0017] Figure 3 A schematic diagram of a three-dimensional model editing method provided in Example 3;

[0018] Figure 4 A schematic flow chart of a three-dimensional model editing method provided in Example 4;

[0019] Figure 5 A schematic flow chart of a three-dimensional model editing method provided in Example 6;

[0020] Figure 6 A schematic flow chart of a three-dimensional model editing method provided in Example 8;

[0021] Figure 7 A schematic flow chart of a three-dimensional model editing method provided in Example 10;

[0022] Figure 8 A schematic flow chart of a three-dimensional model editing method provided in Example 11;

[0023] Fig. 9 A schematic flow chart of a three-dimensional model editing method provided in Example 12;

[0024] Fig.10 A schematic diagram of the structure of a three-dimensional model editing device provided in Example 13;

[0025] Fig.11 This is a schematic diagram of the structure of the editing device provided in Example 13. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0027] It should be noted that, in the description of this application, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. The terms "first", "second", etc. in this application are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0028] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0029] In conjunction with the specific application environment architecture or the specific hardware architecture on which the execution of the three-dimensional model editing method depends, the specific application environment architecture or the specific hardware architecture is described herein.

[0030] In the related art, the source 3D model and the text prompt are input into the pre-trained image editing model, so that the pre-trained image editing model performs the editing of the text prompt-related description on the image corresponding to the source 3D model to output the edited image, and then generates the target 3D model based on the edited image. However, the objects in the 3D model can only achieve color changes through text drive, and cannot achieve dynamic changes, and the quality of the generated 3D model is low.

[0031] In order to solve the defects of the related technology, the inventor of this solution has designed a new solution after creative research. This solution provides a three-dimensional model editing method to solve the problem that dynamic changes cannot be achieved in the related technology. In this solution, the global sparse anchor point and the preset resolution global reconstruction loss function are iterated on the non-rigid global sub-editing in sequence to obtain the global anchor point after non-rigid editing, wherein the global anchor point after non-rigid editing is the anchor point of the target object from the source state to the target state after non-rigid editing, thereby reflecting dynamic changes, so that the global anchor point after non-rigid editing is finally densified to obtain the target global densified Gaussian point. At this time, the densified three-dimensional model generated based on the target global densified Gaussian point is the target three-dimensional model. The target object in the target three-dimensional model is the target state, which reflects the state change of the target object from the source three-dimensional model to the target three-dimensional model, that is, it reflects dynamic changes, so this scheme can achieve dynamic changes; in addition, in order to solve the problem of low accuracy of the target three-dimensional model in the related technology, in this scheme, since the target three-dimensional model is generated by target global densified Gaussian points, and the target global densified Gaussian points can describe the appearance features of the target object, and the target global densified Gaussian points are densified based on global sparse anchor points, the target three-dimensional model combines geometric features and appearance features, thereby making the target three-dimensional model more accurate.

[0032] Figure 1 This is a 3D model editing method application scenario diagram provided in this application. Figure 1 As shown, the application scenario includes an editing device 101.

[0033] The editing device 101 may be a computer or a server, which is not limited here.

[0034] Specifically, in response to the non-rigid edit being a global edit, the editing device 101 obtains a global sparse anchor point from the source three-dimensional model. Further, multiple non-rigid global sub-edits are generated according to the non-rigid edit, and the global sparse anchor point and the preset resolution global reconstruction loss function are sequentially iterated on each rigid global sub-edit to obtain a global anchor point after the non-rigid edit.

[0035] Furthermore, based on the densification of the global anchor points after non-rigid editing, the target global densified Gaussian points are obtained, and the densified three-dimensional model generated by the target global densified Gaussian points is the target three-dimensional model.

[0036] The embodiment of the present application provides a three-dimensional model editing method, and the method is described in detail in conjunction with the execution flow of the three-dimensional model editing method.

[0037] Embodiment 1

[0038] The execution subject of Examples 1 to 12 of the present application is a three-dimensional model editing device (hereinafter referred to as the editing device), which is located in the editing equipment.

[0039] Figure 2 A schematic diagram of a three-dimensional model editing method provided in Example 1. Figure 2 As shown, including:

[0040] S201, in response to the non-rigid editing being global editing, obtaining global sparse anchor points from the global source three-dimensional model; the source three-dimensional model includes a target object; the target object is in a source state; the global sparse anchor points are anchor points that describe geometric features of the target object.

[0041] Among them, the global sparse anchor point refers to the sparse anchor point corresponding to the source 3D model globally.

[0042] It should be noted that global editing refers to a global state change of a target object in a source three-dimensional model, and thus the non-rigid editing is a global editing.

[0043] It should be noted that the global sparse anchor points can describe the global geometric features of the target object.

[0044] In one method, the user can enter the target text prompt information and the source three-dimensional model along with a non-rigid edit type, where the non-rigid edit type is a global edit or a local edit. The editing device can then determine from the non-rigid edit type that the non-rigid edit at this time is a global edit.

[0045] S202, generating a plurality of non-rigid global sub-edits according to the non-rigid edit; the editing degree of a subsequent non-rigid global sub-edit is greater than the editing degree of a previous non-rigid global sub-edit.

[0046] In one approach, the non-rigid edits may be layered to generate multiple non-rigid global sub-edits.

[0047] Among them, non-rigid global sub-editing is a sub-editing under non-rigid editing.

[0048] S203, based on the global sparse anchor point and the preset resolution global reconstruction loss function, iterate on each non-rigid global sub-edit in turn to obtain the global anchor point after non-rigid editing; the global anchor point after non-rigid editing is the anchor point of the target object formed from the source state after non-rigid editing.

[0049] The preset resolution global reconstruction loss function is a preset reconstruction loss function that takes resolution into consideration. In this embodiment, the preset resolution global reconstruction loss function may take into consideration the impact of low resolution on reconstruction loss.

[0050] In this embodiment, each non-rigid global sub-edit is iterated in turn according to the editing degree, and then a global anchor point after non-rigid editing (hereinafter referred to as the global anchor point) is obtained under the last non-rigid global sub-edit. In this embodiment, the global anchor point can represent that the target object is formed from the source state to the target state after non-rigid editing, so the target state is represented in the global anchor point.

[0051] S204, based on the densification of the global anchor points after non-rigid editing, to obtain the target global densified Gaussian points that describe the appearance features of the target object; the densified three-dimensional model generated based on the target global densified Gaussian points is the target three-dimensional model, and the target object in the target three-dimensional model forms a target state from the source state after performing the target action; the target three-dimensional model is a global state change.

[0052] Furthermore, the global anchor points are densified to obtain target global densified Gaussian points with higher density. Specifically, initialization is performed based on the global anchor points and random Gaussian points to obtain initial Gaussian points, and then the initial Gaussian points are iteratively densified to finally determine the target global densified Gaussian points. The random Gaussian points can come from the source 3D model or be randomly generated.

[0053] In this embodiment, the target three-dimensional model is generated based on the target global densified Gaussian points. At this time, the target object in the target three-dimensional model is in the target state.

[0054] In this embodiment, the target object in the target 3D model is in a global state change. For example, in the source 3D model, the target object is a cat, and the source state is hands on hips, sitting, and after non-rigid editing, the cat with both hands raised and jumping is obtained. At this time, the target object changes from the state of hands on hips, sitting to the state of hands raised, jumping, and global editing is implemented to obtain the target object with a global state change.

[0055] The present embodiment provides a three-dimensional model editing method. In the present embodiment, first, non-rigid editing is responded to as global editing, and then global sparse anchor points are obtained from the source three-dimensional model. Then, multiple non-rigid global sub-edits are generated according to the non-rigid editing, and then, based on the global sparse anchor points and the preset resolution global reconstruction loss function, each non-rigid global sub-edit is iterated in turn to obtain the global anchor point after non-rigid editing. In the present application, the target object in the source three-dimensional model is in the source state. After iterating on each non-rigid global sub-edit, the obtained global anchor point after non-rigid editing is actually the anchor point of the target object that forms the target state from the source state after non-rigid editing. Therefore, the global anchor point after non-rigid editing reflects the target state. Furthermore, based on the global anchor point after non-rigid editing, densification is performed to obtain the target global densified Gaussian point. In the present application, the global sparse anchor point can describe the geometric features of the target object, and the target global densified Gaussian point can be To describe the appearance features of the target object, so that the target three-dimensional model is more accurate in terms of geometric features and appearance features; in addition, in the present application, the editing degree of the latter non-rigid global sub-edit is greater than the editing degree of the previous non-rigid global sub-edit, so the non-rigid editing is layered, and in fact the target state is progressively changed in layers, and then iterated on each non-rigid global sub-edit in turn, that is, the target state of the target object is layered and progressively changed based on multiple non-rigid global sub-edits, and the non-rigid editing is gradually realized until the final editing is completed on the last non-rigid global sub-edit, so that the iterative editing realizes controllable non-rigid editing, and the global anchor point after non-rigid editing can be made more accurate and stable; in addition, the preset resolution global reconstruction loss function is used in the present application in the non-rigid transformation of the global sparse anchor point, and the resolution is taken into account, and the global sparse anchor point is optimized at the preset resolution, which can reduce the optimization burden.

[0056] Embodiment 2

[0057] This embodiment is a further refinement of any of the above embodiments. This embodiment is an optional way to generate multiple non-rigid global sub-edits based on non-rigid edits, specifically including:

[0058] Adjust the size of the semantic difference parameter and the semantic commonality parameter to generate multiple non-rigid global sub-edits in order of editing degree from small to large; the last non-rigid global sub-edit is the maximum editing degree; the semantic difference parameter is used to control the importance of the semantic difference; the semantic commonality parameter is used to control the importance of the semantic commonality.

[0059] It should be noted that a non-rigid edit corresponds to a semantic difference parameter and a semantic commonality parameter. In order to obtain edits of different sizes, the semantic difference parameter and the semantic commonality parameter are adjusted in this embodiment to obtain adjusted semantic difference parameters and semantic commonality parameters. Each set of adjusted semantic difference parameters and semantic commonality parameters corresponds to a non-rigid global sub-edit, and when adjusting the semantic difference parameters and the semantic commonality parameters, it is ensured that the previous non-rigid global sub-edit is smaller than the next non-rigid global sub-edit. It can be seen that when adjusting the semantic difference parameters and the semantic commonality parameters, non-rigid global sub-edits are generated in layers according to the principle of small to large edit degrees.

[0060] It is understood that when non-rigid edits are layered, the last non-rigid global sub-edit is the largest edit degree.

[0061] For example, assuming that the non-rigid edit is the target object raising both hands from 0° to 90°, when the non-rigid edit is layered, the semantic difference parameters and the semantic commonality parameters are adjusted to obtain the non-rigid global sub-edits, namely the first non-rigid global sub-edit 0-10°, the second non-rigid global sub-edit 0-30°, the third non-rigid global sub-edit 0-60° and the fourth non-rigid global sub-edit 0-90°.

[0062] In this embodiment, the editing degree of the next non-rigid global sub-edit is greater than and includes the editing degree of the previous non-rigid global sub-edit, and the last non-rigid global sub-edit is a complete non-rigid edit. Since the editing degree of the next non-rigid global sub-edit in this embodiment includes the editing degree of the previous non-rigid global sub-edit, it can be realized that the layers are gradually divided from small to large, so that non-rigid anchor points corresponding to different editing degrees are gradually obtained on each non-rigid global sub-edit, and the next non-rigid global anchor point is obtained on the basis of the previous non-rigid global anchor point, and then iterates step by step, and the next iterative optimization learns the previous iterative optimization result, so that the final global anchor point after non-rigid editing is more accurate.

[0063] This embodiment provides a three-dimensional model editing method. In this embodiment, by adjusting the size of the semantic difference parameters and the semantic commonality parameters, a non-rigid global sub-edit can be accurately generated according to the principle of editing degree from small to large. In this embodiment, the last non-rigid global sub-edit is the non-rigid global sub-edit with the maximum editing degree, that is, a complete non-rigid edit.

[0064] Embodiment 3

[0065] The present embodiment is a further refinement of any of the above embodiments. The present embodiment is an optional method of iterating on each non-rigid global sub-edit in sequence based on a global sparse anchor point and a preset resolution global reconstruction loss function to obtain a global anchor point after non-rigid editing.

[0066] Figure 3 A schematic diagram of a three-dimensional model editing method provided in Example 3. Figure 3 As shown, including:

[0067] S301, determining the true value of each non-rigid global sub-edited sub-image.

[0068] The sub-image true value is the image true value obtained by sampling the target multi-view sub-image at a preset resolution.

[0069] S302, iteratively optimize the global sparse anchor point based on a preset resolution global reconstruction loss function and the sub-image true value corresponding to the first non-rigid global sub-edit to determine a first non-rigid global anchor point corresponding to the first non-rigid global sub-edit.

[0070] In this embodiment, iterations are sequentially performed on multiple non-rigid global sub-edits, so iterations are first performed on the first non-rigid global sub-edit.

[0071] The first non-rigid global anchor point refers to a non-rigid global anchor point generated by iteratively optimizing the first non-rigid global sub-edit based on the global sparse anchor point. The first non-rigid global anchor point completes the first non-rigid global sub-edit.

[0072] S303, iteratively optimize the previous non-rigid global anchor point based on the preset resolution global reconstruction loss function and the sub-image true value corresponding to the current non-rigid global sub-edit to determine the current non-rigid global anchor point corresponding to the current non-rigid global sub-edit; if the current non-rigid global sub-edit is the second non-rigid global sub-edit, the first non-rigid global anchor point is the previous non-rigid global anchor point in the second non-rigid global sub-edit.

[0073] In this step, during the current non-rigid global sub-editing, the previous non-rigid global anchor point is iteratively optimized based on the preset resolution global reconstruction loss function and the sub-image true value corresponding to the current non-rigid global sub-editing to determine the current non-rigid global anchor point.

[0074] Among them, the current non-rigid global anchor point is the optimal iteratively optimized anchor point corresponding to the current non-rigid global sub-editing.

[0075] The present embodiment provides a three-dimensional model editing method. In the present embodiment, iterative optimization is performed in sequence according to the editing degree of the non-rigid global sub-editing based on a preset resolution global reconstruction loss function and the true value of each sub-image. When the current non-rigid global sub-editing is performed, iterative optimization is performed based on the previous non-rigid global anchor point, so that the current non-rigid global anchor point corresponding to the current non-rigid global sub-edit can be obtained. Since iterative optimization is based on the previous non-rigid global anchor point, the current non-rigid global anchor point is more accurate and more in line with the editing degree of the current non-rigid global sub-edit.

[0076] Embodiment 4

[0077] This embodiment is a further refinement of any of the above embodiments. This embodiment is an optional way to determine the true value of each non-rigid global sub-edited sub-image.

[0078] Figure 4 A schematic diagram of a three-dimensional model editing method provided in Example 4. Figure 4 As shown, including:

[0079] S401, generating target text embedding data corresponding to each non-rigid global sub-edit based on the semantic difference parameter, semantic commonality parameter and target text prompt information corresponding to each non-rigid global sub-edit.

[0080] It should be noted that each non-rigid global sub-edit corresponds to its own semantic difference parameter , semantic commonality parameters , based on the source 3D model, the target text prompt information and the preset text embedding fusion algorithm, the projection component and the vertical component are calculated, and then input into the preset text embedding optimization algorithm to obtain the target text embedding data. Among them, the preset text embedding optimization algorithm includes semantic difference parameters and semantic commonality parameters.

[0081] Specifically, each non-rigid global sub-editing Indicates that the target text embedding data corresponding to each non-rigid global sub-edit is express.

[0082] Assume there are three non-rigid global sub-edits, namely , the generated target text embedding data are as well as ,in, It is the target text embedding data corresponding to the non-rigid global sub-edit with the largest editing degree, that is, it is the target text embedding data corresponding to the non-rigid edit.

[0083] S402, input each target text embedding data and the source images rendered by the source three-dimensional model at different perspectives into a multi-perspective image editing model that has been trained to convergence, so as to output a target multi-perspective sub-image set corresponding to each non-rigid global sub-edit; the target multi-perspective sub-image set includes target multi-perspective sub-images at different perspectives.

[0084] Furthermore, the output from the multi-view image editing model that has been trained to convergence is as well as The corresponding target multi-view sub-image set is expressed as , where i represents the i-th perspective.

[0085] For example, we get as well as .in, is the image set corresponding to the maximum editing degree, which is actually the target multi-view image set corresponding to non-rigid editing.

[0086] S403, down-sampling each target multi-view sub-image according to a preset resolution to obtain the true value of each non-rigid global sub-edited sub-image at different viewpoints.

[0087] Among them, the true value of the globally edited sub-image is expressed as .

[0088] The preset resolution is smaller than the standard resolution of the source image. For example, assuming that the standard resolution corresponding to the source image itself is 1024*1024, the preset resolution is smaller than the standard resolution, and can be 512*512, or 256*256, or other resolutions smaller than the standard resolution.

[0089] Exemplarily, for the target multi-view sub-image set corresponding to the first non-rigid global sub-edit , assuming that it includes four target multi-view sub-images under different viewing angles, the above target multi-view sub-images are downsampled according to a preset resolution to obtain the sub-image true value corresponding to the first non-rigid global sub-edit.

[0090] The present embodiment provides a three-dimensional model editing method. In the present embodiment, based on different semantic difference parameters and semantic commonality parameters, the target text embedding data corresponding to each non-rigid global sub-edit can be output, and then input into a multi-view image editing model that has been trained to convergence, and then each target multi-view sub-image set is output, and then each target multi-view sub-image is downsampled according to a preset resolution to obtain the sub-image true value. In the present embodiment, the preset resolution is used for downsampling, which reduces the burden of subsequent optimization.

[0091] Embodiment 5

[0092] This embodiment is a further refinement of any of the above embodiments. This embodiment is an optional method of iteratively optimizing the global sparse anchor point based on a preset resolution global reconstruction loss function and the sub-image true value corresponding to the first non-rigid global sub-edit to determine the first non-rigid global anchor point corresponding to the first non-rigid global sub-edit, specifically including:

[0093] Step 1: For the first non-rigid global sub-edit, generate an edited 3D model corresponding to the first iteration under the first non-rigid global sub-edit based on the global sparse anchor points to obtain the first edited 3D model.

[0094] Specifically, on the first non-rigid global sub-edit, the corresponding edited three-dimensional model is first generated for the global sparse anchor points on the basis of the source three-dimensional model according to the first non-rigid global sub-edit. This is the edited three-dimensional model of the first iteration on the first non-rigid global sub-edit, that is, the first edited three-dimensional model.

[0095] Step 2: Iteratively optimize the first edited 3D model based on a preset resolution global reconstruction loss function to determine the first non-rigid global anchor point corresponding to the first non-rigid global sub-edit.

[0096] Specifically, an iterative optimization is performed on the first non-rigid global sub-edit to obtain the corresponding first non-rigid global anchor point.

[0097] The present embodiment provides a three-dimensional model editing method. In the present embodiment, a first edited three-dimensional model corresponding to the first non-rigid global sub-edit is obtained based on the global sparse anchor point. Further, it is iteratively optimized to determine the first non-rigid global anchor point. It can be seen that in the present embodiment, the first non-rigid global sub-edit needs to be iteratively optimized under the preset resolution global reconstruction loss function, thereby taking the preset resolution into account and reducing the optimization burden; in addition, the present embodiment determines the first non-rigid global anchor point corresponding to the first non-rigid global sub-edit, thereby realizing the hierarchical determination of the non-rigid global anchor points corresponding to each non-rigid global sub-edit.

[0098] Embodiment 6

[0099] This embodiment is a further refinement of any of the above embodiments. This embodiment is an optional method of iteratively optimizing the first edited three-dimensional model based on a preset resolution global reconstruction loss function to determine the first non-rigid global anchor point corresponding to the first non-rigid global sub-edit.

[0100] Figure 5 A schematic diagram of a three-dimensional model editing method provided in Example 6. Figure 5 As shown, including:

[0101] S501, rendering a first edited three-dimensional model at different viewing angles to obtain a first global rendering image set; the first global rendering image set includes first global rendering images at different viewing angles.

[0102] The global rendering image set is obtained by rendering the edited three-dimensional model during global editing.

[0103] S502, inputting the first global rendering image set and the true value of the sub-image corresponding to the first non-rigid global sub-editing into a preset resolution global reconstruction loss function to obtain a global editing loss value of the first iteration.

[0104] Among them, the preset resolution global reconstruction loss function is shown as (1):

[0105] (1)

[0106] in, For the The target multi-view sub-image set corresponding to the non-rigid global sub-edit The true value of the sub-image after being sampled according to the preset resolution; is a set of global rendering images obtained by rendering the edited 3D model at the i-th viewing angle; Edit the loss value globally.

[0107] Specifically, the first global rendering image set and Input into (1), we get , which is the global editing loss value of the first iteration on the first non-rigid global sub-edit. Among them, refers to the ground-truth sub-image corresponding to the first non-rigid global sub-edit.

[0108] S503, in response to the global editing loss value of the first iteration not satisfying the preset resolution global loss condition, continuing to iteratively optimize the first edited three-dimensional model on the first non-rigid global sub-edit until the global editing loss value of the edited three-dimensional model corresponding to a certain number of iterations satisfies the preset resolution global loss condition, and determining the iteratively optimized anchor point corresponding to the edited three-dimensional model that satisfies the preset resolution global loss condition as the first non-rigid global anchor point corresponding to the first non-rigid global sub-edit.

[0109] The preset resolution global loss condition refers to a pre-set global loss condition corresponding to the preset resolution.

[0110] Specifically, if If the preset resolution global loss condition is not met, the first edited 3D model is then iteratively optimized on the first non-rigid global sub-edit. In the second iteration, the global sparse anchor point optimization is performed to obtain the anchor point optimized for the second iteration, and the edited 3D model corresponding to the second iteration is generated based on the anchor point optimized for the second iteration. The global editing loss value of the second iteration is calculated based on the edited 3D model corresponding to the second iteration, the true value of the sub-image corresponding to the first non-rigid global sub-edit, and the preset resolution global reconstruction loss function. It is further determined whether the global editing loss value of the second iteration meets the preset resolution global loss condition. If not, continue to iterate according to the above method until the global editing loss value of the edited 3D model corresponding to a certain number of iterations meets the preset resolution global loss condition. The iteration ends, and the iteratively optimized anchor point at this time is determined as the first non-rigid global anchor point corresponding to the first non-rigid global sub-edit; if satisfied, the anchor point optimized for the second iteration is determined as the first non-rigid global anchor point corresponding to the first non-rigid global sub-edit.

[0111] The present embodiment provides a three-dimensional model editing method. In the present embodiment, iterative optimization is continuously performed on the first non-rigid global sub-editing, so that the global editing loss value of the edited three-dimensional model corresponding to a certain number of iterations satisfies a preset resolution global loss condition, thereby determining the first non-rigid global anchor point. In the present embodiment, due to iterative optimization and the preset resolution global loss condition, the accuracy of the first non-rigid global anchor point is improved.

[0112] Embodiment 7

[0113] This embodiment is a further refinement of any of the above embodiments, and includes:

[0114] In response to the global edit loss value of the first iteration satisfying a preset resolution global loss condition, the global sparse anchor point is determined as a first non-rigid global anchor point corresponding to the first non-rigid global sub-edit.

[0115] This embodiment provides a three-dimensional model editing method. In this embodiment, as long as the global editing loss value of the first iteration meets the preset resolution global loss condition, the first non-rigid global anchor point can be determined. Therefore, the first non-rigid global anchor point can be accurately controlled based on the preset resolution global loss condition.

[0116] It should be noted that embodiments 1 to 7 are descriptions of non-rigid editing as global editing. In the present application, for global editing, the global source three-dimensional model is determined as the model to be edited. When non-rigid editing is performed on the global sparse anchor point, the non-rigid editing is layered to generate multiple non-rigid global sub-edits. Further, the true value of the sub-image corresponding to each non-rigid global sub-edit is determined, and the non-rigid global anchor points corresponding to each non-rigid global sub-edit are determined in order from small to large according to the editing degree. Since the last non-rigid global sub-edit is the largest non-rigid global sub-edit, the non-rigid global anchor point corresponding to the last non-rigid global sub-edit is the global anchor point after non-rigid editing. The non-rigid global anchor point corresponding to the previous non-rigid global sub-edit is the basic anchor point for the next non-rigid global sub-edit, that is, iterative optimization is performed on the next non-rigid global sub-edit based on the basic anchor point.

[0117] Exemplarily, assume that a non-rigid edit includes two non-rigid global sub-edits, namely and ,in, is the largest non-rigid global sub-edit, i.e., a complete non-rigid edit, First, an edited 3D model corresponding to the first iteration is generated based on the global sparse anchor point, which is the first edited 3D model. Then, a global editing loss value of the first iteration corresponding to the first edited 3D model is calculated. In response to the global editing loss value of the first iteration not satisfying the preset global resolution loss condition, the global sparse anchor point is optimized for a specified time step to obtain an anchor point optimized for the second iteration. The first edited 3D model is further iteratively optimized based on the anchor point optimized for the second iteration to obtain an edited 3D model corresponding to the second iteration, and the global editing loss value of the second iteration is calculated. If the global editing loss value of the second iteration satisfies the preset global resolution loss condition, the iteration ends, and the anchor point optimized for the second iteration is determined as The corresponding non-rigid global anchor point. Among them, the edited 3D model corresponding to the second iteration is Generated based on the degree of editing.

[0118] Further, continue in Iterative optimization, using The corresponding non-rigid global anchor point is used as the basic anchor point. The editing level is adopted The corresponding non-rigid global anchor point generates this non-rigid global sub-edit (i.e. ), continue to calculate the global editing loss value of the first iteration corresponding to this time, if it does not meet the preset global resolution loss condition, continue to iteratively optimize the basic anchor point until the iteratively optimized anchor point that meets the preset global resolution loss condition is obtained, and determine the iteratively optimized anchor point as Therefore, the non-rigid global anchor point corresponding to the previous non-rigid global sub-edit is the base anchor point for the next non-rigid global sub-edit.

[0119] In the present application, on each non-rigid global sub-edit, the anchor point optimized in the previous iteration is optimized for a specified time step to obtain the anchor point optimized in the next iteration.

[0120] In the present application, non-rigid editing is considered to be divided into gradual changes, so the non-rigid global anchor points corresponding to each non-rigid global sub-edit are obtained in sequence, thereby achieving controllable non-rigid editing of the target object.

[0121] Furthermore, in the present application, iterative densification is performed based on the global anchor points after non-rigid editing. In the iterative densification process, a preset global densification reconstruction loss function, a preset global densification loss condition, and a current global densification loss value are used to determine whether the global densification loss result of the current iteration number satisfies the preset global densification loss condition. If so, the densified three-dimensional model corresponding to the current iteration number is determined to be the target three-dimensional model; if not, the current densified Gaussian point is iteratively densified based on the densified three-dimensional model corresponding to the current iteration number until the global densification loss value of the densified three-dimensional model corresponding to a certain iteration number satisfies the preset global densification loss condition. The iteration ends, and the densified three-dimensional model that satisfies the iteration number is determined as the target three-dimensional model, and the densified Gaussian point corresponding to the densified three-dimensional model corresponding to the iteration number is determined as the target global densified Gaussian point. Among them, the current global densification loss value is the global densification loss value of the densified three-dimensional model corresponding to the current iteration number.

[0122] In the present application, the preset global densification reconstruction loss function is the reconstruction loss function under the standard resolution, the preset resolution global reconstruction loss function is the loss function used for non-rigid editing of the global sparse anchor points to generate the global anchor points after non-rigid editing, and the preset resolution global reconstruction loss function is the reconstruction loss function under the preset resolution, wherein the preset resolution is smaller than the standard resolution. Therefore, in the process of generating the global anchor points after non-rigid editing, the present application considers a smaller preset resolution to reduce the optimization burden. It can be seen that the preset global densification reconstruction loss function corresponding to the standard resolution is used in the present application to realize densification, and the preset resolution global reconstruction loss function is used to realize non-rigid editing of the global sparse anchor points, so that the editing of the three-dimensional model with multiple resolutions is realized to obtain the target three-dimensional model; the present application realizes the global editing of the source three-dimensional model, and uses the reconstruction loss functions of multiple resolutions to change the state of the target object in the source three-dimensional model to obtain the target three-dimensional model with global state change.

[0123] In the present application, a sparse control-dense generation method is used to achieve the generation of the target three-dimensional model, wherein the global sparse anchor point can describe the global geometric features of the target object, and the obtained target global densified Gaussian point can describe the global appearance features of the target object, thereby combining the geometry-appearance set, while achieving non-rigid editing of the target object, it also ensures the accuracy and quality of the target three-dimensional model. In the present application, obtaining the target global densified Gaussian point can improve the quality of the rendered image. The present application decouples geometry and appearance, facilitates non-rigid editing of the global sparse anchor point, and also facilitates iterative densification.

[0124] Embodiment 8

[0125] This embodiment is a further refinement of any of the above embodiments. Figure 6 A schematic diagram of a three-dimensional model editing method provided in Example 8. Figure 6 As shown, including:

[0126] S601: In response to the non-rigid editing being a local editing, determine a three-dimensional editing frame from the source three-dimensional model. The three-dimensional editing frame includes a to-be-edited local model of the target object, which is a part of the source three-dimensional model.

[0127] The 3D editing frame refers to a 3D frame structure surrounding a local model to be edited when a source 3D model is partially edited.

[0128] Exemplarily, assuming that only local editing is required for the target object in the source 3D model, the local editing content is: the target object raises his right hand, then the local model to be edited is the space area occupied by the right hand of the target object.

[0129] S602: Determine a plurality of source Gaussian points as local sparse anchor points from the source Gaussian points included in the three-dimensional editing box.

[0130] It should be noted that, assuming that the source three-dimensional model includes a total of 1000 source Gaussian points, the three-dimensional editing box includes 500 source Gaussian points, and then multiple source Gaussian points are determined from the 500 source Gaussian points as local sparse anchor points.

[0131] S603: Determine a target three-dimensional model with a local state change based on the local sparse anchor points.

[0132] In one approach, non-rigid editing is performed on local sparse anchor points to obtain non-rigidly edited local anchor points, and then densification is performed based on the non-rigidly edited local anchor points to obtain a locally state edited target 3D model.

[0133] It should be noted that in this embodiment, a three-dimensional editing box is determined, and only the local sparse anchor points included in the three-dimensional editing box (i.e., the sparse anchor points corresponding to some target objects) are processed, and the models outside the three-dimensional editing box are not processed, so computing resources can be saved and the editing speed can be improved.

[0134] This embodiment provides a three-dimensional model editing method. In this embodiment, local sparse anchor points are obtained from a three-dimensional model editing box, and then a target three-dimensional model with a local state change is determined. In this embodiment, for local editing, only part of the target objects included in the three-dimensional editing box need to be edited to determine the target three-dimensional model, thereby saving computing resources.

[0135] Embodiment 9

[0136] This embodiment is a further refinement of any of the above embodiments. This embodiment is an optional method for determining a target three-dimensional model with a local state change based on a local sparse anchor point, and specifically includes:

[0137] Step 1: Based on the preset resolution local reconstruction loss function, the local sparse anchor points are iteratively optimized on each non-rigid local sub-edit to obtain the local anchor points after non-rigid editing; the editing degree of the latter non-rigid local sub-edit is greater than the editing degree of the former non-rigid local sub-edit.

[0138] Among them, the preset resolution local reconstruction loss function is shown in formula (2):

[0139] (2)

[0140] in, The editing mask is obtained by projecting the 3D editing box onto the preset resolution. is element-wise multiplication; is the local editing loss value.

[0141] Specifically, the sizes of the semantic difference parameters and the semantic commonality parameters are adjusted to generate multiple non-rigid local sub-edits in layers, the editing degree of the latter non-rigid local sub-edit is greater than and includes the editing degree of the former non-rigid local sub-edit, and the last non-rigid local sub-edit is the non-rigid edit at this time.

[0142] Furthermore, in the same way as the global edit, iterative optimization is performed on each non-rigid local sub-edit based on the edit degree, until the best iteratively optimized anchor point is obtained on the last non-rigid local sub-edit, and the iteratively optimized anchor point is determined as the local anchor point after the non-rigid edit. When iterative optimization is performed on each non-rigid local sub-edit, iterative optimization is started based on the local sparse anchor point when iterative optimization is performed on the first non-rigid local sub-edit.

[0143] Specifically, for The corresponding sub-image truth value, the local rendering image set corresponding to the current iteration number, and the editing mask are input into the preset resolution local reconstruction loss function, and the local editing loss value corresponding to the current iteration number is output. If the local editing loss value at this time does not meet the preset resolution local editing loss condition, continue to On each non-rigid local sub-edit, the iteratively optimized anchor point corresponding to the current iteration number is optimized, and the corresponding edited 3D model is generated based on the new iteratively optimized anchor point, until the local editing loss value corresponding to a certain iteration number meets the preset resolution local editing loss condition, and then the local editing loss value is generated on the first non-rigid local sub-edit. After the iteration on the first non-rigid local sub-edit is completed, the iteration on the next non-rigid local sub-edit is continued until the optimal iteratively optimized anchor point on the last non-rigid local sub-edit is obtained, which is the local anchor point after non-rigid editing (hereinafter referred to as the local anchor point).

[0144] Step 2: Densify the local anchor points after non-rigid editing to determine the target 3D model with local state changes.

[0145] Further, specifically, initialization is performed based on local anchor points and random Gaussian points to obtain initial Gaussian points, and then the initial Gaussian points are iteratively densified to finally determine the target local densified Gaussian points. The densified three-dimensional model generated based on the target local densified Gaussian points is the target three-dimensional model with local state changes.

[0146] It should be noted that the random Gaussian points in this step are multiple source Gaussian points in the three-dimensional editing box. For example, assuming that the source three-dimensional model includes 1000 source Gaussian points and the three-dimensional editing box includes 500 source Gaussian points, a random mechanism is used to determine multiple source Gaussian points from the 500 source Gaussian points as random Gaussian points. The number of random Gaussian points is greater than the number of local sparse anchor points to ensure that the total number of Gaussian points is dense and improve accuracy.

[0147] This embodiment provides a three-dimensional model editing method. In this embodiment, only the local sparse anchor points within the three-dimensional editing box are densified, thereby ensuring that the remaining space of the source three-dimensional model is not edited, thereby improving the densification speed and obtaining the target three-dimensional model with local state changes more quickly.

[0148] Embodiment 10

[0149] This embodiment is a further refinement of any of the above embodiments. This embodiment is an optional way to determine the target three-dimensional model with local state changes based on densification of local anchor points after non-rigid editing.

[0150] Figure 7 A schematic flow chart of a three-dimensional model editing method provided in Example 10. Figure 7 As shown, including:

[0151] S701, for the first iteration, iteratively densify the local anchor points after non-rigid editing to obtain a densified three-dimensional model of the first iteration.

[0152] In one approach, when performing the first iterative densification on the local anchor points after non-rigid editing, the iterative densification can be performed using a replication and splitting method. This step densifies the model within the three-dimensional editing box.

[0153] S702, determining the local density loss result of the three-dimensional model after densification in the first iteration.

[0154] S703, in response to the local densification loss result not satisfying the preset local densification loss condition, iteratively densify the local densified Gaussian points corresponding to the previous number of iterations until the densified three-dimensional model of a certain number of iterations satisfies the preset local densification loss condition, and the densified three-dimensional model satisfying the preset local densification loss condition is determined as the target three-dimensional model of the local state change.

[0155] S704, in response to the local densification loss result satisfying a preset local densification loss condition, determining the densified three-dimensional model of the first iteration as the target three-dimensional model of the local state change.

[0156] This embodiment also includes:

[0157] The locally densified Gaussian points corresponding to the target three-dimensional model that meet the preset local densification loss conditions are determined as target locally densified Gaussian points.

[0158] It should be noted that the target 3D model is generated by the target local densified Gaussian points and the locally optimized anchor points, wherein the locally optimized anchor points are generated by optimizing the properties of the local anchor points after non-rigid editing while keeping the positions of the local anchor points unchanged after non-rigid editing.

[0159] This embodiment is a further refinement of any of the above embodiments. In this embodiment, if the local densification loss result of the densified three-dimensional model after the first iteration does not meet the preset local densification loss condition, it is necessary to continue iterative densification, and densify the local densified Gaussian points corresponding to the previous iteration number until the preset densification loss condition is met. The iteration ends, and then the target three-dimensional model with local state changes is determined. Due to continuous iterative densification, the density of the densified three-dimensional model that meets the preset local densification loss condition is guaranteed, thereby making the target three-dimensional model more accurate.

[0160] Embodiment 11

[0161] This embodiment is a further refinement of any of the above embodiments. This embodiment is an optional way to iteratively densify the local anchor points after non-rigid editing to obtain a densified three-dimensional model after the first iteration.

[0162] Figure 8 A schematic flow chart of a three-dimensional model editing method provided in Example 11. Figure 8 As shown, including:

[0163] S801, iteratively densify the local anchor points after non-rigid editing to obtain the local densified Gaussian points of the first iteration.

[0164] Specifically, for any local anchor point, it is first determined whether the position gradient of the local anchor point satisfies a first preset gradient, and the reconstruction type of the area where the local anchor point that satisfies the first preset gradient is located is determined. If it is an under-reconstruction type, the local anchor point is copied to obtain the corresponding copied Gaussian point; if it is a transitional reconstruction type, the local anchor point is split to obtain the corresponding split Gaussian point, thereby realizing the first iterative densification based on the reconstruction type, and obtaining the first iterative local densified Gaussian points based on each copied Gaussian point, each split Gaussian point, and the local anchor point that does not satisfy the first preset gradient.

[0165] S802, generating a corresponding densified local model based on the local densified Gaussian points of the first iteration.

[0166] In one embodiment, the editing device may generate a densified local model based on the local densified Gaussian points of the first iteration on the basis of the local part of the target object included in the three-dimensional editing box.

[0167] S803, combining the densified local model with the model in the non-3D editing box of the source 3D model to obtain a densified 3D model of the first iteration.

[0168] Among them, the densified three-dimensional model is a complete three-dimensional model.

[0169] This embodiment provides a three-dimensional model editing method. In this embodiment, only the three-dimensional editing box is processed, which realizes local processing and improves the speed. In addition, the densified local model of the first iteration is combined with the model in the non-three-dimensional editing box to obtain an accurate densified three-dimensional model.

[0170] Embodiment 12

[0171] This embodiment is a further refinement of any of the above embodiments. This embodiment is an optional way to determine the local density loss result of the three-dimensional model after the first iteration of densification.

[0172] Fig. 9 A schematic flow chart of a three-dimensional model editing method provided in Example 12. Fig. 9 As shown, including:

[0173] S901, rendering the densified three-dimensional model of the first iteration at different viewing angles to obtain a first local rendering image set corresponding to the first iteration; the first local rendering image set includes first local rendering images at different viewing angles.

[0174] Among them, the local rendering image set is obtained by rendering the densified three-dimensional model during local editing.

[0175] S902, inputting the first local rendering image set and the target multi-view sub-image set into a preset local densification reconstruction loss function to output a local densification loss value corresponding to the first iteration. The target multi-view sub-image set is an image set corresponding to the maximum non-rigid local sub-editing, and the maximum non-rigid local sub-editing is a complete non-rigid editing.

[0176] The preset local densification reconstruction loss function is a pre-set function for reconstructing the densified three-dimensional model.

[0177] Among them, the preset local densification reconstruction loss function is shown in formula (3):

[0178] (3)

[0179] in, For the The target multi-view sub-image set corresponding to the non-rigid global sub-edit The true value of the sub-image at standard resolution. The non-rigid global sub-edit is the largest non-rigid global sub-edit, that is, the complete non-rigid edit. The corresponding target multi-view sub-image set is the target multi-view image set; is a set of local rendered images obtained by rendering the densified 3D model at the i-th perspective; is the local densification loss value; The edit mask obtained by projecting the 3D edit box onto the standard resolution. is element-wise multiplication.

[0180] It should be noted that the preset local densification reconstruction loss function in this embodiment is a reconstruction loss function corresponding to the standard resolution.

[0181] Specifically, the editing device inputs the first local rendering image set and the target multi-view sub-image set into equation (3) to output ,in, is the local densification loss value corresponding to the first iteration.

[0182] S903 , in response to the local densification loss value satisfying a preset local densification loss condition, determining that the local densification loss result satisfies the preset local loss condition.

[0183] S904 , in response to the local densification loss value not satisfying a preset local densification loss condition, determining that the local densification loss result does not satisfy the preset local densification loss condition.

[0184] This embodiment provides a three-dimensional model editing method. In this embodiment, a preset local densification reconstruction loss function and a preset local densification loss condition are used to accurately determine a local densification loss result.

[0185] Embodiments 8 to 12 in the present application are descriptions of non-rigid editing as global editing. In the present application, when performing local editing, a three-dimensional editing box is determined from the source three-dimensional model, and the model within the three-dimensional editing box is the local model to be edited. Further, local sparse anchor points are determined from the local model to be edited, and the non-rigid local anchor points corresponding to the last non-rigid local sub-edit are obtained in the same processing method as the global sparse anchor points, which are the local anchor points after non-rigid editing and are also the optimal iteratively optimized anchor points corresponding to the last non-rigid local sub-edit.

[0186] Furthermore, the random Gaussian points are initialized using non-rigid local anchor points to obtain the initial Gaussian points corresponding to the three-dimensional editing box, and then the initial Gaussian points are iteratively densified to obtain the target local densified Gaussian points and the target three-dimensional target with the corresponding local state change at this time. Among them, the random Gaussian points are the source Gaussian points in the three-dimensional editing box.

[0187] In the present application, only the model in the three-dimensional editing box is processed, which saves the processing of the model in the non-three-dimensional editing box, thereby saving computing resources.

[0188] In this application, during local editing, a multi-resolution approach can be used to edit the model.

[0189] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method.

[0190] Embodiment 13

[0191] The following is an embodiment of the device of the present application. Fig.10 This is a schematic diagram of the structure of a 3D model editing device provided in Example 13. The 3D model editing device 1000 includes the following modules:

[0192] The acquisition module 1001 is used for obtaining global sparse anchor points from the global of the source three-dimensional model in response to the non-rigid editing being global editing; the source three-dimensional model includes the target object; the target object is in the source state; the global sparse anchor points are anchor points that describe the geometric features of the target object;

[0193] A generating module 1002, for generating a plurality of non-rigid global sub-edits according to a non-rigid edit; the editing degree of a subsequent non-rigid global sub-edit is greater than the editing degree of a previous non-rigid global sub-edit;

[0194] The editing module 1003 is used to iterate on each non-rigid global sub-edit in turn based on the global sparse anchor point and the preset resolution global reconstruction loss function to obtain a global anchor point after non-rigid editing; the global anchor point after non-rigid editing is the anchor point of the target object formed from the source state to the target state after non-rigid editing;

[0195] The densification module 1004 is used to densify the global anchor points after non-rigid editing to obtain target global densified Gaussian points that describe the appearance characteristics of the target object; the densified three-dimensional model generated based on the target global densified Gaussian points is the target three-dimensional model, and the target object in the target three-dimensional model forms a target state from a source state after performing a target action; the target three-dimensional model is a global state change.

[0196] Optionally, when generating multiple non-rigid global sub-edits according to a non-rigid edit, the generating module 1002 is specifically configured to:

[0197] Adjust the size of the semantic difference parameter and the semantic commonality parameter to generate multiple non-rigid global sub-edits in order of editing degree from small to large; the last non-rigid global sub-edit is the maximum editing degree; the semantic difference parameter is used to control the importance of the semantic difference; the semantic commonality parameter is used to control the importance of the semantic commonality.

[0198] Optionally, the editing module 1003, when iterating on each non-rigid global sub-edit in sequence based on the global sparse anchor point and the preset resolution global reconstruction loss function to obtain the global anchor point after non-rigid editing, is specifically used to:

[0199] Determine the sub-image true value of each non-rigid global sub-edit; iteratively optimize the global sparse anchor point based on the preset resolution global reconstruction loss function and the sub-image true value corresponding to the first non-rigid global sub-edit to determine the first non-rigid global anchor point corresponding to the first non-rigid global sub-edit; iteratively optimize the previous non-rigid global anchor point based on the preset resolution global reconstruction loss function and the sub-image true value corresponding to the current non-rigid global sub-edit to determine the current non-rigid global anchor point corresponding to the current non-rigid global sub-edit; if the current non-rigid global sub-edit is the second non-rigid global sub-edit, the first non-rigid global anchor point is the previous non-rigid global anchor point in the second non-rigid global sub-edit.

[0200] Optionally, the editing module 1003, when determining the true value of each non-rigid global sub-edited sub-image, is specifically configured to:

[0201] Based on the semantic difference parameters, semantic commonality parameters and target text prompt information corresponding to each non-rigid global sub-edit, target text embedding data corresponding to each non-rigid global sub-edit is generated; each target text embedding data and the source images rendered by the source three-dimensional model at different perspectives are input into a multi-perspective image editing model that has been trained to convergence to output a target multi-perspective sub-image set corresponding to each non-rigid global sub-edit; the target multi-perspective sub-image set includes target multi-perspective sub-images at different perspectives; each target multi-perspective sub-image is downsampled according to a preset resolution to obtain the sub-image true value of each non-rigid global sub-edit at different perspectives.

[0202] The editing module 1003, when iteratively optimizing the global sparse anchor point based on the preset resolution global reconstruction loss function and the sub-image true value corresponding to the first non-rigid global sub-edit to determine the first non-rigid global anchor point corresponding to the first non-rigid global sub-edit, is specifically used to:

[0203] For the first non-rigid global sub-edit, an edited three-dimensional model corresponding to the first iteration is generated under the first non-rigid global sub-edit based on the global sparse anchor point to obtain the first edited three-dimensional model; the first edited three-dimensional model is iteratively optimized based on a preset resolution global reconstruction loss function to determine the first non-rigid global anchor point corresponding to the first non-rigid global sub-edit.

[0204] Optionally, the editing module 1003, when iteratively optimizing the first edited 3D model based on a preset resolution global reconstruction loss function to determine a first non-rigid global anchor point corresponding to the first non-rigid global sub-edit, is specifically configured to:

[0205] The first edited three-dimensional model is rendered at different perspectives to obtain a first global rendering image set; the first global rendering image set includes the first global rendering images at different perspectives; the first global rendering image set and the true values ​​of the sub-images corresponding to the first non-rigid global sub-edit are input into a preset resolution global reconstruction loss function to obtain a global editing loss value of the first iteration; in response to the global editing loss value of the first iteration not satisfying a preset resolution global loss condition, the first edited three-dimensional model is continuously iteratively optimized on the first non-rigid global sub-edit until the global editing loss value of the edited three-dimensional model corresponding to a certain number of iterations satisfies the preset resolution global loss condition, and the iteratively optimized anchor point corresponding to the edited three-dimensional model that satisfies the preset resolution global loss condition is determined as the first non-rigid global anchor point corresponding to the first non-rigid global sub-edit.

[0206] The three-dimensional model editing device provided in this embodiment also includes: a determination module;

[0207] A determination module is used to determine the global sparse anchor point as the first non-rigid global anchor point corresponding to the first non-rigid global sub-edit in response to the global edit loss value of the first iteration satisfying the preset resolution global loss condition.

[0208] In this embodiment, the determination module is also used for:

[0209] In response to the non-rigid edit being a local edit, a three-dimensional edit box is determined from the source three-dimensional model; a plurality of source Gaussian points are determined from the source Gaussian points included in the three-dimensional edit box as local sparse anchor points; and a target three-dimensional model with a local state change is determined based on the local sparse anchor points.

[0210] The determination module is specifically used for determining the target three-dimensional model with local state changes based on the local sparse anchor points:

[0211] Iteratively optimizing the local sparse anchor points on each non-rigid local sub-edit based on a preset resolution local reconstruction loss function to obtain the local anchor points after non-rigid editing; the editing degree of the latter non-rigid local sub-edit is greater than the editing degree of the former non-rigid local sub-edit;

[0212] Densification is performed based on the local anchor points after non-rigid editing to determine the target 3D model with local state changes.

[0213] Optionally, when the determination module densifies the local anchor points after non-rigid editing to determine the target three-dimensional model with local state changes, it is specifically used to:

[0214] For the first iteration, the local anchor points after non-rigid editing are iteratively densified to obtain a densified three-dimensional model of the first iteration; the local dense loss result of the densified three-dimensional model of the first iteration is determined; in response to the local dense loss result not satisfying the preset local densification loss condition, the local densified Gaussian points corresponding to the previous iteration number are iteratively densified until the densified three-dimensional model of a certain iteration number satisfies the preset local densification loss condition, and the densified three-dimensional model that satisfies the preset local densification loss condition is determined as the target three-dimensional model of the local state change; in response to the local dense loss result satisfying the preset local densification loss condition, the densified three-dimensional model of the first iteration is determined as the target three-dimensional model of the local state change.

[0215] Optionally, when the determination module iteratively densifies the local anchor points after non-rigid editing to obtain a first iterative densified three-dimensional model, it is specifically used to:

[0216] The local anchor points after non-rigid editing are iteratively densified to obtain the local densified Gaussian points of the first iteration; the corresponding densified local model is generated based on the local densified Gaussian points of the first iteration; the densified local model is combined with the model in the non-3D editing box of the source 3D model to obtain the densified 3D model of the first iteration.

[0217] Optionally, the determination module, when determining the local density loss result of the three-dimensional model after the first iteration of densification, is specifically used to:

[0218] The densified three-dimensional model of the first iteration is rendered at different perspectives to obtain a first local rendering image set corresponding to the first iteration; the first local rendering image set includes the first local rendering images at different perspectives; the first local rendering image set and the target multi-view sub-image set are input into a preset local densification reconstruction loss function to output a local densification loss value corresponding to the first iteration; the target multi-view sub-image set is an image set corresponding to the maximum non-rigid local sub-edit, and the maximum non-rigid local sub-edit is a complete non-rigid edit. In response to the local densification loss value satisfying the preset local densification loss condition, the local densification loss result is determined to satisfy the preset local loss condition; in response to the local densification loss value not satisfying the preset local densification loss condition, the local densification loss result is determined to not satisfy the preset local densification loss condition.

[0219] The description of the features in the embodiment corresponding to the three-dimensional model editing device can refer to the relevant description of the embodiment corresponding to the three-dimensional model editing method, which will not be repeated here.

[0220] The embodiment of the present application also provides an editing device, Fig.11 This is a schematic diagram of the structure of the editing device provided in Example 13. Fig.11 As shown, the editing device 1100 includes a processor 1101 and a memory 1102, wherein the processor 1101, the memory 1102 and the communication component 1103 are connected via a bus 1104. The memory 1102 stores a computer program, and the processor 1101 is configured to run the computer program to execute the steps in any of the above three-dimensional model editing method embodiments.

[0221] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any one of the above-mentioned three-dimensional model editing method embodiments when running.

[0222] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0223] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any one of the above-mentioned three-dimensional model editing method embodiments are implemented.

[0224] An embodiment of the present application also provides another computer program product, including a non-volatile computer-readable storage medium, the non-volatile computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the steps in any of the above-mentioned three-dimensional model editing method embodiments are implemented.

[0225] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0226] The above is a detailed introduction to a three-dimensional model editing method provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A three-dimensional model editing method, characterized in that: include: In response to the non-rigid editing being a global editing, obtaining a global sparse anchor point from the global of the source three-dimensional model; The source three-dimensional model includes a target object; the target object is in a source state; the global sparse anchor point is an anchor point that describes the geometric features of the target object; generating a plurality of non-rigid global sub-edits according to the non-rigid edit; the editing degree of a latter non-rigid global sub-edit is greater than the editing degree of a former non-rigid global sub-edit; Iterate on each non-rigid global sub-edit in turn based on the global sparse anchor point and the preset resolution global reconstruction loss function to obtain the global anchor point after non-rigid editing; The global anchor point after non-rigid editing is the anchor point of the target object formed from the source state to the target state after non-rigid editing; Densification of the global anchor points after the non-rigid editing to obtain target global densified Gaussian points describing the appearance features of the target object; The densified three-dimensional model generated based on the target global densified Gaussian points is the target three-dimensional model. After the target object in the target three-dimensional model performs the target action, the target state is formed from the source state; the target three-dimensional model is a global state change.

2. The method according to claim 1, characterized in that The generating of a plurality of non-rigid global sub-edits according to the non-rigid edit comprises: Adjust the size of the semantic difference parameter and the semantic commonality parameter to generate multiple non-rigid global sub-edits in order of editing degree from small to large; the last non-rigid global sub-edit is the maximum editing degree; the semantic difference parameter is used to control the importance of the semantic difference; the semantic commonality parameter is used to control the importance of the semantic commonality.

3. The method according to claim 2, characterized in that The global sparse anchor point and preset resolution global reconstruction loss function are sequentially iterated on each non-rigid global sub-edit to obtain a global anchor point after non-rigid editing, including: Determining the sub-image truth value of each non-rigid global sub-edit; Iteratively optimizing the global sparse anchor point based on a preset resolution global reconstruction loss function and a sub-image true value corresponding to the first non-rigid global sub-edit to determine a first non-rigid global anchor point corresponding to the first non-rigid global sub-edit; The previous non-rigid global anchor point is iteratively optimized based on the preset resolution global reconstruction loss function and the sub-image true value corresponding to the current non-rigid global sub-edit to determine the current non-rigid global anchor point corresponding to the current non-rigid global sub-edit; if the current non-rigid global sub-edit is the second non-rigid global sub-edit, then the first non-rigid global anchor point is the previous non-rigid global anchor point in the second non-rigid global sub-edit.

4. The method according to claim 3, characterized in that Determining the true value of each non-rigid global sub-edited sub-image includes: Generate target text embedding data corresponding to each non-rigid global sub-edit based on the semantic difference parameters, semantic commonality parameters and target text prompt information corresponding to each non-rigid global sub-edit; Input each target text embedding data and source images rendered by the source three-dimensional model at different viewing angles into a multi-view image editing model that has been trained to convergence, so as to output a target multi-view sub-image set corresponding to each non-rigid global sub-edit; the target multi-view sub-image set includes target multi-view sub-images at different viewing angles; Each of the target multi-view sub-images is down-sampled according to a preset resolution to obtain the true value of the sub-image of each non-rigid global sub-edit at different viewpoints.

5. The method according to claim 3, characterized in that: The iterative optimization of the global sparse anchor point based on the preset resolution global reconstruction loss function and the sub-image true value corresponding to the first non-rigid global sub-edit to determine the first non-rigid global anchor point corresponding to the first non-rigid global sub-edit includes: For a first non-rigid global sub-edit, generating an edited three-dimensional model corresponding to a first iteration under the first non-rigid global sub-edit based on the global sparse anchor point to obtain a first edited three-dimensional model; The first edited three-dimensional model is iteratively optimized based on a preset resolution global reconstruction loss function to determine a first non-rigid global anchor point corresponding to the first non-rigid global sub-edit.

6. The method according to claim 5, characterized in that The iterative optimization of the first edited three-dimensional model based on a preset resolution global reconstruction loss function to determine a first non-rigid global anchor point corresponding to the first non-rigid global sub-edit includes: Rendering the first edited three-dimensional model at different viewing angles to obtain a first global rendering image set; the first global rendering image set includes first global rendering images at different viewing angles; Inputting the first global rendered image set and the true value of the sub-image corresponding to the first non-rigid global sub-edit into the preset resolution global reconstruction loss function to obtain a global editing loss value of the first iteration; In response to the global editing loss value of the first iteration not satisfying the preset resolution global loss condition, the first edited three-dimensional model continues to be iteratively optimized on the first non-rigid global sub-edit until the global editing loss value of the edited three-dimensional model corresponding to a certain number of iterations satisfies the preset resolution global loss condition, and the iteratively optimized anchor point corresponding to the edited three-dimensional model that satisfies the preset resolution global loss condition is determined as the first non-rigid global anchor point corresponding to the first non-rigid global sub-edit.

7. The method according to claim 6, characterized in that The method further comprises: In response to the global edit loss value of the first iteration satisfying a preset resolution global loss condition, the global sparse anchor point is determined as a first non-rigid global anchor point corresponding to a first non-rigid global sub-edit.

8. The method according to claim 1, characterized in that The method further comprises: In response to the non-rigid edit being a local edit, determining a three-dimensional edit box from the source three-dimensional model; Determine a plurality of source Gaussian points as local sparse anchor points from the source Gaussian points included in the three-dimensional editing frame; A target three-dimensional model with a local state change is determined based on the local sparse anchor points.

9. The method according to claim 8, characterized in that The target three-dimensional model of local state change determined based on the local sparse anchor point includes: Iteratively optimizing the local sparse anchor points on each non-rigid local sub-edit based on a preset resolution local reconstruction loss function to obtain a non-rigidly edited local anchor point; the editing degree of the latter non-rigid local sub-edit is greater than the editing degree of the former non-rigid local sub-edit; Densification is performed based on the non-rigidly edited local anchor points to determine a target three-dimensional model with a local state change.

10. The method according to claim 9, characterized in that The densification based on the local anchor points after the non-rigid editing to determine the target three-dimensional model with local state changes includes: For the first iteration, iteratively densify the local anchor points after the non-rigid editing to obtain a densified three-dimensional model of the first iteration; Determining a local density loss result of the densified three-dimensional model of the first iteration; In response to the local densification loss result not satisfying the preset local densification loss condition, iteratively densifying the local densified Gaussian points corresponding to the previous iteration number until the densified three-dimensional model of a certain iteration number satisfies the preset local densification loss condition, and determining the densified three-dimensional model satisfying the preset local densification loss condition as the target three-dimensional model of the local state change; In response to the local densification loss result satisfying a preset local densification loss condition, the densified three-dimensional model of the first iteration is determined as a target three-dimensional model of a local state change.

11. The method according to claim 10, characterized in that The iterative densification of the non-rigidly edited local anchor points to obtain a first iterative densified three-dimensional model includes: Iteratively densifying the non-rigidly edited local anchor points to obtain locally densified Gaussian points of the first iteration; Generating a corresponding densified local model based on the local densified Gaussian points of the first iteration; The densified local model is combined with the model in the non-3D editing frame of the source 3D model to obtain a densified 3D model of the first iteration.

12. The method according to claim 10, characterized in that The determining of the local density loss result of the three-dimensional model after the first iteration of densification includes: Rendering the densified three-dimensional model of the first iteration at different viewing angles to obtain a first local rendering image set corresponding to the first iteration; the first local rendering image set includes first local rendering images at different viewing angles; Inputting the first local rendering image set and the target multi-view sub-image set into a preset local densification reconstruction loss function to output a local densification loss value corresponding to the first iteration; the target multi-view sub-image set is an image set corresponding to a maximum non-rigid local sub-edit, and the maximum non-rigid local sub-edit is a complete non-rigid edit; In response to the local densification loss value satisfying a preset local densification loss condition, determining that the local densification loss result satisfies a preset local loss condition; In response to the local densification loss value not satisfying a preset local densification loss condition, determining the local densification loss result as not satisfying the preset local densification loss condition.

13. An editing device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the three-dimensional model editing method according to any one of claims 1 to 12 when executing the computer program.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the three-dimensional model editing method according to any one of claims 1 to 12.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the three-dimensional model editing method according to any one of claims 1 to 12 are implemented.

Images